[Claude][Azure Functions] Add dev/test workflow skill and scripts#8173
[Claude][Azure Functions] Add dev/test workflow skill and scripts#8173lucaspimentel merged 36 commits intomasterfrom
Conversation
cb474c2 to
c7dc6f5
Compare
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (8173) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8173) - mean (77ms) : 74, 79
master - mean (77ms) : 74, 79
section Bailout
This PR (8173) - mean (81ms) : 79, 83
master - mean (81ms) : 79, 83
section CallTarget+Inlining+NGEN
This PR (8173) - mean (1,100ms) : 1053, 1147
master - mean (1,101ms) : 1052, 1150
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8173) - mean (119ms) : 115, 123
master - mean (119ms) : 116, 122
section Bailout
This PR (8173) - mean (121ms) : 118, 125
master - mean (120ms) : 118, 123
section CallTarget+Inlining+NGEN
This PR (8173) - mean (783ms) : 716, 850
master - mean (786ms) : 717, 854
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8173) - mean (108ms) : 105, 111
master - mean (107ms) : 103, 110
section Bailout
This PR (8173) - mean (108ms) : 105, 111
master - mean (108ms) : 104, 112
section CallTarget+Inlining+NGEN
This PR (8173) - mean (773ms) : 698, 848
master - mean (766ms) : 686, 847
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8173) - mean (106ms) : 103, 108
master - mean (105ms) : 102, 107
section Bailout
This PR (8173) - mean (106ms) : 103, 108
master - mean (107ms) : 105, 110
section CallTarget+Inlining+NGEN
This PR (8173) - mean (690ms) : 668, 711
master - mean (690ms) : 665, 715
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8173) - mean (201ms) : 192, 210
master - mean (199ms) : 191, 206
section Bailout
This PR (8173) - mean (207ms) : 198, 216
master - mean (202ms) : 198, 207
section CallTarget+Inlining+NGEN
This PR (8173) - mean (1,183ms) : 1114, 1253
master - mean (1,169ms) : 1106, 1232
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8173) - mean (291ms) : 280, 302
master - mean (305ms) : 290, 320
section Bailout
This PR (8173) - mean (290ms) : 280, 301
master - mean (297ms) : 282, 311
section CallTarget+Inlining+NGEN
This PR (8173) - mean (961ms) : 909, 1013
master - mean (977ms) : 924, 1030
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8173) - mean (282ms) : 267, 297
master - mean (290ms) : 278, 303
section Bailout
This PR (8173) - mean (288ms) : 270, 306
master - mean (291ms) : 279, 303
section CallTarget+Inlining+NGEN
This PR (8173) - mean (958ms) : 918, 999
master - mean (963ms) : 913, 1013
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (8173) - mean (278ms) : 266, 290
master - mean (290ms) : 274, 306
section Bailout
This PR (8173) - mean (279ms) : 268, 289
master - mean (284ms) : 269, 299
section CallTarget+Inlining+NGEN
This PR (8173) - mean (863ms) : 815, 910
master - mean (866ms) : 831, 901
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
ec003e5 to
a3f103f
Compare
dd7b807 to
e998fb1
Compare
e998fb1 to
cd3e78c
Compare
5e318e6 to
5ad4c4d
Compare
ebaa387 to
b3e14a6
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: bb470e5339
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
This comment was marked as off-topic.
This comment was marked as off-topic.
d40d96d to
eb591f1
Compare
Use SemVer-compliant version without dots in prerelease: - Old: 3.38.0-dev.20260209.143022 - New: 3.38.0-dev20260209143022 Fixes MSB4018 error with stricter version validation. 🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
Check for required native/managed files when -BuildId not provided. Fails with helpful error message if files are missing. 🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
Specify -OutputPath $env:TEMP in Get-AzureFunctionLogs.ps1 examples to save downloaded logs to system temp folder instead of cluttering the repository directory. 🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
- Add environment-variables.md with complete reference for Azure Functions configuration - Document required, recommended, and debugging variables - Add Windows-specific paths (C:\home\site\wwwroot vs /home/site/wwwroot) - Add /azure-functions configure command to SKILL.md - Include platform detection and interactive configuration workflow - Update README.md and SKILL.md with references to new guide 🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
Added PowerShell script to search for nuget.config up the directory hierarchy. Updated skill documentation to reference the script in deployment workflow and scripts reference. 🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
- Rename build command to build-nuget everywhere - Add missing allowed-tools (pwsh, publish, restart, webapp log download) - Fix case-sensitive log path /home/LogFiles/Datadog -> /home/LogFiles/datadog - Fix wrong CLI command az functionapp logs download -> az webapp log download - Update .csproj step to verify state before modifying - Fix net6.0+net461 docs (only net6.0 is built) - Add missing -OutputPath $env:TEMP in pipeline example - Remove references to non-existent scripts 🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
- Reframe skill.md/README.md as dev/test workflow for tracer engineers - Add test command (trigger discovery + timestamp capture) - Add trace command with Get-DatadogTrace.ps1/Get-DatadogLogs.ps1 - Fix wrong log file patterns in environment-variables.md - Fix "Profiler DLL" -> native profiler library (.so/.dll) - Remove production-oriented sections (perf tuning, agent config) - Rewrite Best Practices as dev/test Tips - Trim scripts-reference.md: remove bash duplicates, add PS script docs - Trim README.md PowerShell install section - Add intro to log-analysis-guide.md for manual investigation context 🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
Update skill documentation to ensure DD_API_KEY is never retrieved, only checked for existence. Replace grep/list commands with Azure CLI queries that return only the key name, not the value. Changes: - SKILL.md: Update troubleshooting and configure sections - environment-variables.md: Add safe verification example - scripts-reference.md: Add existence-check command 🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
Use Azure CLI --query to exclude DD_API_KEY at the source instead of fetching then redacting. The API key value never leaves Azure. 🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
Extract env var configuration logic from SKILL.md prose into a reusable PowerShell script with SupportsShouldProcess. Update SKILL.md and scripts-reference.md to use it. 🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
313496b to
3e7e971
Compare
BenchmarksBenchmark execution time: 2026-02-24 19:55:52 Comparing candidate commit 3e7e971 in PR branch Found 13 performance improvements and 10 performance regressions! Performance is the same for 154 metrics, 15 unstable metrics. scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces net6.0
scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody net6.0
scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody netcoreapp3.1
scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs net6.0
scenario:Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack net6.0
scenario:Benchmarks.Trace.AspNetCoreBenchmark.SendRequest net6.0
scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces net6.0
scenario:Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces netcoreapp3.1
scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice net6.0
scenario:Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice net472
scenario:Benchmarks.Trace.CharSliceBenchmark.OriginalCharSlice net6.0
scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark net6.0
scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog netcoreapp3.1
scenario:Benchmarks.Trace.RedisBenchmark.SendReceive netcoreapp3.1
scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore netcoreapp3.1
scenario:Benchmarks.Trace.SpanBenchmark.StartFinishScope netcoreapp3.1
scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan netcoreapp3.1
|
andrewlock
left a comment
There was a problem hiding this comment.
If it works for you, why not 😄
Summary of changes
Adds a Claude Code skill (
/azure-functions) and standalone PowerShell scripts to automate the Azure Functions development workflow: build a localDatadog.AzureFunctionsNuGet package, deploy to a test Azure Function App, trigger the function, and analyze traces/logs to verify instrumentation behavior.Skill (
.claude/skills/azure-functions/):SKILL.md— Interactive workflow with build-nuget, configure, deploy, test, trace, and analyze commandsREADME.md— User-facing documentationFind-NuGetConfig.ps1— Locatesnuget.configby searching up directory hierarchyTest-EnvVars.ps1— Verifies required/recommended Datadog env vars on a Function App (checks app state, detects platform, never exposesDD_API_KEY)environment-variables.md— Complete reference for Azure Functions environment variable configurationlog-analysis-guide.md— Log analysis patterns (grep commands, timestamp filtering, span tree construction)scripts-reference.md— Script documentation and reusable one-linersScripts (
tracer/tools/):Build-AzureFunctionsNuget.ps1— Builds NuGet package with auto-incrementing timestamp-based prerelease versions; validates bundle files; supports-BuildId(download release artifacts) and-CopyTo(local feed)Deploy-AzureFunction.ps1— Automates dotnet restore,func azure functionapp publish, wait, trigger, and timestamp captureGet-AzureFunctionLogs.ps1— Downloads logs viaaz webapp log download, extracts, analyzes tracer version, counts spans (filters noisy host spans), validates parentingBuild change (
Build.cs):--Versionthrough toBuildAzureFunctionsNugetNuke targetReason for change
Working with Azure Functions instrumentation requires coordinating multiple manual steps across different tools (build scripts, Azure CLI,
funcCLI, log extraction, grep for spans/errors). The versioning workflow was error-prone — rebuilding with the same version required manually clearing NuGet caches.This consolidates the workflow into reusable scripts and a guided skill, switches to auto-incrementing versions (each build is unique), and adds comprehensive validation/troubleshooting.
Implementation details
Build-AzureFunctionsNuget.ps1generates SemVer-compliant versions like3.38.0-dev20260209143022from base version + UTC timestamp. Sample apps use floating versions (3.38.0-dev*) to auto-resolve latest local builds.Test-EnvVars.ps1checks app state, detects platform (Linux/Windows), validates required vars (profiling, API key, tracer home, startup hooks, profiler paths) and recommended vars (feature disables).DD_API_KEYis checked for existence only — never retrieved or displayed.Deploy-AzureFunction.ps1outputs aPSCustomObjectthat pipes directly intoGet-AzureFunctionLogs.ps1for seamless deploy → trigger → analyze workflows..Counterrors, improves tracer version extraction.--query "[?name=='DD_API_KEY'].name"— never fetching or displaying the actual value.Test coverage
Development tooling (not production code). Manually verified:
BuildIdparameter downloads bundle files from CI releasesCopyToparameter copies to local NuGet feed-OutputPath $env:TEMPFind-NuGetConfig.ps1directory hierarchy searchTest-EnvVars.ps1against running Linux Function App (all required vars pass, recommended warns on missing)Other details
Internal development tooling engineering teams. Not included in any customer-facing artifacts.