[Debugger-Test Optimization] Add agent-less support to ER#7859
[Debugger-Test Optimization] Add agent-less support to ER#7859tonyredondo merged 22 commits intomasterfrom
Conversation
This comment has been minimized.
This comment has been minimized.
BenchmarksBenchmarks Report for benchmark platform 🐌Benchmarks for #7859 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ Same allocations ✔️Raw results
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ More allocations
|
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 | 3.26 KB | 3.39 KB | 128 B | 3.92% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | WriteAndFlushEnrichedTraces |
net6.0 | 1.27ms | 176ns | 683ns | 0 | 0 | 0 | 2.7 KB |
| master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 1.39ms | 69.4ns | 240ns | 0 | 0 | 0 | 2.7 KB |
| master | WriteAndFlushEnrichedTraces |
net472 | 1.75ms | 1.67μs | 6.46μs | 0 | 0 | 0 | 3.26 KB |
| #7859 | WriteAndFlushEnrichedTraces |
net6.0 | 1.25ms | 106ns | 366ns | 0 | 0 | 0 | 2.7 KB |
| #7859 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 1.38ms | 1.52μs | 5.87μs | 0 | 0 | 0 | 2.7 KB |
| #7859 | WriteAndFlushEnrichedTraces |
net472 | 1.72ms | 2.23μs | 8.65μs | 0 | 0 | 0 | 3.39 KB |
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | AllCycleSimpleBody |
net6.0 | 1.08μs | 4.95ns | 19.2ns | 0 | 0 | 0 | 1.22 KB |
| master | AllCycleSimpleBody |
netcoreapp3.1 | 1.42μs | 7.39ns | 38.4ns | 0 | 0 | 0 | 1.2 KB |
| master | AllCycleSimpleBody |
net472 | 1.03μs | 0.369ns | 1.43ns | 0.19 | 0 | 0 | 1.23 KB |
| master | AllCycleMoreComplexBody |
net6.0 | 7.34μs | 35.4ns | 137ns | 0 | 0 | 0 | 4.72 KB |
| master | AllCycleMoreComplexBody |
netcoreapp3.1 | 9.07μs | 43.7ns | 185ns | 0 | 0 | 0 | 4.62 KB |
| master | AllCycleMoreComplexBody |
net472 | 7.61μs | 4.07ns | 15.8ns | 0.724 | 0 | 0 | 4.74 KB |
| master | ObjectExtractorSimpleBody |
net6.0 | 328ns | 1.79ns | 10.4ns | 0 | 0 | 0 | 280 B |
| master | ObjectExtractorSimpleBody |
netcoreapp3.1 | 402ns | 2.14ns | 10.5ns | 0 | 0 | 0 | 272 B |
| master | ObjectExtractorSimpleBody |
net472 | 296ns | 0.0491ns | 0.19ns | 0.0432 | 0 | 0 | 281 B |
| master | ObjectExtractorMoreComplexBody |
net6.0 | 6.33μs | 33.6ns | 175ns | 0 | 0 | 0 | 3.78 KB |
| master | ObjectExtractorMoreComplexBody |
netcoreapp3.1 | 7.81μs | 40.9ns | 192ns | 0 | 0 | 0 | 3.69 KB |
| master | ObjectExtractorMoreComplexBody |
net472 | 6.71μs | 2.46ns | 9.51ns | 0.572 | 0 | 0 | 3.8 KB |
| #7859 | AllCycleSimpleBody |
net6.0 | 1.09μs | 5.32ns | 22.6ns | 0 | 0 | 0 | 1.22 KB |
| #7859 | AllCycleSimpleBody |
netcoreapp3.1 | 1.43μs | 0.952ns | 3.69ns | 0 | 0 | 0 | 1.2 KB |
| #7859 | AllCycleSimpleBody |
net472 | 1.01μs | 0.515ns | 1.99ns | 0.193 | 0 | 0 | 1.23 KB |
| #7859 | AllCycleMoreComplexBody |
net6.0 | 7.38μs | 5.69ns | 22ns | 0 | 0 | 0 | 4.72 KB |
| #7859 | AllCycleMoreComplexBody |
netcoreapp3.1 | 8.92μs | 44.5ns | 199ns | 0 | 0 | 0 | 4.62 KB |
| #7859 | AllCycleMoreComplexBody |
net472 | 7.56μs | 3.4ns | 13.2ns | 0.72 | 0 | 0 | 4.74 KB |
| #7859 | ObjectExtractorSimpleBody |
net6.0 | 322ns | 1.8ns | 10.8ns | 0 | 0 | 0 | 280 B |
| #7859 | ObjectExtractorSimpleBody |
netcoreapp3.1 | 395ns | 2.19ns | 13ns | 0 | 0 | 0 | 272 B |
| #7859 | ObjectExtractorSimpleBody |
net472 | 303ns | 1.62ns | 8.59ns | 0.0432 | 0 | 0 | 281 B |
| #7859 | ObjectExtractorMoreComplexBody |
net6.0 | 6.42μs | 19ns | 71.2ns | 0 | 0 | 0 | 3.78 KB |
| #7859 | ObjectExtractorMoreComplexBody |
netcoreapp3.1 | 8.11μs | 38.3ns | 153ns | 0 | 0 | 0 | 3.69 KB |
| #7859 | ObjectExtractorMoreComplexBody |
net472 | 6.73μs | 2.42ns | 9.36ns | 0.572 | 0 | 0 | 3.8 KB |
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EncodeArgs |
net6.0 | 77.1μs | 41.5ns | 155ns | 0 | 0 | 0 | 32.4 KB |
| master | EncodeArgs |
netcoreapp3.1 | 98.1μs | 363ns | 1.41μs | 0 | 0 | 0 | 32.4 KB |
| master | EncodeArgs |
net472 | 109μs | 29.9ns | 116ns | 4.9 | 0 | 0 | 32.5 KB |
| master | EncodeLegacyArgs |
net6.0 | 143μs | 44ns | 171ns | 0 | 0 | 0 | 2.14 KB |
| master | EncodeLegacyArgs |
netcoreapp3.1 | 197μs | 26ns | 93.6ns | 0 | 0 | 0 | 2.14 KB |
| master | EncodeLegacyArgs |
net472 | 261μs | 18.9ns | 73.3ns | 0 | 0 | 0 | 2.14 KB |
| #7859 | EncodeArgs |
net6.0 | 76.6μs | 227ns | 879ns | 0 | 0 | 0 | 32.4 KB |
| #7859 | EncodeArgs |
netcoreapp3.1 | 98μs | 211ns | 818ns | 0 | 0 | 0 | 32.4 KB |
| #7859 | EncodeArgs |
net472 | 108μs | 55.2ns | 214ns | 4.88 | 0 | 0 | 32.5 KB |
| #7859 | EncodeLegacyArgs |
net6.0 | 146μs | 13.2ns | 49.3ns | 0 | 0 | 0 | 2.14 KB |
| #7859 | EncodeLegacyArgs |
netcoreapp3.1 | 200μs | 189ns | 730ns | 0 | 0 | 0 | 2.14 KB |
| #7859 | EncodeLegacyArgs |
net472 | 264μs | 68.7ns | 266ns | 0 | 0 | 0 | 2.14 KB |
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunWafRealisticBenchmark |
net6.0 | 430μs | 1.21μs | 4.36μs | 0 | 0 | 0 | 4.55 KB |
| master | RunWafRealisticBenchmark |
netcoreapp3.1 | 446μs | 1.62μs | 6.07μs | 0 | 0 | 0 | 4.48 KB |
| master | RunWafRealisticBenchmark |
net472 | 502μs | 453ns | 1.63μs | 0 | 0 | 0 | 0 b |
| master | RunWafRealisticBenchmarkWithAttack |
net6.0 | 320μs | 1.08μs | 3.9μs | 0 | 0 | 0 | 2.24 KB |
| master | RunWafRealisticBenchmarkWithAttack |
netcoreapp3.1 | 350μs | 3.6μs | 34.4μs | 0 | 0 | 0 | 2.22 KB |
| master | RunWafRealisticBenchmarkWithAttack |
net472 | 378μs | 668ns | 2.59μs | 0 | 0 | 0 | 0 b |
| #7859 | RunWafRealisticBenchmark |
net6.0 | 424μs | 1.32μs | 5.28μs | 0 | 0 | 0 | 4.55 KB |
| #7859 | RunWafRealisticBenchmark |
netcoreapp3.1 | 448μs | 1.01μs | 3.5μs | 0 | 0 | 0 | 4.48 KB |
| #7859 | RunWafRealisticBenchmark |
net472 | 498μs | 486ns | 1.75μs | 0 | 0 | 0 | 0 b |
| #7859 | RunWafRealisticBenchmarkWithAttack |
net6.0 | 309μs | 715ns | 2.48μs | 0 | 0 | 0 | 2.24 KB |
| #7859 | RunWafRealisticBenchmarkWithAttack |
netcoreapp3.1 | 345μs | 2.57μs | 24.4μs | 0 | 0 | 0 | 2.22 KB |
| #7859 | RunWafRealisticBenchmarkWithAttack |
net472 | 373μs | 167ns | 579ns | 0 | 0 | 0 | 0 b |
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendRequest |
net6.0 | 84.7μs | 221ns | 856ns | 0 | 0 | 0 | 19.05 KB |
| master | SendRequest |
netcoreapp3.1 | 96.7μs | 248ns | 1.51μs | 0 | 0 | 0 | 21.18 KB |
| master | SendRequest |
net472 | 0ns | 0ns | 0ns | 0 | 0 | 0 | 0 b |
| #7859 | SendRequest |
net6.0 | 85.1μs | 227ns | 848ns | 0 | 0 | 0 | 18.97 KB |
| #7859 | SendRequest |
netcoreapp3.1 | 97.7μs | 363ns | 1.41μs | 0 | 0 | 0 | 21.18 KB |
| #7859 | SendRequest |
net472 | 0ns | 0ns | 0ns | 0 | 0 | 0 | 0 b |
Benchmarks.Trace.CharSliceBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | OriginalCharSlice |
net6.0 | 1.97ms | 1.51μs | 5.86μs | 0 | 0 | 0 | 640 KB |
| master | OriginalCharSlice |
netcoreapp3.1 | 3.95ms | 710ns | 2.75μs | 0 | 0 | 0 | 640.05 KB |
| master | OriginalCharSlice |
net472 | 2.64ms | 559ns | 2.09μs | 0 | 0 | 0 | 638.98 KB |
| master | OptimizedCharSlice |
net6.0 | 1.44ms | 954ns | 3.57μs | 0 | 0 | 0 | 0 b |
| master | OptimizedCharSlice |
netcoreapp3.1 | 2.73ms | 914ns | 3.42μs | 0 | 0 | 0 | 0 b |
| master | OptimizedCharSlice |
net472 | 1.96ms | 920ns | 3.56μs | 0 | 0 | 0 | 0 b |
| master | OptimizedCharSliceWithPool |
net6.0 | 1.01ms | 761ns | 2.85μs | 0 | 0 | 0 | 0 b |
| master | OptimizedCharSliceWithPool |
netcoreapp3.1 | 1.86ms | 1.28μs | 4.97μs | 0 | 0 | 0 | 0 b |
| master | OptimizedCharSliceWithPool |
net472 | 1.17ms | 576ns | 2.23μs | 0 | 0 | 0 | 0 b |
| #7859 | OriginalCharSlice |
net6.0 | 1.9ms | 1.48μs | 5.74μs | 0 | 0 | 0 | 640 KB |
| #7859 | OriginalCharSlice |
netcoreapp3.1 | 3.87ms | 6μs | 22.4μs | 0 | 0 | 0 | 640 KB |
| #7859 | OriginalCharSlice |
net472 | 2.77ms | 977ns | 3.78μs | 0 | 0 | 0 | 638.98 KB |
| #7859 | OptimizedCharSlice |
net6.0 | 1.46ms | 484ns | 1.81μs | 0 | 0 | 0 | 0 b |
| #7859 | OptimizedCharSlice |
netcoreapp3.1 | 2.73ms | 896ns | 3.47μs | 0 | 0 | 0 | 0 b |
| #7859 | OptimizedCharSlice |
net472 | 1.99ms | 949ns | 3.68μs | 0 | 0 | 0 | 0 b |
| #7859 | OptimizedCharSliceWithPool |
net6.0 | 1.09ms | 800ns | 2.99μs | 0 | 0 | 0 | 0 b |
| #7859 | OptimizedCharSliceWithPool |
netcoreapp3.1 | 1.87ms | 1.99μs | 7.69μs | 0 | 0 | 0 | 0 b |
| #7859 | OptimizedCharSliceWithPool |
net472 | 1.19ms | 1.84μs | 7.12μs | 0 | 0 | 0 | 0 b |
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️
More allocations ⚠️ in #7859
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472
55.87 KB
56.15 KB
280 B
0.50%
Fewer allocations 🎉 in #7859
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1
42.27 KB
41.93 KB
-344 B
-0.81%
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 | 55.87 KB | 56.15 KB | 280 B | 0.50% |
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 | 42.27 KB | 41.93 KB | -344 B | -0.81% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | WriteAndFlushEnrichedTraces |
net6.0 | 709μs | 3.12μs | 11.2μs | 0 | 0 | 0 | 41.71 KB |
| master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 665μs | 3.56μs | 18.5μs | 0 | 0 | 0 | 42.27 KB |
| master | WriteAndFlushEnrichedTraces |
net472 | 963μs | 4.58μs | 17.7μs | 4.46 | 0 | 0 | 55.87 KB |
| #7859 | WriteAndFlushEnrichedTraces |
net6.0 | 708μs | 3.87μs | 21.9μs | 0 | 0 | 0 | 41.64 KB |
| #7859 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 713μs | 3.68μs | 18μs | 0 | 0 | 0 | 41.93 KB |
| #7859 | WriteAndFlushEnrichedTraces |
net472 | 881μs | 2.46μs | 9.54μs | 8.33 | 4.17 | 0 | 56.15 KB |
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteNonQuery |
net6.0 | 1.91μs | 1.99ns | 7.69ns | 0 | 0 | 0 | 968 B |
| master | ExecuteNonQuery |
netcoreapp3.1 | 2.62μs | 8.36ns | 32.4ns | 0 | 0 | 0 | 960 B |
| master | ExecuteNonQuery |
net472 | 2.79μs | 4.28ns | 16.6ns | 0.138 | 0 | 0 | 931 B |
| #7859 | ExecuteNonQuery |
net6.0 | 1.89μs | 6.4ns | 24.8ns | 0 | 0 | 0 | 968 B |
| #7859 | ExecuteNonQuery |
netcoreapp3.1 | 2.48μs | 9.14ns | 35.4ns | 0 | 0 | 0 | 960 B |
| #7859 | ExecuteNonQuery |
net472 | 2.79μs | 2.22ns | 8.29ns | 0.138 | 0 | 0 | 931 B |
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | CallElasticsearch |
net6.0 | 1.67μs | 2.1ns | 7.85ns | 0 | 0 | 0 | 952 B |
| master | CallElasticsearch |
netcoreapp3.1 | 2.25μs | 6.14ns | 23ns | 0 | 0 | 0 | 968 B |
| master | CallElasticsearch |
net472 | 3.26μs | 4ns | 15.5ns | 0.146 | 0 | 0 | 955 B |
| master | CallElasticsearchAsync |
net6.0 | 1.84μs | 8.76ns | 33.9ns | 0 | 0 | 0 | 928 B |
| master | CallElasticsearchAsync |
netcoreapp3.1 | 2.46μs | 3.28ns | 12.7ns | 0 | 0 | 0 | 1.02 KB |
| master | CallElasticsearchAsync |
net472 | 3.44μs | 1.65ns | 6.39ns | 0.155 | 0 | 0 | 1.01 KB |
| #7859 | CallElasticsearch |
net6.0 | 1.71μs | 7.85ns | 31.4ns | 0 | 0 | 0 | 952 B |
| #7859 | CallElasticsearch |
netcoreapp3.1 | 2.24μs | 10.7ns | 41.6ns | 0 | 0 | 0 | 968 B |
| #7859 | CallElasticsearch |
net472 | 3.29μs | 6.12ns | 23.7ns | 0.147 | 0 | 0 | 955 B |
| #7859 | CallElasticsearchAsync |
net6.0 | 1.79μs | 1.16ns | 4.51ns | 0 | 0 | 0 | 928 B |
| #7859 | CallElasticsearchAsync |
netcoreapp3.1 | 2.36μs | 11ns | 43.9ns | 0 | 0 | 0 | 1.02 KB |
| #7859 | CallElasticsearchAsync |
net472 | 3.47μs | 4.11ns | 15.9ns | 0.155 | 0 | 0 | 1.01 KB |
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteAsync |
net6.0 | 1.92μs | 0.392ns | 1.52ns | 0 | 0 | 0 | 896 B |
| master | ExecuteAsync |
netcoreapp3.1 | 2.43μs | 2.31ns | 8.34ns | 0 | 0 | 0 | 896 B |
| master | ExecuteAsync |
net472 | 2.66μs | 2.86ns | 11.1ns | 0.134 | 0 | 0 | 858 B |
| #7859 | ExecuteAsync |
net6.0 | 1.81μs | 10ns | 34.8ns | 0 | 0 | 0 | 896 B |
| #7859 | ExecuteAsync |
netcoreapp3.1 | 2.33μs | 7.57ns | 29.3ns | 0 | 0 | 0 | 896 B |
| #7859 | ExecuteAsync |
net472 | 2.61μs | 4.96ns | 19.2ns | 0.129 | 0 | 0 | 859 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendAsync |
net6.0 | 6.91μs | 17ns | 65.9ns | 0 | 0 | 0 | 2.29 KB |
| master | SendAsync |
netcoreapp3.1 | 8.45μs | 24.5ns | 94.9ns | 0 | 0 | 0 | 2.83 KB |
| master | SendAsync |
net472 | 12.1μs | 8.58ns | 33.2ns | 0.481 | 0 | 0 | 3.08 KB |
| #7859 | SendAsync |
net6.0 | 7.04μs | 14.5ns | 56.2ns | 0 | 0 | 0 | 2.29 KB |
| #7859 | SendAsync |
netcoreapp3.1 | 8.31μs | 32.1ns | 124ns | 0 | 0 | 0 | 2.83 KB |
| #7859 | SendAsync |
net472 | 12.2μs | 5.84ns | 21.1ns | 0.489 | 0 | 0 | 3.08 KB |
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️
More allocations ⚠️ in #7859
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0
274.06 KB
278.34 KB
4.28 KB
1.56%
Fewer allocations 🎉 in #7859
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1
273.49 KB
257.43 KB
-16.06 KB
-5.87%
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 | 274.06 KB | 278.34 KB | 4.28 KB | 1.56% |
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 | 273.49 KB | 257.43 KB | -16.06 KB | -5.87% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StringConcatBenchmark |
net6.0 | 44.5μs | 245ns | 1.76μs | 0 | 0 | 0 | 42.51 KB |
| master | StringConcatBenchmark |
netcoreapp3.1 | 48.9μs | 258ns | 1.26μs | 0 | 0 | 0 | 42.54 KB |
| master | StringConcatBenchmark |
net472 | 57μs | 259ns | 1.03μs | 0 | 0 | 0 | 49.15 KB |
| master | StringConcatAspectBenchmark |
net6.0 | 488μs | 2.5μs | 11.4μs | 0 | 0 | 0 | 274.06 KB |
| master | StringConcatAspectBenchmark |
netcoreapp3.1 | 513μs | 1.79μs | 8.76μs | 0 | 0 | 0 | 273.49 KB |
| master | StringConcatAspectBenchmark |
net472 | 410μs | 2.13μs | 12.1μs | 0 | 0 | 0 | 270.34 KB |
| #7859 | StringConcatBenchmark |
net6.0 | 43.6μs | 204ns | 761ns | 0 | 0 | 0 | 42.51 KB |
| #7859 | StringConcatBenchmark |
netcoreapp3.1 | 49.3μs | 273ns | 1.57μs | 0 | 0 | 0 | 42.54 KB |
| #7859 | StringConcatBenchmark |
net472 | 56.9μs | 273ns | 1.06μs | 0 | 0 | 0 | 49.15 KB |
| #7859 | StringConcatAspectBenchmark |
net6.0 | 491μs | 2.38μs | 9.51μs | 0 | 0 | 0 | 278.34 KB |
| #7859 | StringConcatAspectBenchmark |
netcoreapp3.1 | 491μs | 1.75μs | 10μs | 0 | 0 | 0 | 257.43 KB |
| #7859 | StringConcatAspectBenchmark |
net472 | 409μs | 2.04μs | 8.9μs | 0 | 0 | 0 | 270.34 KB |
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 2.77μs | 14.4ns | 52ns | 0 | 0 | 0 | 1.69 KB |
| master | EnrichedLog |
netcoreapp3.1 | 3.55μs | 12.5ns | 48.5ns | 0 | 0 | 0 | 1.7 KB |
| master | EnrichedLog |
net472 | 3.84μs | 4.36ns | 16.9ns | 0.25 | 0 | 0 | 1.6 KB |
| #7859 | EnrichedLog |
net6.0 | 2.75μs | 0.619ns | 2.32ns | 0 | 0 | 0 | 1.69 KB |
| #7859 | EnrichedLog |
netcoreapp3.1 | 3.5μs | 15.4ns | 59.7ns | 0 | 0 | 0 | 1.7 KB |
| #7859 | EnrichedLog |
net472 | 3.85μs | 3.93ns | 15.2ns | 0.249 | 0 | 0 | 1.6 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 124μs | 105ns | 393ns | 0 | 0 | 0 | 4.31 KB |
| master | EnrichedLog |
netcoreapp3.1 | 128μs | 68.2ns | 255ns | 0 | 0 | 0 | 4.31 KB |
| master | EnrichedLog |
net472 | 169μs | 145ns | 562ns | 0 | 0 | 0 | 4.51 KB |
| #7859 | EnrichedLog |
net6.0 | 123μs | 70.5ns | 264ns | 0 | 0 | 0 | 4.31 KB |
| #7859 | EnrichedLog |
netcoreapp3.1 | 128μs | 131ns | 471ns | 0 | 0 | 0 | 4.31 KB |
| #7859 | EnrichedLog |
net472 | 168μs | 127ns | 493ns | 0 | 0 | 0 | 4.51 KB |
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 5.11μs | 24.6ns | 95.4ns | 0 | 0 | 0 | 2.24 KB |
| master | EnrichedLog |
netcoreapp3.1 | 6.87μs | 24.7ns | 95.8ns | 0 | 0 | 0 | 2.26 KB |
| master | EnrichedLog |
net472 | 7.7μs | 6.88ns | 26.6ns | 0.307 | 0 | 0 | 2.05 KB |
| #7859 | EnrichedLog |
net6.0 | 5.22μs | 16.6ns | 64.2ns | 0 | 0 | 0 | 2.24 KB |
| #7859 | EnrichedLog |
netcoreapp3.1 | 6.72μs | 17.3ns | 67ns | 0 | 0 | 0 | 2.26 KB |
| #7859 | EnrichedLog |
net472 | 7.71μs | 5.08ns | 19ns | 0.308 | 0 | 0 | 2.05 KB |
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendReceive |
net6.0 | 1.89μs | 1.9ns | 7.35ns | 0 | 0 | 0 | 1.12 KB |
| master | SendReceive |
netcoreapp3.1 | 2.43μs | 12.8ns | 61.3ns | 0 | 0 | 0 | 1.14 KB |
| master | SendReceive |
net472 | 2.88μs | 1.12ns | 4.34ns | 0.175 | 0 | 0 | 1.12 KB |
| #7859 | SendReceive |
net6.0 | 1.91μs | 8.99ns | 34.8ns | 0 | 0 | 0 | 1.12 KB |
| #7859 | SendReceive |
netcoreapp3.1 | 2.52μs | 7.97ns | 30.9ns | 0 | 0 | 0 | 1.14 KB |
| #7859 | SendReceive |
net472 | 2.96μs | 1.09ns | 3.91ns | 0.176 | 0 | 0 | 1.12 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 4.39μs | 12.2ns | 45.7ns | 0 | 0 | 0 | 1.58 KB |
| master | EnrichedLog |
netcoreapp3.1 | 5.92μs | 6.95ns | 26.9ns | 0 | 0 | 0 | 1.63 KB |
| master | EnrichedLog |
net472 | 6.58μs | 12.5ns | 48.3ns | 0.295 | 0 | 0 | 2.03 KB |
| #7859 | EnrichedLog |
net6.0 | 4.46μs | 4.57ns | 17.7ns | 0 | 0 | 0 | 1.58 KB |
| #7859 | EnrichedLog |
netcoreapp3.1 | 5.52μs | 10.8ns | 41.7ns | 0 | 0 | 0 | 1.63 KB |
| #7859 | EnrichedLog |
net472 | 6.62μs | 12.3ns | 47.6ns | 0.298 | 0 | 0 | 2.03 KB |
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartFinishSpan |
net6.0 | 763ns | 1.4ns | 5.04ns | 0 | 0 | 0 | 520 B |
| master | StartFinishSpan |
netcoreapp3.1 | 973ns | 4.7ns | 19.9ns | 0 | 0 | 0 | 520 B |
| master | StartFinishSpan |
net472 | 895ns | 0.173ns | 0.649ns | 0.0807 | 0 | 0 | 522 B |
| master | StartFinishScope |
net6.0 | 883ns | 5ns | 33.2ns | 0 | 0 | 0 | 640 B |
| master | StartFinishScope |
netcoreapp3.1 | 1.14μs | 5.44ns | 21.7ns | 0 | 0 | 0 | 640 B |
| master | StartFinishScope |
net472 | 1.14μs | 0.341ns | 1.27ns | 0.0915 | 0 | 0 | 602 B |
| master | StartFinishTwoScopes |
net6.0 | 1.86μs | 9.03ns | 37.2ns | 0 | 0 | 0 | 1.19 KB |
| master | StartFinishTwoScopes |
netcoreapp3.1 | 2.22μs | 11.2ns | 47.5ns | 0 | 0 | 0 | 1.19 KB |
| master | StartFinishTwoScopes |
net472 | 2.29μs | 0.469ns | 1.82ns | 0.16 | 0 | 0 | 1.08 KB |
| #7859 | StartFinishSpan |
net6.0 | 768ns | 3.46ns | 13.4ns | 0 | 0 | 0 | 520 B |
| #7859 | StartFinishSpan |
netcoreapp3.1 | 976ns | 0.377ns | 1.41ns | 0 | 0 | 0 | 520 B |
| #7859 | StartFinishSpan |
net472 | 896ns | 0.147ns | 0.53ns | 0.0808 | 0 | 0 | 522 B |
| #7859 | StartFinishScope |
net6.0 | 925ns | 1.93ns | 7.46ns | 0 | 0 | 0 | 640 B |
| #7859 | StartFinishScope |
netcoreapp3.1 | 1.14μs | 5.63ns | 25.8ns | 0 | 0 | 0 | 640 B |
| #7859 | StartFinishScope |
net472 | 1.08μs | 0.213ns | 0.796ns | 0.0916 | 0 | 0 | 602 B |
| #7859 | StartFinishTwoScopes |
net6.0 | 1.82μs | 8.96ns | 36.9ns | 0 | 0 | 0 | 1.19 KB |
| #7859 | StartFinishTwoScopes |
netcoreapp3.1 | 2.25μs | 1.68ns | 6.51ns | 0 | 0 | 0 | 1.19 KB |
| #7859 | StartFinishTwoScopes |
net472 | 2.12μs | 0.281ns | 1.05ns | 0.169 | 0 | 0 | 1.08 KB |
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunOnMethodBegin |
net6.0 | 1.07μs | 5.56ns | 24.3ns | 0 | 0 | 0 | 640 B |
| master | RunOnMethodBegin |
netcoreapp3.1 | 1.4μs | 6.44ns | 24.1ns | 0 | 0 | 0 | 640 B |
| master | RunOnMethodBegin |
net472 | 1.42μs | 0.403ns | 1.51ns | 0.0924 | 0 | 0 | 602 B |
| #7859 | RunOnMethodBegin |
net6.0 | 1.06μs | 5.31ns | 24.3ns | 0 | 0 | 0 | 640 B |
| #7859 | RunOnMethodBegin |
netcoreapp3.1 | 1.4μs | 6.93ns | 31ns | 0 | 0 | 0 | 640 B |
| #7859 | RunOnMethodBegin |
net472 | 1.44μs | 0.645ns | 2.5ns | 0.0937 | 0 | 0 | 602 B |
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (7859) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7859) - mean (76ms) : 72, 81
master - mean (75ms) : 70, 80
section Bailout
This PR (7859) - mean (81ms) : 76, 86
master - mean (81ms) : 75, 86
section CallTarget+Inlining+NGEN
This PR (7859) - mean (1,072ms) : 1016, 1128
master - mean (1,067ms) : 1015, 1120
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7859) - mean (120ms) : 112, 128
master - mean (117ms) : 110, 125
section Bailout
This PR (7859) - mean (120ms) : 113, 127
master - mean (120ms) : 114, 127
section CallTarget+Inlining+NGEN
This PR (7859) - mean (764ms) : 733, 796
master - mean (758ms) : 721, 795
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7859) - mean (105ms) : 100, 111
master - mean (105ms) : 98, 112
section Bailout
This PR (7859) - mean (107ms) : 101, 112
master - mean (106ms) : 101, 111
section CallTarget+Inlining+NGEN
This PR (7859) - mean (709ms) : 682, 735
master - mean (705ms) : 681, 730
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7859) - mean (104ms) : 96, 112
master - mean (105ms) : 98, 113
section Bailout
This PR (7859) - mean (105ms) : 99, 111
master - mean (105ms) : 100, 110
section CallTarget+Inlining+NGEN
This PR (7859) - mean (684ms) : 641, 726
master - mean (684ms) : 648, 720
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7859) - mean (194ms) : 189, 200
master - mean (193ms) : 188, 198
section Bailout
This PR (7859) - mean (197ms) : 194, 199
master - mean (197ms) : 193, 201
section CallTarget+Inlining+NGEN
This PR (7859) - mean (1,124ms) : 1057, 1190
master - mean (1,117ms) : 1056, 1179
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7859) - mean (278ms) : 272, 284
master - mean (277ms) : 273, 282
section Bailout
This PR (7859) - mean (278ms) : 275, 282
master - mean (279ms) : 274, 283
section CallTarget+Inlining+NGEN
This PR (7859) - mean (909ms) : 865, 953
master - mean (914ms) : 867, 961
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7859) - mean (271ms) : 267, 276
master - mean (272ms) : 267, 277
section Bailout
This PR (7859) - mean (269ms) : 266, 272
master - mean (271ms) : 267, 275
section CallTarget+Inlining+NGEN
This PR (7859) - mean (882ms) : 842, 921
master - mean (889ms) : 845, 932
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7859) - mean (270ms) : 263, 276
master - mean (270ms) : 265, 276
section Bailout
This PR (7859) - mean (270ms) : 264, 275
master - mean (271ms) : 266, 275
section CallTarget+Inlining+NGEN
This PR (7859) - mean (826ms) : 803, 849
master - mean (828ms) : 803, 853
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
Is this something we'll eventually need to support across the board? If so, we should probably add an accompanying system-test to confirm parity |
andrewlock
left a comment
There was a problem hiding this comment.
Lots of minor suggestions, but overall LGTM, only potential issue is the new process for adding configurations.
No integration tests though - is it possible to add any (may need to be a separate PR I realise)
|
|
||
| /// <summary> | ||
| /// Configuration key for enabling or disabling agentless Exception Replay uploads. | ||
| /// Default value is false. | ||
| /// </summary> | ||
| public const string ExceptionReplayAgentlessEnabled = "DD_EXCEPTION_REPLAY_AGENTLESS_ENABLED"; | ||
|
|
||
| /// <summary> | ||
| /// Configuration key for overriding the agentless Exception Replay intake URL. | ||
| /// Default value is derived from DD_SITE (https://debugger-intake.<site>/api/v2/debugger). | ||
| /// </summary> |
There was a problem hiding this comment.
You shouldn't add the keys here directly, right @anna-git? They need to be added to the supported-configurations.json and supported-configurations-docs.yaml files instead
There was a problem hiding this comment.
I'm lost, what is the new process?
There was a problem hiding this comment.
yes for now he can only add the key directly... and I'll rebase my open PRs adding the key and regenerate 😅
after this PR is merged in particular, yes, we'll all have to follow the "new way" but for now it's all good 👍
(btw, for now the generated keys exist but as ConfigurationKeys2 in master )
Summary of changes
SnapshotUploadApi/ExceptionReplayto use static debugger intakes when the agent is absent.Reason for change
For complete Test Optimization + Exception Replay integration in environments without the Datadog Agent need a documented path to send snapshots directly to Datadog without impacting Shared Debugger/DI behaviors. The plan isolates ER agentless support, reusing
DD_API_KEY/DD_SITE, and ensures we do not silently initialize features (symbol uploads) that still require the agent/RCM.Implementation details
ConfigurationKeys.Debugger+ExceptionReplaySettingsnow surfaceDD_EXCEPTION_REPLAY_AGENTLESS_ENABLED/DD_EXCEPTION_REPLAY_AGENTLESS_URL, defaulting todatadoghq.comand the existingDD_API_KEY.ExceptionReplayTransportFactorycreates either the existing agent transport or an HTTPS transport that injectsDD-API-KEY,DD-EVP-ORIGIN=dd-trace-dotnet, and a per-requestDD-REQUEST-ID, honouring site overrides.SnapshotUploadApiaccepts a static endpoint so ER can bypass discovery when agentless, andExceptionReplaywires the new transport info throughDebuggerUploadApiFactory.DebuggerManagernow emits a clear log and skips symbol uploader initialization whenever ER is agentless, avoiding futile RCM attempts.Test coverage
SnapshotUploadApiTests) continue to validate static endpoint behavior.Other details