Skip to content

[Test Optimization] - Add support for BenchmarkDotNet 0.15.x#7909

Merged
andrewlock merged 2 commits intomasterfrom
tony/add-benchmarkdotnet-0.15.x-support
Dec 5, 2025
Merged

[Test Optimization] - Add support for BenchmarkDotNet 0.15.x#7909
andrewlock merged 2 commits intomasterfrom
tony/add-benchmarkdotnet-0.15.x-support

Conversation

@tonyredondo
Copy link
Member

@tonyredondo tonyredondo commented Dec 5, 2025

Summary of changes

This PR adds support to BenchmarkDotNet 0.15.x

Reason for change

BenchmarkDotNet 0.15.x changed the public api.

Implementation details

Use the new api.

Test coverage

If we get benchmark results in the CI run then we are good!

Other details

@tonyredondo tonyredondo requested review from a team as code owners December 5, 2025 10:03
@andrewlock andrewlock added the area:benchmarks Benchmarks, throughput tests, Crank, Bombardier, etc label Dec 5, 2025
@datadog-official

This comment has been minimized.

Sign in to view
@andrewlock andrewlock enabled auto-merge (squash) December 5, 2025 11:30
@pr-commenter
Copy link

pr-commenter bot commented Dec 5, 2025

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7909 compared to master:

  • 1 benchmarks are faster, with geometric mean 2.232
  • 2 benchmarks have fewer allocations
  • 1 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7909

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 5.94 KB 6 KB 60 B 1.01%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 11.1μs 60.3ns 336ns 0 0 0 5.5 KB
master StartStopWithChild netcoreapp3.1 13.5μs 70.9ns 361ns 0 0 0 5.71 KB
master StartStopWithChild net472 22.1μs 83.8ns 324ns 1.01 0.336 0.112 5.94 KB
#7909 StartStopWithChild net6.0 11.2μs 55.6ns 243ns 0 0 0 5.5 KB
#7909 StartStopWithChild netcoreapp3.1 13.9μs 57.9ns 224ns 0 0 0 5.72 KB
#7909 StartStopWithChild net472 21.5μs 97.6ns 365ns 0.873 0.327 0 6 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 1.27ms 176ns 683ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.39ms 69.4ns 240ns 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.75ms 1.67μs 6.46μs 0 0 0 3.26 KB
#7909 WriteAndFlushEnrichedTraces net6.0 1.28ms 1.81μs 6.99μs 0 0 0 2.7 KB
#7909 WriteAndFlushEnrichedTraces netcoreapp3.1 1.39ms 375ns 1.45μs 0 0 0 2.7 KB
#7909 WriteAndFlushEnrichedTraces net472 1.71ms 1.03μs 4.01μs 0 0 0 3.26 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 1.08μs 4.95ns 19.2ns 0 0 0 1.22 KB
master AllCycleSimpleBody netcoreapp3.1 1.42μs 7.39ns 38.4ns 0 0 0 1.2 KB
master AllCycleSimpleBody net472 1.03μs 0.369ns 1.43ns 0.19 0 0 1.23 KB
master AllCycleMoreComplexBody net6.0 7.34μs 35.4ns 137ns 0 0 0 4.72 KB
master AllCycleMoreComplexBody netcoreapp3.1 9.07μs 43.7ns 185ns 0 0 0 4.62 KB
master AllCycleMoreComplexBody net472 7.61μs 4.07ns 15.8ns 0.724 0 0 4.74 KB
master ObjectExtractorSimpleBody net6.0 328ns 1.79ns 10.4ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 402ns 2.14ns 10.5ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 296ns 0.0491ns 0.19ns 0.0432 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.33μs 33.6ns 175ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.81μs 40.9ns 192ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.71μs 2.46ns 9.51ns 0.572 0 0 3.8 KB
#7909 AllCycleSimpleBody net6.0 1.12μs 4.32ns 16.7ns 0 0 0 1.22 KB
#7909 AllCycleSimpleBody netcoreapp3.1 1.45μs 8.37ns 65.4ns 0 0 0 1.2 KB
#7909 AllCycleSimpleBody net472 1.01μs 0.384ns 1.38ns 0.193 0 0 1.23 KB
#7909 AllCycleMoreComplexBody net6.0 7.33μs 16.7ns 58ns 0 0 0 4.72 KB
#7909 AllCycleMoreComplexBody netcoreapp3.1 9.07μs 44.3ns 188ns 0 0 0 4.62 KB
#7909 AllCycleMoreComplexBody net472 7.75μs 6.01ns 23.3ns 0.74 0 0 4.74 KB
#7909 ObjectExtractorSimpleBody net6.0 322ns 1.55ns 6.02ns 0 0 0 280 B
#7909 ObjectExtractorSimpleBody netcoreapp3.1 402ns 2.19ns 10.3ns 0 0 0 272 B
#7909 ObjectExtractorSimpleBody net472 295ns 0.0409ns 0.158ns 0.0432 0 0 281 B
#7909 ObjectExtractorMoreComplexBody net6.0 6.43μs 28.2ns 109ns 0 0 0 3.78 KB
#7909 ObjectExtractorMoreComplexBody netcoreapp3.1 7.9μs 38.1ns 162ns 0 0 0 3.69 KB
#7909 ObjectExtractorMoreComplexBody net472 6.74μs 4.44ns 16.6ns 0.57 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 77.1μs 41.5ns 155ns 0 0 0 32.4 KB
master EncodeArgs netcoreapp3.1 98.1μs 363ns 1.41μs 0 0 0 32.4 KB
master EncodeArgs net472 109μs 29.9ns 116ns 4.9 0 0 32.5 KB
master EncodeLegacyArgs net6.0 143μs 44ns 171ns 0 0 0 2.14 KB
master EncodeLegacyArgs netcoreapp3.1 197μs 26ns 93.6ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 261μs 18.9ns 73.3ns 0 0 0 2.14 KB
#7909 EncodeArgs net6.0 76.5μs 272ns 1.06μs 0 0 0 32.4 KB
#7909 EncodeArgs netcoreapp3.1 99.8μs 59.4ns 230ns 0 0 0 32.4 KB
#7909 EncodeArgs net472 108μs 17.4ns 67.4ns 4.87 0 0 32.5 KB
#7909 EncodeLegacyArgs net6.0 143μs 70.2ns 263ns 0 0 0 2.14 KB
#7909 EncodeLegacyArgs netcoreapp3.1 197μs 92.4ns 358ns 0 0 0 2.14 KB
#7909 EncodeLegacyArgs net472 273μs 22.1ns 76.6ns 0 0 0 2.15 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 430μs 1.21μs 4.36μs 0 0 0 4.55 KB
master RunWafRealisticBenchmark netcoreapp3.1 446μs 1.62μs 6.07μs 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 502μs 453ns 1.63μs 0 0 0 0 b
master RunWafRealisticBenchmarkWithAttack net6.0 320μs 1.08μs 3.9μs 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 350μs 3.6μs 34.4μs 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 378μs 668ns 2.59μs 0 0 0 0 b
#7909 RunWafRealisticBenchmark net6.0 427μs 671ns 2.42μs 0 0 0 4.55 KB
#7909 RunWafRealisticBenchmark netcoreapp3.1 450μs 2.5μs 20.6μs 0 0 0 4.48 KB
#7909 RunWafRealisticBenchmark net472 499μs 599ns 2.32μs 0 0 0 0 b
#7909 RunWafRealisticBenchmarkWithAttack net6.0 312μs 1.34μs 4.82μs 0 0 0 2.24 KB
#7909 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 352μs 2.8μs 26.3μs 0 0 0 2.22 KB
#7909 RunWafRealisticBenchmarkWithAttack net472 375μs 502ns 1.88μs 0 0 0 0 b
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 84.7μs 221ns 856ns 0 0 0 19.05 KB
master SendRequest netcoreapp3.1 96.7μs 248ns 1.51μs 0 0 0 21.18 KB
master SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
#7909 SendRequest net6.0 87.6μs 401ns 1.5μs 0 0 0 18.98 KB
#7909 SendRequest netcoreapp3.1 98μs 424ns 2.58μs 0 0 0 21.18 KB
#7909 SendRequest net472 0ns 0ns 0ns 0 0 0 0 b
Benchmarks.Trace.CharSliceBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #7909

Benchmark base/diff Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑netcoreapp3.1 2.232 1,854,600.00 830,800.00

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master OriginalCharSlice net6.0 1.97ms 1.51μs 5.86μs 0 0 0 640 KB
master OriginalCharSlice netcoreapp3.1 3.95ms 710ns 2.75μs 0 0 0 640.05 KB
master OriginalCharSlice net472 2.64ms 559ns 2.09μs 0 0 0 638.98 KB
master OptimizedCharSlice net6.0 1.44ms 954ns 3.57μs 0 0 0 0 b
master OptimizedCharSlice netcoreapp3.1 2.73ms 914ns 3.42μs 0 0 0 0 b
master OptimizedCharSlice net472 1.96ms 920ns 3.56μs 0 0 0 0 b
master OptimizedCharSliceWithPool net6.0 1.01ms 761ns 2.85μs 0 0 0 0 b
master OptimizedCharSliceWithPool netcoreapp3.1 1.86ms 1.28μs 4.97μs 0 0 0 0 b
master OptimizedCharSliceWithPool net472 1.17ms 576ns 2.23μs 0 0 0 0 b
#7909 OriginalCharSlice net6.0 2.04ms 283ns 980ns 0 0 0 640 KB
#7909 OriginalCharSlice netcoreapp3.1 4.03ms 1.31μs 5.07μs 0 0 0 640.05 KB
#7909 OriginalCharSlice net472 2.6ms 370ns 1.33μs 0 0 0 638.98 KB
#7909 OptimizedCharSlice net6.0 1.38ms 209ns 724ns 0 0 0 0 b
#7909 OptimizedCharSlice netcoreapp3.1 2.75ms 807ns 3.02μs 0 0 0 0 b
#7909 OptimizedCharSlice net472 1.91ms 943ns 3.65μs 0 0 0 0 b
#7909 OptimizedCharSliceWithPool net6.0 1.09ms 448ns 1.55μs 0 0 0 0 b
#7909 OptimizedCharSliceWithPool netcoreapp3.1 834μs 2.3μs 15.6μs 0 0 0 0 b
#7909 OptimizedCharSliceWithPool net472 1.2ms 441ns 1.71μs 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7909

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 42.27 KB 41.77 KB -502 B -1.19%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 709μs 3.12μs 11.2μs 0 0 0 41.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 665μs 3.56μs 18.5μs 0 0 0 42.27 KB
master WriteAndFlushEnrichedTraces net472 963μs 4.58μs 17.7μs 4.46 0 0 55.87 KB
#7909 WriteAndFlushEnrichedTraces net6.0 715μs 4.17μs 37.1μs 0 0 0 41.67 KB
#7909 WriteAndFlushEnrichedTraces netcoreapp3.1 699μs 3.69μs 18.1μs 0 0 0 41.77 KB
#7909 WriteAndFlushEnrichedTraces net472 878μs 3.63μs 14.1μs 4.46 0 0 55.62 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.91μs 1.99ns 7.69ns 0 0 0 968 B
master ExecuteNonQuery netcoreapp3.1 2.62μs 8.36ns 32.4ns 0 0 0 960 B
master ExecuteNonQuery net472 2.79μs 4.28ns 16.6ns 0.138 0 0 931 B
#7909 ExecuteNonQuery net6.0 2.06μs 8.99ns 34.8ns 0 0 0 968 B
#7909 ExecuteNonQuery netcoreapp3.1 2.61μs 6.04ns 23.4ns 0 0 0 960 B
#7909 ExecuteNonQuery net472 2.8μs 6.91ns 26.8ns 0.139 0 0 931 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.67μs 2.1ns 7.85ns 0 0 0 952 B
master CallElasticsearch netcoreapp3.1 2.25μs 6.14ns 23ns 0 0 0 968 B
master CallElasticsearch net472 3.26μs 4ns 15.5ns 0.146 0 0 955 B
master CallElasticsearchAsync net6.0 1.84μs 8.76ns 33.9ns 0 0 0 928 B
master CallElasticsearchAsync netcoreapp3.1 2.46μs 3.28ns 12.7ns 0 0 0 1.02 KB
master CallElasticsearchAsync net472 3.44μs 1.65ns 6.39ns 0.155 0 0 1.01 KB
#7909 CallElasticsearch net6.0 1.72μs 8.48ns 36ns 0 0 0 952 B
#7909 CallElasticsearch netcoreapp3.1 2.2μs 10.5ns 41.8ns 0 0 0 968 B
#7909 CallElasticsearch net472 3.42μs 8.4ns 32.5ns 0.137 0 0 955 B
#7909 CallElasticsearchAsync net6.0 1.76μs 0.321ns 1.2ns 0 0 0 928 B
#7909 CallElasticsearchAsync netcoreapp3.1 2.53μs 3.65ns 14.1ns 0 0 0 1.02 KB
#7909 CallElasticsearchAsync net472 3.59μs 4.17ns 16.1ns 0.16 0 0 1.01 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.92μs 0.392ns 1.52ns 0 0 0 896 B
master ExecuteAsync netcoreapp3.1 2.43μs 2.31ns 8.34ns 0 0 0 896 B
master ExecuteAsync net472 2.66μs 2.86ns 11.1ns 0.134 0 0 858 B
#7909 ExecuteAsync net6.0 1.82μs 8.89ns 35.6ns 0 0 0 896 B
#7909 ExecuteAsync netcoreapp3.1 2.38μs 10.1ns 39.3ns 0 0 0 896 B
#7909 ExecuteAsync net472 2.57μs 2.35ns 9.11ns 0.13 0 0 858 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 6.91μs 17ns 65.9ns 0 0 0 2.29 KB
master SendAsync netcoreapp3.1 8.45μs 24.5ns 94.9ns 0 0 0 2.83 KB
master SendAsync net472 12.1μs 8.58ns 33.2ns 0.481 0 0 3.08 KB
#7909 SendAsync net6.0 6.9μs 6.83ns 24.6ns 0 0 0 2.29 KB
#7909 SendAsync netcoreapp3.1 8.74μs 20.3ns 78.7ns 0 0 0 2.83 KB
#7909 SendAsync net472 12.1μs 9.04ns 35ns 0.483 0 0 3.08 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7909

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 274.06 KB 247.26 KB -26.8 KB -9.78%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 44.5μs 245ns 1.76μs 0 0 0 42.51 KB
master StringConcatBenchmark netcoreapp3.1 48.9μs 258ns 1.26μs 0 0 0 42.54 KB
master StringConcatBenchmark net472 57μs 259ns 1.03μs 0 0 0 49.15 KB
master StringConcatAspectBenchmark net6.0 488μs 2.5μs 11.4μs 0 0 0 274.06 KB
master StringConcatAspectBenchmark netcoreapp3.1 513μs 1.79μs 8.76μs 0 0 0 273.49 KB
master StringConcatAspectBenchmark net472 410μs 2.13μs 12.1μs 0 0 0 270.34 KB
#7909 StringConcatBenchmark net6.0 45.1μs 245ns 1.77μs 0 0 0 42.51 KB
#7909 StringConcatBenchmark netcoreapp3.1 48.1μs 239ns 1.04μs 0 0 0 42.54 KB
#7909 StringConcatBenchmark net472 56.6μs 243ns 908ns 0 0 0 49.15 KB
#7909 StringConcatAspectBenchmark net6.0 442μs 2.21μs 14.2μs 0 0 0 247.26 KB
#7909 StringConcatAspectBenchmark netcoreapp3.1 539μs 2.08μs 9.08μs 0 0 0 274.38 KB
#7909 StringConcatAspectBenchmark net472 398μs 1.14μs 4.28μs 0 0 0 270.34 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.77μs 14.4ns 52ns 0 0 0 1.69 KB
master EnrichedLog netcoreapp3.1 3.55μs 12.5ns 48.5ns 0 0 0 1.7 KB
master EnrichedLog net472 3.84μs 4.36ns 16.9ns 0.25 0 0 1.6 KB
#7909 EnrichedLog net6.0 2.67μs 13.7ns 67ns 0 0 0 1.69 KB
#7909 EnrichedLog netcoreapp3.1 3.53μs 18.6ns 81.1ns 0 0 0 1.7 KB
#7909 EnrichedLog net472 3.82μs 4.34ns 16.8ns 0.249 0 0 1.6 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 124μs 105ns 393ns 0 0 0 4.31 KB
master EnrichedLog netcoreapp3.1 128μs 68.2ns 255ns 0 0 0 4.31 KB
master EnrichedLog net472 169μs 145ns 562ns 0 0 0 4.51 KB
#7909 EnrichedLog net6.0 123μs 88.2ns 318ns 0 0 0 4.31 KB
#7909 EnrichedLog netcoreapp3.1 128μs 239ns 894ns 0 0 0 4.31 KB
#7909 EnrichedLog net472 167μs 202ns 784ns 0 0 0 4.51 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 5.11μs 24.6ns 95.4ns 0 0 0 2.24 KB
master EnrichedLog netcoreapp3.1 6.87μs 24.7ns 95.8ns 0 0 0 2.26 KB
master EnrichedLog net472 7.7μs 6.88ns 26.6ns 0.307 0 0 2.05 KB
#7909 EnrichedLog net6.0 5.08μs 12.8ns 49.5ns 0 0 0 2.24 KB
#7909 EnrichedLog netcoreapp3.1 6.84μs 9.92ns 38.4ns 0 0 0 2.26 KB
#7909 EnrichedLog net472 7.83μs 10.8ns 42ns 0.313 0 0 2.05 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 1.89μs 1.9ns 7.35ns 0 0 0 1.12 KB
master SendReceive netcoreapp3.1 2.43μs 12.8ns 61.3ns 0 0 0 1.14 KB
master SendReceive net472 2.88μs 1.12ns 4.34ns 0.175 0 0 1.12 KB
#7909 SendReceive net6.0 1.88μs 8.73ns 34.9ns 0 0 0 1.12 KB
#7909 SendReceive netcoreapp3.1 2.51μs 8.75ns 33.9ns 0 0 0 1.14 KB
#7909 SendReceive net472 2.96μs 3.34ns 12.5ns 0.163 0 0 1.12 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.39μs 12.2ns 45.7ns 0 0 0 1.58 KB
master EnrichedLog netcoreapp3.1 5.92μs 6.95ns 26.9ns 0 0 0 1.63 KB
master EnrichedLog net472 6.58μs 12.5ns 48.3ns 0.295 0 0 2.03 KB
#7909 EnrichedLog net6.0 4.65μs 11.2ns 43.4ns 0 0 0 1.58 KB
#7909 EnrichedLog netcoreapp3.1 5.64μs 10.8ns 41.7ns 0 0 0 1.63 KB
#7909 EnrichedLog net472 6.75μs 10.7ns 41.4ns 0.304 0 0 2.03 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 763ns 1.4ns 5.04ns 0 0 0 520 B
master StartFinishSpan netcoreapp3.1 973ns 4.7ns 19.9ns 0 0 0 520 B
master StartFinishSpan net472 895ns 0.173ns 0.649ns 0.0807 0 0 522 B
master StartFinishScope net6.0 883ns 5ns 33.2ns 0 0 0 640 B
master StartFinishScope netcoreapp3.1 1.14μs 5.44ns 21.7ns 0 0 0 640 B
master StartFinishScope net472 1.14μs 0.341ns 1.27ns 0.0915 0 0 602 B
master StartFinishTwoScopes net6.0 1.86μs 9.03ns 37.2ns 0 0 0 1.19 KB
master StartFinishTwoScopes netcoreapp3.1 2.22μs 11.2ns 47.5ns 0 0 0 1.19 KB
master StartFinishTwoScopes net472 2.29μs 0.469ns 1.82ns 0.16 0 0 1.08 KB
#7909 StartFinishSpan net6.0 769ns 3.64ns 14.1ns 0 0 0 520 B
#7909 StartFinishSpan netcoreapp3.1 972ns 4.72ns 20ns 0 0 0 520 B
#7909 StartFinishSpan net472 887ns 0.233ns 0.902ns 0.08 0 0 522 B
#7909 StartFinishScope net6.0 897ns 4.43ns 19.8ns 0 0 0 640 B
#7909 StartFinishScope netcoreapp3.1 1.14μs 4.84ns 18.7ns 0 0 0 640 B
#7909 StartFinishScope net472 1.12μs 1.08ns 4.17ns 0.0954 0 0 602 B
#7909 StartFinishTwoScopes net6.0 1.83μs 8.75ns 36.1ns 0 0 0 1.19 KB
#7909 StartFinishTwoScopes netcoreapp3.1 2.23μs 11.6ns 54.5ns 0 0 0 1.19 KB
#7909 StartFinishTwoScopes net472 2.14μs 4.1ns 15.9ns 0.17 0 0 1.08 KB
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.07μs 5.56ns 24.3ns 0 0 0 640 B
master RunOnMethodBegin netcoreapp3.1 1.4μs 6.44ns 24.1ns 0 0 0 640 B
master RunOnMethodBegin net472 1.42μs 0.403ns 1.51ns 0.0924 0 0 602 B
#7909 RunOnMethodBegin net6.0 1.04μs 5.01ns 20ns 0 0 0 640 B
#7909 RunOnMethodBegin netcoreapp3.1 1.37μs 6.56ns 27.1ns 0 0 0 640 B
#7909 RunOnMethodBegin net472 1.41μs 0.267ns 0.964ns 0.0925 0 0 602 B

@dd-trace-dotnet-ci-bot
Copy link

dd-trace-dotnet-ci-bot bot commented Dec 5, 2025

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (7909) and master.

✅ No regressions detected - check the details below

Full Metrics Comparison

FakeDbCommand

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration75.29 ± (75.02 - 75.68) ms76.19 ± (76.09 - 76.69) ms+1.2%✅⬆️
.NET Framework 4.8 - Bailout
duration80.23 ± (80.18 - 80.85) ms80.18 ± (79.91 - 80.67) ms-0.1%✅
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1064.32 ± (1063.80 - 1071.11) ms1065.18 ± (1066.90 - 1076.39) ms+0.1%✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms22.79 ± (22.72 - 22.86) ms22.75 ± (22.67 - 22.82) ms-0.2%✅
process.time_to_main_ms87.14 ± (86.81 - 87.47) ms87.34 ± (86.98 - 87.70) ms+0.2%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.92 ± (10.92 - 10.92) MB10.91 ± (10.91 - 10.92) MB-0.1%✅
runtime.dotnet.threads.count12 ± (12 - 12)12 ± (12 - 12)+0.0%✅
.NET Core 3.1 - Bailout
process.internal_duration_ms22.98 ± (22.91 - 23.06) ms22.72 ± (22.65 - 22.78) ms-1.2%✅
process.time_to_main_ms90.12 ± (89.75 - 90.49) ms88.71 ± (88.31 - 89.11) ms-1.6%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.96 ± (10.96 - 10.97) MB10.96 ± (10.96 - 10.96) MB-0.0%✅
runtime.dotnet.threads.count13 ± (13 - 13)13 ± (13 - 13)+0.0%✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms220.47 ± (219.05 - 221.88) ms221.24 ± (219.79 - 222.68) ms+0.3%✅⬆️
process.time_to_main_ms501.39 ± (500.09 - 502.69) ms499.81 ± (498.71 - 500.90) ms-0.3%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed47.86 ± (47.84 - 47.88) MB47.97 ± (47.95 - 48.00) MB+0.2%✅⬆️
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)-0.1%✅
.NET 6 - Baseline
process.internal_duration_ms21.60 ± (21.53 - 21.67) ms21.64 ± (21.57 - 21.70) ms+0.2%✅⬆️
process.time_to_main_ms76.50 ± (76.12 - 76.87) ms76.68 ± (76.38 - 76.98) ms+0.2%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.61 ± (10.61 - 10.61) MB10.63 ± (10.63 - 10.64) MB+0.2%✅⬆️
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%✅
.NET 6 - Bailout
process.internal_duration_ms21.62 ± (21.56 - 21.69) ms21.73 ± (21.65 - 21.81) ms+0.5%✅⬆️
process.time_to_main_ms77.17 ± (76.91 - 77.43) ms78.07 ± (77.68 - 78.45) ms+1.2%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.65 ± (10.65 - 10.66) MB10.68 ± (10.67 - 10.68) MB+0.2%✅⬆️
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms210.68 ± (209.58 - 211.77) ms211.51 ± (210.40 - 212.62) ms+0.4%✅⬆️
process.time_to_main_ms466.04 ± (465.17 - 466.91) ms465.60 ± (464.50 - 466.70) ms-0.1%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed48.15 ± (48.12 - 48.17) MB48.10 ± (48.08 - 48.12) MB-0.1%✅
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)+0.1%✅⬆️
.NET 8 - Baseline
process.internal_duration_ms20.01 ± (19.94 - 20.08) ms20.07 ± (19.99 - 20.16) ms+0.3%✅⬆️
process.time_to_main_ms76.41 ± (76.00 - 76.81) ms78.52 ± (78.08 - 78.96) ms+2.8%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed7.66 ± (7.66 - 7.67) MB7.65 ± (7.64 - 7.65) MB-0.2%✅
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%✅
.NET 8 - Bailout
process.internal_duration_ms19.90 ± (19.85 - 19.96) ms19.74 ± (19.68 - 19.80) ms-0.8%✅
process.time_to_main_ms76.42 ± (76.13 - 76.72) ms76.47 ± (76.12 - 76.82) ms+0.1%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed7.73 ± (7.72 - 7.73) MB7.69 ± (7.68 - 7.69) MB-0.5%✅
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms194.14 ± (193.17 - 195.12) ms191.93 ± (190.97 - 192.89) ms-1.1%✅
process.time_to_main_ms455.66 ± (454.79 - 456.53) ms454.23 ± (453.13 - 455.32) ms-0.3%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed36.45 ± (36.41 - 36.50) MB36.38 ± (36.34 - 36.42) MB-0.2%✅
runtime.dotnet.threads.count27 ± (27 - 27)27 ± (27 - 27)-0.3%✅

HttpMessageHandler

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration192.71 ± (192.88 - 193.81) ms193.47 ± (193.53 - 194.36) ms+0.4%✅⬆️
.NET Framework 4.8 - Bailout
duration196.61 ± (196.64 - 197.39) ms196.77 ± (196.51 - 197.34) ms+0.1%✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1109.95 ± (1113.01 - 1121.69) ms1106.04 ± (1108.19 - 1118.99) ms-0.4%✅
.NET Core 3.1 - Baseline
process.internal_duration_ms187.92 ± (187.61 - 188.23) ms188.17 ± (187.78 - 188.57) ms+0.1%✅⬆️
process.time_to_main_ms80.68 ± (80.51 - 80.84) ms80.54 ± (80.35 - 80.73) ms-0.2%✅
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed16.07 ± (16.04 - 16.09) MB16.08 ± (16.05 - 16.10) MB+0.1%✅⬆️
runtime.dotnet.threads.count20 ± (19 - 20)20 ± (19 - 20)-0.1%✅
.NET Core 3.1 - Bailout
process.internal_duration_ms187.80 ± (187.44 - 188.16) ms187.51 ± (187.20 - 187.82) ms-0.2%✅
process.time_to_main_ms82.40 ± (82.23 - 82.58) ms81.98 ± (81.80 - 82.15) ms-0.5%✅
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed16.12 ± (16.10 - 16.15) MB16.16 ± (16.13 - 16.20) MB+0.2%✅⬆️
runtime.dotnet.threads.count21 ± (21 - 21)21 ± (20 - 21)-0.2%✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms397.99 ± (395.31 - 400.68) ms401.17 ± (398.52 - 403.83) ms+0.8%✅⬆️
process.time_to_main_ms474.81 ± (474.20 - 475.43) ms475.81 ± (475.03 - 476.59) ms+0.2%✅⬆️
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed58.80 ± (58.68 - 58.93) MB58.74 ± (58.62 - 58.86) MB-0.1%✅
runtime.dotnet.threads.count29 ± (29 - 30)29 ± (29 - 29)-0.1%✅
.NET 6 - Baseline
process.internal_duration_ms192.89 ± (192.56 - 193.23) ms192.25 ± (191.89 - 192.61) ms-0.3%✅
process.time_to_main_ms70.28 ± (70.13 - 70.43) ms70.07 ± (69.91 - 70.24) ms-0.3%✅
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed16.19 ± (16.07 - 16.30) MB16.19 ± (16.07 - 16.31) MB+0.0%✅⬆️
runtime.dotnet.threads.count18 ± (18 - 19)18 ± (18 - 19)-0.5%✅
.NET 6 - Bailout
process.internal_duration_ms191.95 ± (191.66 - 192.24) ms191.22 ± (190.88 - 191.56) ms-0.4%✅
process.time_to_main_ms71.10 ± (71.01 - 71.19) ms70.88 ± (70.77 - 70.99) ms-0.3%✅
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed16.22 ± (16.09 - 16.35) MB16.09 ± (15.94 - 16.25) MB-0.8%✅
runtime.dotnet.threads.count20 ± (20 - 20)19 ± (19 - 20)-2.0%✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms410.45 ± (408.25 - 412.66) ms407.82 ± (405.75 - 409.90) ms-0.6%✅
process.time_to_main_ms443.18 ± (442.56 - 443.80) ms440.69 ± (440.11 - 441.26) ms-0.6%✅
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed59.06 ± (58.94 - 59.19) MB59.10 ± (58.97 - 59.23) MB+0.1%✅⬆️
runtime.dotnet.threads.count30 ± (29 - 30)30 ± (30 - 30)+0.1%✅⬆️
.NET 8 - Baseline
process.internal_duration_ms191.29 ± (190.92 - 191.66) ms190.18 ± (189.84 - 190.52) ms-0.6%✅
process.time_to_main_ms69.74 ± (69.57 - 69.92) ms69.45 ± (69.21 - 69.68) ms-0.4%✅
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed11.77 ± (11.75 - 11.80) MB11.74 ± (11.71 - 11.77) MB-0.3%✅
runtime.dotnet.threads.count18 ± (18 - 18)18 ± (18 - 18)-0.7%✅
.NET 8 - Bailout
process.internal_duration_ms190.44 ± (190.15 - 190.74) ms189.31 ± (189.08 - 189.55) ms-0.6%✅
process.time_to_main_ms70.59 ± (70.47 - 70.71) ms70.35 ± (70.27 - 70.43) ms-0.3%✅
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed11.82 ± (11.79 - 11.85) MB11.80 ± (11.77 - 11.84) MB-0.2%✅
runtime.dotnet.threads.count19 ± (19 - 19)19 ± (19 - 19)-0.1%✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms367.76 ± (366.61 - 368.90) ms365.17 ± (363.77 - 366.57) ms-0.7%✅
process.time_to_main_ms430.31 ± (429.76 - 430.86) ms428.05 ± (427.42 - 428.67) ms-0.5%✅
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed47.94 ± (47.91 - 47.96) MB47.97 ± (47.94 - 47.99) MB+0.1%✅⬆️
runtime.dotnet.threads.count29 ± (29 - 29)29 ± (29 - 29)+0.0%✅⬆️
Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts
FakeDbCommand (.NET Framework 4.8) 
gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7909) - mean (76ms)  : 72, 81
    master - mean (75ms)  : 70, 80

    section Bailout
    This PR (7909) - mean (80ms)  : 74, 86
    master - mean (81ms)  : 75, 86

    section CallTarget+Inlining+NGEN
    This PR (7909) - mean (1,072ms)  : 1004, 1139
    master - mean (1,067ms)  : 1015, 1120
Loading
FakeDbCommand (.NET Core 3.1) 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7909) - mean (118ms)  : 112, 124
    master - mean (117ms)  : 110, 125

    section Bailout
    This PR (7909) - mean (119ms)  : 112, 126
    master - mean (120ms)  : 114, 127

    section CallTarget+Inlining+NGEN
    This PR (7909) - mean (761ms)  : 716, 806
    master - mean (758ms)  : 721, 795
Loading
FakeDbCommand (.NET 6) 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7909) - mean (105ms)  : 99, 111
    master - mean (105ms)  : 98, 112

    section Bailout
    This PR (7909) - mean (107ms)  : 99, 115
    master - mean (106ms)  : 101, 111

    section CallTarget+Inlining+NGEN
    This PR (7909) - mean (705ms)  : 674, 736
    master - mean (705ms)  : 681, 730
Loading
FakeDbCommand (.NET 8) 
gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7909) - mean (108ms)  : 100, 116
    master - mean (105ms)  : 98, 113

    section Bailout
    This PR (7909) - mean (105ms)  : 99, 111
    master - mean (105ms)  : 100, 110

    section CallTarget+Inlining+NGEN
    This PR (7909) - mean (679ms)  : 641, 716
    master - mean (684ms)  : 648, 720
Loading
HttpMessageHandler (.NET Framework 4.8) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7909) - mean (194ms)  : 189, 199
    master - mean (193ms)  : 188, 198

    section Bailout
    This PR (7909) - mean (197ms)  : 193, 201
    master - mean (197ms)  : 193, 201

    section CallTarget+Inlining+NGEN
    This PR (7909) - mean (1,114ms)  : 1054, 1173
    master - mean (1,117ms)  : 1056, 1179
Loading
HttpMessageHandler (.NET Core 3.1) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7909) - mean (277ms)  : 272, 282
    master - mean (277ms)  : 273, 282

    section Bailout
    This PR (7909) - mean (278ms)  : 273, 282
    master - mean (279ms)  : 274, 283

    section CallTarget+Inlining+NGEN
    This PR (7909) - mean (914ms)  : 861, 967
    master - mean (914ms)  : 867, 961
Loading
HttpMessageHandler (.NET 6) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7909) - mean (271ms)  : 266, 275
    master - mean (272ms)  : 267, 277

    section Bailout
    This PR (7909) - mean (270ms)  : 265, 275
    master - mean (271ms)  : 267, 275

    section CallTarget+Inlining+NGEN
    This PR (7909) - mean (881ms)  : 845, 918
    master - mean (889ms)  : 845, 932
Loading
HttpMessageHandler (.NET 8) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7909) - mean (269ms)  : 263, 275
    master - mean (270ms)  : 265, 276

    section Bailout
    This PR (7909) - mean (269ms)  : 266, 272
    master - mean (271ms)  : 266, 275

    section CallTarget+Inlining+NGEN
    This PR (7909) - mean (824ms)  : 805, 844
    master - mean (828ms)  : 803, 853
Loading

@andrewlock andrewlock merged commit 5d20e39 into master Dec 5, 2025
150 checks passed
@andrewlock andrewlock deleted the tony/add-benchmarkdotnet-0.15.x-support branch December 5, 2025 11:59
@github-actions github-actions bot added this to the vNext-v3 milestone Dec 5, 2025
@andrewlock andrewlock mentioned this pull request Dec 5, 2025
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewlock andrewlock andrewlock approved these changes

Assignees

No one assigned

Labels

area:benchmarks Benchmarks, throughput tests, Crank, Bombardier, etc area:ci-visibility

Projects

None yet

Milestone

3.33.0

Development

Successfully merging this pull request may close these issues.

2 participants

@tonyredondo @andrewlock