Simplify Windows macrobenchmarks AMI build#7928
Conversation
|
Warning This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
This stack of pull requests is managed by Graphite. Learn more about stacking. |
d91ac87 to
a28cf91
Compare
|
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (7928) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7928) - mean (75ms) : 70, 81
master - mean (75ms) : 70, 80
section Bailout
This PR (7928) - mean (80ms) : 75, 85
master - mean (81ms) : 75, 87
section CallTarget+Inlining+NGEN
This PR (7928) - mean (1,081ms) : 1014, 1148
master - mean (1,080ms) : 1027, 1133
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7928) - mean (119ms) : 111, 126
master - mean (117ms) : 112, 123
section Bailout
This PR (7928) - mean (118ms) : 113, 123
master - mean (120ms) : 113, 127
section CallTarget+Inlining+NGEN
This PR (7928) - mean (759ms) : 726, 791
master - mean (770ms) : 735, 805
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7928) - mean (102ms) : 96, 109
master - mean (105ms) : 99, 110
section Bailout
This PR (7928) - mean (105ms) : 99, 110
master - mean (105ms) : 99, 110
section CallTarget+Inlining+NGEN
This PR (7928) - mean (705ms) : 676, 734
master - mean (707ms) : 680, 733
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7928) - mean (103ms) : 97, 108
master - mean (103ms) : 96, 109
section Bailout
This PR (7928) - mean (104ms) : 99, 109
master - mean (104ms) : 98, 110
section CallTarget+Inlining+NGEN
This PR (7928) - mean (681ms) : 649, 713
master - mean (679ms) : 645, 713
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7928) - mean (195ms) : 190, 200
master - mean (195ms) : 190, 200
section Bailout
This PR (7928) - mean (199ms) : 194, 204
master - mean (199ms) : 195, 204
section CallTarget+Inlining+NGEN
This PR (7928) - mean (1,124ms) : 1055, 1193
master - mean (1,123ms) : 1069, 1178
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7928) - mean (280ms) : 273, 288
master - mean (280ms) : 272, 287
section Bailout
This PR (7928) - mean (280ms) : 274, 285
master - mean (280ms) : 273, 286
section CallTarget+Inlining+NGEN
This PR (7928) - mean (911ms) : 869, 952
master - mean (921ms) : 871, 971
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7928) - mean (275ms) : 265, 285
master - mean (273ms) : 264, 282
section Bailout
This PR (7928) - mean (273ms) : 268, 278
master - mean (273ms) : 267, 279
section CallTarget+Inlining+NGEN
This PR (7928) - mean (897ms) : 842, 952
master - mean (897ms) : 848, 946
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7928) - mean (272ms) : 264, 281
master - mean (273ms) : 265, 281
section Bailout
This PR (7928) - mean (272ms) : 267, 277
master - mean (272ms) : 268, 277
section CallTarget+Inlining+NGEN
This PR (7928) - mean (832ms) : 808, 856
master - mean (834ms) : 809, 859
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
9132e3f to
bec3c08
Compare
5a19606 to
98ad60a
Compare
2482a42 to
96b22f7
Compare
BenchmarksBenchmarks Report for benchmark platform 🐌Benchmarks for #7928 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ Fewer allocations 🎉
|
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 | 6.01 KB | 5.93 KB | -74 B | -1.23% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartStopWithChild |
net6.0 | 10.8μs | 59.4ns | 366ns | 0 | 0 | 0 | 5.5 KB |
| master | StartStopWithChild |
netcoreapp3.1 | 13.2μs | 67.8ns | 303ns | 0 | 0 | 0 | 5.7 KB |
| master | StartStopWithChild |
net472 | 22.1μs | 109ns | 534ns | 0.89 | 0.334 | 0 | 6.01 KB |
| #7928 | StartStopWithChild |
net6.0 | 11.1μs | 60.6ns | 348ns | 0 | 0 | 0 | 5.51 KB |
| #7928 | StartStopWithChild |
netcoreapp3.1 | 13.6μs | 64.8ns | 259ns | 0 | 0 | 0 | 5.71 KB |
| #7928 | StartStopWithChild |
net472 | 22.2μs | 115ns | 628ns | 0.877 | 0.219 | 0 | 5.93 KB |
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | WriteAndFlushEnrichedTraces |
net6.0 | 1.26ms | 83.4ns | 312ns | 0 | 0 | 0 | 2.7 KB |
| master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 1.39ms | 96.8ns | 362ns | 0 | 0 | 0 | 2.7 KB |
| master | WriteAndFlushEnrichedTraces |
net472 | 1.72ms | 177ns | 685ns | 0 | 0 | 0 | 3.26 KB |
| #7928 | WriteAndFlushEnrichedTraces |
net6.0 | 1.25ms | 856ns | 3.2μs | 0 | 0 | 0 | 2.7 KB |
| #7928 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 1.37ms | 424ns | 1.64μs | 0 | 0 | 0 | 2.7 KB |
| #7928 | WriteAndFlushEnrichedTraces |
net472 | 1.7ms | 389ns | 1.51μs | 0 | 0 | 0 | 3.26 KB |
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | AllCycleSimpleBody |
net6.0 | 1.14μs | 1.12ns | 4.2ns | 0 | 0 | 0 | 1.22 KB |
| master | AllCycleSimpleBody |
netcoreapp3.1 | 1.4μs | 7.95ns | 62.1ns | 0 | 0 | 0 | 1.2 KB |
| master | AllCycleSimpleBody |
net472 | 1.02μs | 1.51ns | 5.85ns | 0.194 | 0 | 0 | 1.23 KB |
| master | AllCycleMoreComplexBody |
net6.0 | 7.43μs | 2.28ns | 8.23ns | 0 | 0 | 0 | 4.72 KB |
| master | AllCycleMoreComplexBody |
netcoreapp3.1 | 8.82μs | 43.2ns | 188ns | 0 | 0 | 0 | 4.62 KB |
| master | AllCycleMoreComplexBody |
net472 | 7.61μs | 4.96ns | 19.2ns | 0.722 | 0 | 0 | 4.74 KB |
| master | ObjectExtractorSimpleBody |
net6.0 | 343ns | 1.92ns | 13ns | 0 | 0 | 0 | 280 B |
| master | ObjectExtractorSimpleBody |
netcoreapp3.1 | 394ns | 2.03ns | 9.73ns | 0 | 0 | 0 | 272 B |
| master | ObjectExtractorSimpleBody |
net472 | 300ns | 0.0141ns | 0.0529ns | 0.0439 | 0 | 0 | 281 B |
| master | ObjectExtractorMoreComplexBody |
net6.0 | 6.41μs | 30.8ns | 119ns | 0 | 0 | 0 | 3.78 KB |
| master | ObjectExtractorMoreComplexBody |
netcoreapp3.1 | 8.01μs | 2.4ns | 9.29ns | 0 | 0 | 0 | 3.69 KB |
| master | ObjectExtractorMoreComplexBody |
net472 | 6.73μs | 1.69ns | 6.55ns | 0.572 | 0 | 0 | 3.8 KB |
| #7928 | AllCycleSimpleBody |
net6.0 | 1.09μs | 6.06ns | 40.7ns | 0 | 0 | 0 | 1.22 KB |
| #7928 | AllCycleSimpleBody |
netcoreapp3.1 | 1.41μs | 7.42ns | 37.1ns | 0 | 0 | 0 | 1.2 KB |
| #7928 | AllCycleSimpleBody |
net472 | 1.04μs | 1.24ns | 4.65ns | 0.192 | 0 | 0 | 1.23 KB |
| #7928 | AllCycleMoreComplexBody |
net6.0 | 7.41μs | 39.3ns | 192ns | 0 | 0 | 0 | 4.72 KB |
| #7928 | AllCycleMoreComplexBody |
netcoreapp3.1 | 8.84μs | 46.8ns | 234ns | 0 | 0 | 0 | 4.62 KB |
| #7928 | AllCycleMoreComplexBody |
net472 | 7.67μs | 2.55ns | 9.87ns | 0.728 | 0 | 0 | 4.74 KB |
| #7928 | ObjectExtractorSimpleBody |
net6.0 | 332ns | 0.168ns | 0.628ns | 0 | 0 | 0 | 280 B |
| #7928 | ObjectExtractorSimpleBody |
netcoreapp3.1 | 395ns | 2.23ns | 14.1ns | 0 | 0 | 0 | 272 B |
| #7928 | ObjectExtractorSimpleBody |
net472 | 306ns | 0.156ns | 0.582ns | 0.0446 | 0 | 0 | 281 B |
| #7928 | ObjectExtractorMoreComplexBody |
net6.0 | 6.41μs | 28.5ns | 110ns | 0 | 0 | 0 | 3.78 KB |
| #7928 | ObjectExtractorMoreComplexBody |
netcoreapp3.1 | 7.82μs | 32.4ns | 125ns | 0 | 0 | 0 | 3.69 KB |
| #7928 | ObjectExtractorMoreComplexBody |
net472 | 6.85μs | 3.89ns | 15.1ns | 0.583 | 0 | 0 | 3.8 KB |
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EncodeArgs |
net6.0 | 75.9μs | 216ns | 836ns | 0 | 0 | 0 | 32.4 KB |
| master | EncodeArgs |
netcoreapp3.1 | 97.8μs | 220ns | 854ns | 0 | 0 | 0 | 32.4 KB |
| master | EncodeArgs |
net472 | 110μs | 67.3ns | 261ns | 4.94 | 0 | 0 | 32.5 KB |
| master | EncodeLegacyArgs |
net6.0 | 146μs | 14.9ns | 55.9ns | 0 | 0 | 0 | 2.14 KB |
| master | EncodeLegacyArgs |
netcoreapp3.1 | 198μs | 177ns | 684ns | 0 | 0 | 0 | 2.14 KB |
| master | EncodeLegacyArgs |
net472 | 266μs | 36.8ns | 143ns | 0 | 0 | 0 | 2.15 KB |
| #7928 | EncodeArgs |
net6.0 | 77.7μs | 89.8ns | 348ns | 0 | 0 | 0 | 32.4 KB |
| #7928 | EncodeArgs |
netcoreapp3.1 | 96.9μs | 236ns | 912ns | 0 | 0 | 0 | 32.4 KB |
| #7928 | EncodeArgs |
net472 | 108μs | 10.4ns | 40.2ns | 4.88 | 0 | 0 | 32.5 KB |
| #7928 | EncodeLegacyArgs |
net6.0 | 156μs | 25.6ns | 99.3ns | 0 | 0 | 0 | 2.14 KB |
| #7928 | EncodeLegacyArgs |
netcoreapp3.1 | 197μs | 56.3ns | 211ns | 0 | 0 | 0 | 2.14 KB |
| #7928 | EncodeLegacyArgs |
net472 | 265μs | 26.3ns | 102ns | 0 | 0 | 0 | 2.14 KB |
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunWafRealisticBenchmark |
net6.0 | 426μs | 1.65μs | 5.96μs | 0 | 0 | 0 | 4.55 KB |
| master | RunWafRealisticBenchmark |
netcoreapp3.1 | 450μs | 1.53μs | 5.52μs | 0 | 0 | 0 | 4.48 KB |
| master | RunWafRealisticBenchmark |
net472 | 490μs | 555ns | 2.08μs | 0 | 0 | 0 | 0 b |
| master | RunWafRealisticBenchmarkWithAttack |
net6.0 | 313μs | 1.12μs | 4.19μs | 0 | 0 | 0 | 2.24 KB |
| master | RunWafRealisticBenchmarkWithAttack |
netcoreapp3.1 | 338μs | 2.72μs | 25.9μs | 0 | 0 | 0 | 2.22 KB |
| master | RunWafRealisticBenchmarkWithAttack |
net472 | 360μs | 342ns | 1.18μs | 0 | 0 | 0 | 0 b |
| #7928 | RunWafRealisticBenchmark |
net6.0 | 424μs | 1.28μs | 4.95μs | 0 | 0 | 0 | 4.55 KB |
| #7928 | RunWafRealisticBenchmark |
netcoreapp3.1 | 449μs | 1.26μs | 4.35μs | 0 | 0 | 0 | 4.48 KB |
| #7928 | RunWafRealisticBenchmark |
net472 | 487μs | 475ns | 1.78μs | 0 | 0 | 0 | 0 b |
| #7928 | RunWafRealisticBenchmarkWithAttack |
net6.0 | 320μs | 1.04μs | 3.75μs | 0 | 0 | 0 | 2.24 KB |
| #7928 | RunWafRealisticBenchmarkWithAttack |
netcoreapp3.1 | 338μs | 2.24μs | 21.4μs | 0 | 0 | 0 | 2.22 KB |
| #7928 | RunWafRealisticBenchmarkWithAttack |
net472 | 365μs | 189ns | 680ns | 0 | 0 | 0 | 0 b |
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendRequest |
net6.0 | 83.6μs | 60.1ns | 233ns | 0 | 0 | 0 | 18.97 KB |
| master | SendRequest |
netcoreapp3.1 | 98.1μs | 565ns | 4.52μs | 0 | 0 | 0 | 21.18 KB |
| master | SendRequest |
net472 | 0ns | 0ns | 0ns | 0 | 0 | 0 | 0 b |
| #7928 | SendRequest |
net6.0 | 83.1μs | 67.7ns | 253ns | 0 | 0 | 0 | 18.97 KB |
| #7928 | SendRequest |
netcoreapp3.1 | 95.9μs | 60.9ns | 220ns | 0 | 0 | 0 | 21.18 KB |
| #7928 | SendRequest |
net472 | 0ns | 0ns | 0ns | 0 | 0 | 0 | 0 b |
Benchmarks.Trace.CharSliceBenchmark - Faster 🎉 Same allocations ✔️
Faster 🎉 in #7928
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑netcoreapp3.1
1.162
3,235,500.00
2,784,600.00
| Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑netcoreapp3.1 | 1.162 | 3,235,500.00 | 2,784,600.00 |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | OriginalCharSlice |
net6.0 | 2.03ms | 1.87μs | 7.26μs | 0 | 0 | 0 | 640 KB |
| master | OriginalCharSlice |
netcoreapp3.1 | 4.22ms | 7.51μs | 29.1μs | 0 | 0 | 0 | 640.05 KB |
| master | OriginalCharSlice |
net472 | 2.59ms | 280ns | 971ns | 0 | 0 | 0 | 638.98 KB |
| master | OptimizedCharSlice |
net6.0 | 1.56ms | 3.28μs | 12.7μs | 0 | 0 | 0 | 0 b |
| master | OptimizedCharSlice |
netcoreapp3.1 | 3.24ms | 6.21μs | 24.1μs | 0 | 0 | 0 | 0 b |
| master | OptimizedCharSlice |
net472 | 1.94ms | 599ns | 2.16μs | 0 | 0 | 0 | 0 b |
| master | OptimizedCharSliceWithPool |
net6.0 | 1.1ms | 457ns | 1.77μs | 0 | 0 | 0 | 0 b |
| master | OptimizedCharSliceWithPool |
netcoreapp3.1 | 1.9ms | 2.28μs | 8.82μs | 0 | 0 | 0 | 0 b |
| master | OptimizedCharSliceWithPool |
net472 | 1.17ms | 1.05μs | 4.08μs | 0 | 0 | 0 | 0 b |
| #7928 | OriginalCharSlice |
net6.0 | 1.93ms | 648ns | 2.43μs | 0 | 0 | 0 | 640 KB |
| #7928 | OriginalCharSlice |
netcoreapp3.1 | 3.94ms | 919ns | 3.44μs | 0 | 0 | 0 | 640.05 KB |
| #7928 | OriginalCharSlice |
net472 | 2.62ms | 653ns | 2.36μs | 0 | 0 | 0 | 638.98 KB |
| #7928 | OptimizedCharSlice |
net6.0 | 1.48ms | 1.22μs | 4.73μs | 0 | 0 | 0 | 0 b |
| #7928 | OptimizedCharSlice |
netcoreapp3.1 | 2.78ms | 1.18μs | 4.58μs | 0 | 0 | 0 | 0 b |
| #7928 | OptimizedCharSlice |
net472 | 1.91ms | 669ns | 2.5μs | 0 | 0 | 0 | 0 b |
| #7928 | OptimizedCharSliceWithPool |
net6.0 | 1.01ms | 445ns | 1.54μs | 0 | 0 | 0 | 0 b |
| #7928 | OptimizedCharSliceWithPool |
netcoreapp3.1 | 1.89ms | 1.76μs | 6.82μs | 0 | 0 | 0 | 0 b |
| #7928 | OptimizedCharSliceWithPool |
net472 | 1.15ms | 694ns | 2.69μs | 0 | 0 | 0 | 0 b |
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ More allocations ⚠️
More allocations ⚠️ in #7928
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472
55.9 KB
56.25 KB
349 B
0.62%
Fewer allocations 🎉 in #7928
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1
42.78 KB
41.86 KB
-924 B
-2.16%
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 | 55.9 KB | 56.25 KB | 349 B | 0.62% |
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 | 42.78 KB | 41.86 KB | -924 B | -2.16% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | WriteAndFlushEnrichedTraces |
net6.0 | 640μs | 636ns | 2.29μs | 0 | 0 | 0 | 41.72 KB |
| master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 693μs | 4.72μs | 46.9μs | 0 | 0 | 0 | 42.78 KB |
| master | WriteAndFlushEnrichedTraces |
net472 | 863μs | 1.41μs | 5.09μs | 4.46 | 0 | 0 | 55.9 KB |
| #7928 | WriteAndFlushEnrichedTraces |
net6.0 | 659μs | 3.58μs | 19.3μs | 0 | 0 | 0 | 41.69 KB |
| #7928 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 759μs | 4.28μs | 28.1μs | 0 | 0 | 0 | 41.86 KB |
| #7928 | WriteAndFlushEnrichedTraces |
net472 | 868μs | 1.68μs | 6.5μs | 8.33 | 4.17 | 0 | 56.25 KB |
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteNonQuery |
net6.0 | 1.96μs | 1.12ns | 4.19ns | 0 | 0 | 0 | 968 B |
| master | ExecuteNonQuery |
netcoreapp3.1 | 2.66μs | 11.4ns | 44.2ns | 0 | 0 | 0 | 960 B |
| master | ExecuteNonQuery |
net472 | 2.75μs | 7.07ns | 27.4ns | 0.137 | 0 | 0 | 931 B |
| #7928 | ExecuteNonQuery |
net6.0 | 1.99μs | 9.2ns | 35.6ns | 0 | 0 | 0 | 968 B |
| #7928 | ExecuteNonQuery |
netcoreapp3.1 | 2.67μs | 3.91ns | 15.1ns | 0 | 0 | 0 | 960 B |
| #7928 | ExecuteNonQuery |
net472 | 2.7μs | 4.01ns | 15.5ns | 0.135 | 0 | 0 | 931 B |
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | CallElasticsearch |
net6.0 | 1.65μs | 0.885ns | 3.31ns | 0 | 0 | 0 | 952 B |
| master | CallElasticsearch |
netcoreapp3.1 | 2.29μs | 10.5ns | 40.8ns | 0 | 0 | 0 | 968 B |
| master | CallElasticsearch |
net472 | 3.53μs | 2.35ns | 9.11ns | 0.141 | 0 | 0 | 955 B |
| master | CallElasticsearchAsync |
net6.0 | 1.71μs | 8.14ns | 31.5ns | 0 | 0 | 0 | 928 B |
| master | CallElasticsearchAsync |
netcoreapp3.1 | 2.26μs | 11.3ns | 49.3ns | 0 | 0 | 0 | 1.02 KB |
| master | CallElasticsearchAsync |
net472 | 3.38μs | 2.38ns | 8.91ns | 0.153 | 0 | 0 | 1.01 KB |
| #7928 | CallElasticsearch |
net6.0 | 1.71μs | 1.67ns | 6.25ns | 0 | 0 | 0 | 952 B |
| #7928 | CallElasticsearch |
netcoreapp3.1 | 2.18μs | 9.76ns | 37.8ns | 0 | 0 | 0 | 968 B |
| #7928 | CallElasticsearch |
net472 | 3.65μs | 9.76ns | 37.8ns | 0.144 | 0 | 0 | 955 B |
| #7928 | CallElasticsearchAsync |
net6.0 | 1.68μs | 2.1ns | 8.14ns | 0 | 0 | 0 | 928 B |
| #7928 | CallElasticsearchAsync |
netcoreapp3.1 | 2.27μs | 9.24ns | 35.8ns | 0 | 0 | 0 | 1.02 KB |
| #7928 | CallElasticsearchAsync |
net472 | 3.52μs | 2.59ns | 9.7ns | 0.158 | 0 | 0 | 1.01 KB |
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteAsync |
net6.0 | 1.75μs | 7.42ns | 28.7ns | 0 | 0 | 0 | 896 B |
| master | ExecuteAsync |
netcoreapp3.1 | 2.48μs | 8.3ns | 32.2ns | 0 | 0 | 0 | 896 B |
| master | ExecuteAsync |
net472 | 2.58μs | 2.41ns | 9.35ns | 0.129 | 0 | 0 | 858 B |
| #7928 | ExecuteAsync |
net6.0 | 1.85μs | 7.01ns | 27.2ns | 0 | 0 | 0 | 896 B |
| #7928 | ExecuteAsync |
netcoreapp3.1 | 2.36μs | 8.97ns | 34.7ns | 0 | 0 | 0 | 896 B |
| #7928 | ExecuteAsync |
net472 | 2.54μs | 2.58ns | 9.29ns | 0.126 | 0 | 0 | 858 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendAsync |
net6.0 | 6.73μs | 8.54ns | 33.1ns | 0 | 0 | 0 | 2.29 KB |
| master | SendAsync |
netcoreapp3.1 | 8.62μs | 23.2ns | 89.9ns | 0 | 0 | 0 | 2.83 KB |
| master | SendAsync |
net472 | 12μs | 9.54ns | 35.7ns | 0.48 | 0 | 0 | 3.08 KB |
| #7928 | SendAsync |
net6.0 | 6.51μs | 20.4ns | 73.5ns | 0 | 0 | 0 | 2.29 KB |
| #7928 | SendAsync |
netcoreapp3.1 | 8.69μs | 30.5ns | 118ns | 0 | 0 | 0 | 2.83 KB |
| #7928 | SendAsync |
net472 | 11.9μs | 9.38ns | 35.1ns | 0.478 | 0 | 0 | 3.08 KB |
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ Fewer allocations 🎉
Fewer allocations 🎉 in #7928
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0
257.68 KB
255.48 KB
-2.2 KB
-0.85%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1
276.65 KB
272.28 KB
-4.37 KB
-1.58%
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 | 257.68 KB | 255.48 KB | -2.2 KB | -0.85% |
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 | 276.65 KB | 272.28 KB | -4.37 KB | -1.58% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StringConcatBenchmark |
net6.0 | 41.8μs | 217ns | 1.02μs | 0 | 0 | 0 | 42.51 KB |
| master | StringConcatBenchmark |
netcoreapp3.1 | 48.3μs | 219ns | 789ns | 0 | 0 | 0 | 42.54 KB |
| master | StringConcatBenchmark |
net472 | 57.7μs | 186ns | 766ns | 0 | 0 | 0 | 49.15 KB |
| master | StringConcatAspectBenchmark |
net6.0 | 459μs | 2.28μs | 9.4μs | 0 | 0 | 0 | 257.68 KB |
| master | StringConcatAspectBenchmark |
netcoreapp3.1 | 534μs | 1.64μs | 8.67μs | 0 | 0 | 0 | 276.65 KB |
| master | StringConcatAspectBenchmark |
net472 | 401μs | 2.07μs | 15.9μs | 0 | 0 | 0 | 270.34 KB |
| #7928 | StringConcatBenchmark |
net6.0 | 44.2μs | 234ns | 1.22μs | 0 | 0 | 0 | 42.51 KB |
| #7928 | StringConcatBenchmark |
netcoreapp3.1 | 50.8μs | 275ns | 2.38μs | 0 | 0 | 0 | 42.54 KB |
| #7928 | StringConcatBenchmark |
net472 | 57.9μs | 108ns | 389ns | 0 | 0 | 0 | 49.15 KB |
| #7928 | StringConcatAspectBenchmark |
net6.0 | 448μs | 1.84μs | 9.21μs | 0 | 0 | 0 | 255.48 KB |
| #7928 | StringConcatAspectBenchmark |
netcoreapp3.1 | 519μs | 2.3μs | 8.3μs | 0 | 0 | 0 | 272.28 KB |
| #7928 | StringConcatAspectBenchmark |
net472 | 400μs | 1.94μs | 7.78μs | 0 | 0 | 0 | 270.34 KB |
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 2.73μs | 14.6ns | 78.7ns | 0 | 0 | 0 | 1.69 KB |
| master | EnrichedLog |
netcoreapp3.1 | 3.55μs | 18ns | 80.6ns | 0 | 0 | 0 | 1.7 KB |
| master | EnrichedLog |
net472 | 3.89μs | 4.06ns | 15.7ns | 0.252 | 0 | 0 | 1.6 KB |
| #7928 | EnrichedLog |
net6.0 | 2.66μs | 12.8ns | 54.4ns | 0 | 0 | 0 | 1.69 KB |
| #7928 | EnrichedLog |
netcoreapp3.1 | 3.51μs | 16.7ns | 64.8ns | 0 | 0 | 0 | 1.7 KB |
| #7928 | EnrichedLog |
net472 | 3.85μs | 4.15ns | 16.1ns | 0.248 | 0 | 0 | 1.6 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 123μs | 28.6ns | 111ns | 0 | 0 | 0 | 4.31 KB |
| master | EnrichedLog |
netcoreapp3.1 | 127μs | 184ns | 713ns | 0 | 0 | 0 | 4.31 KB |
| master | EnrichedLog |
net472 | 166μs | 36.2ns | 136ns | 0 | 0 | 0 | 4.51 KB |
| #7928 | EnrichedLog |
net6.0 | 123μs | 74.2ns | 287ns | 0 | 0 | 0 | 4.31 KB |
| #7928 | EnrichedLog |
netcoreapp3.1 | 126μs | 20ns | 72.1ns | 0 | 0 | 0 | 4.31 KB |
| #7928 | EnrichedLog |
net472 | 166μs | 35.5ns | 137ns | 0 | 0 | 0 | 4.51 KB |
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 4.94μs | 3.91ns | 15.2ns | 0 | 0 | 0 | 2.24 KB |
| master | EnrichedLog |
netcoreapp3.1 | 6.87μs | 19.4ns | 75.2ns | 0 | 0 | 0 | 2.26 KB |
| master | EnrichedLog |
net472 | 7.66μs | 9.95ns | 38.5ns | 0.304 | 0 | 0 | 2.05 KB |
| #7928 | EnrichedLog |
net6.0 | 5.11μs | 3.67ns | 13.2ns | 0 | 0 | 0 | 2.24 KB |
| #7928 | EnrichedLog |
netcoreapp3.1 | 6.88μs | 20.6ns | 79.8ns | 0 | 0 | 0 | 2.26 KB |
| #7928 | EnrichedLog |
net472 | 7.61μs | 9.81ns | 38ns | 0.301 | 0 | 0 | 2.05 KB |
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendReceive |
net6.0 | 1.94μs | 0.737ns | 2.66ns | 0 | 0 | 0 | 1.12 KB |
| master | SendReceive |
netcoreapp3.1 | 2.48μs | 12.8ns | 58.8ns | 0 | 0 | 0 | 1.14 KB |
| master | SendReceive |
net472 | 3.02μs | 2.33ns | 9.02ns | 0.167 | 0 | 0 | 1.12 KB |
| #7928 | SendReceive |
net6.0 | 1.94μs | 3.9ns | 15.1ns | 0 | 0 | 0 | 1.12 KB |
| #7928 | SendReceive |
netcoreapp3.1 | 2.5μs | 9.97ns | 37.3ns | 0 | 0 | 0 | 1.14 KB |
| #7928 | SendReceive |
net472 | 3μs | 1.27ns | 4.9ns | 0.165 | 0 | 0 | 1.12 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 4.4μs | 3.64ns | 13.1ns | 0 | 0 | 0 | 1.58 KB |
| master | EnrichedLog |
netcoreapp3.1 | 5.82μs | 12.8ns | 49.7ns | 0 | 0 | 0 | 1.63 KB |
| master | EnrichedLog |
net472 | 6.79μs | 5.79ns | 22.4ns | 0.307 | 0 | 0 | 2.03 KB |
| #7928 | EnrichedLog |
net6.0 | 4.41μs | 7.1ns | 27.5ns | 0 | 0 | 0 | 1.58 KB |
| #7928 | EnrichedLog |
netcoreapp3.1 | 5.9μs | 8.1ns | 31.4ns | 0 | 0 | 0 | 1.63 KB |
| #7928 | EnrichedLog |
net472 | 6.69μs | 7.34ns | 28.4ns | 0.299 | 0 | 0 | 2.03 KB |
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartFinishSpan |
net6.0 | 753ns | 3.75ns | 15.5ns | 0 | 0 | 0 | 520 B |
| master | StartFinishSpan |
netcoreapp3.1 | 954ns | 4.7ns | 20.5ns | 0 | 0 | 0 | 520 B |
| master | StartFinishSpan |
net472 | 914ns | 0.0772ns | 0.278ns | 0.0824 | 0 | 0 | 522 B |
| master | StartFinishScope |
net6.0 | 897ns | 4.54ns | 20.3ns | 0 | 0 | 0 | 640 B |
| master | StartFinishScope |
netcoreapp3.1 | 1.12μs | 5.72ns | 21.4ns | 0 | 0 | 0 | 640 B |
| master | StartFinishScope |
net472 | 1.11μs | 1.26ns | 4.72ns | 0.0945 | 0 | 0 | 602 B |
| master | StartFinishTwoScopes |
net6.0 | 1.88μs | 0.685ns | 2.65ns | 0 | 0 | 0 | 1.19 KB |
| master | StartFinishTwoScopes |
netcoreapp3.1 | 2.22μs | 9.96ns | 38.6ns | 0 | 0 | 0 | 1.19 KB |
| master | StartFinishTwoScopes |
net472 | 2.14μs | 4.64ns | 18ns | 0.16 | 0 | 0 | 1.08 KB |
| #7928 | StartFinishSpan |
net6.0 | 761ns | 0.904ns | 3.5ns | 0 | 0 | 0 | 520 B |
| #7928 | StartFinishSpan |
netcoreapp3.1 | 944ns | 4.74ns | 22.7ns | 0 | 0 | 0 | 520 B |
| #7928 | StartFinishSpan |
net472 | 887ns | 0.114ns | 0.443ns | 0.08 | 0 | 0 | 522 B |
| #7928 | StartFinishScope |
net6.0 | 915ns | 4.87ns | 25.8ns | 0 | 0 | 0 | 640 B |
| #7928 | StartFinishScope |
netcoreapp3.1 | 1.13μs | 5.62ns | 23.8ns | 0 | 0 | 0 | 640 B |
| #7928 | StartFinishScope |
net472 | 1.06μs | 0.256ns | 0.99ns | 0.0906 | 0 | 0 | 602 B |
| #7928 | StartFinishTwoScopes |
net6.0 | 1.79μs | 0.876ns | 3.39ns | 0 | 0 | 0 | 1.19 KB |
| #7928 | StartFinishTwoScopes |
netcoreapp3.1 | 2.16μs | 3.01ns | 11.7ns | 0 | 0 | 0 | 1.19 KB |
| #7928 | StartFinishTwoScopes |
net472 | 2.1μs | 1.8ns | 6.96ns | 0.169 | 0 | 0 | 1.08 KB |
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunOnMethodBegin |
net6.0 | 1.05μs | 5.38ns | 26.4ns | 0 | 0 | 0 | 640 B |
| master | RunOnMethodBegin |
netcoreapp3.1 | 1.33μs | 6.49ns | 26.8ns | 0 | 0 | 0 | 640 B |
| master | RunOnMethodBegin |
net472 | 1.42μs | 0.772ns | 2.79ns | 0.0926 | 0 | 0 | 602 B |
| #7928 | RunOnMethodBegin |
net6.0 | 1.08μs | 3.01ns | 11.6ns | 0 | 0 | 0 | 640 B |
| #7928 | RunOnMethodBegin |
netcoreapp3.1 | 1.35μs | 6.41ns | 24.8ns | 0 | 0 | 0 | 640 B |
| #7928 | RunOnMethodBegin |
net472 | 1.41μs | 0.969ns | 3.75ns | 0.0915 | 0 | 0 | 602 B |
44c5c90 to
a3ff9fa
Compare
7a65ba3 to
5e464c3
Compare
01d898b
into
augusto/use-dd-octo-sts-on-windows-benchmarks

HARDCODED_BUILD_IDis set to""before merging.Summary of changes
.setuphidden job and its referencesmicrobenchmarks.yml.Reason for change
Improve readability on dd-trace-dotnet's macrobenchmark AMI build jobs.
Implementation details
Test coverage
AMI build job correctly running: https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-dotnet/-/jobs/1283243374
Other details