Fix broken CI when running tests against a specific sample#7819
Fix broken CI when running tests against a specific sample#7819andrewlock merged 4 commits intomasterfrom
Conversation
This reverts commit 2eb2dbc.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Execution-Time Benchmarks Report ⏱️Execution-time results for samples comparing This PR (7819) and master. ✅ No regressions detected - check the details below Full Metrics ComparisonFakeDbCommand
HttpMessageHandler
Comparison explanationExecution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:
Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard. Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph). Duration chartsFakeDbCommand (.NET Framework 4.8)gantt
title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7819) - mean (75ms) : 69, 80
master - mean (76ms) : 70, 81
section Bailout
This PR (7819) - mean (79ms) : 73, 84
master - mean (80ms) : 75, 85
section CallTarget+Inlining+NGEN
This PR (7819) - mean (1,135ms) : 1036, 1233
master - mean (1,136ms) : 1035, 1237
FakeDbCommand (.NET Core 3.1)gantt
title Execution time (ms) FakeDbCommand (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7819) - mean (118ms) : 112, 125
master - mean (118ms) : 112, 125
section Bailout
This PR (7819) - mean (122ms) : 112, 131
master - mean (118ms) : 111, 124
section CallTarget+Inlining+NGEN
This PR (7819) - mean (813ms) : 775, 851
master - mean (806ms) : 769, 842
FakeDbCommand (.NET 6)gantt
title Execution time (ms) FakeDbCommand (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7819) - mean (108ms) : 101, 114
master - mean (105ms) : 96, 113
section Bailout
This PR (7819) - mean (108ms) : 100, 115
master - mean (105ms) : 100, 111
section CallTarget+Inlining+NGEN
This PR (7819) - mean (759ms) : 715, 804
master - mean (755ms) : 715, 795
FakeDbCommand (.NET 8)gantt
title Execution time (ms) FakeDbCommand (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7819) - mean (104ms) : 95, 113
master - mean (103ms) : 98, 109
section Bailout
This PR (7819) - mean (106ms) : 97, 114
master - mean (107ms) : 99, 114
section CallTarget+Inlining+NGEN
This PR (7819) - mean (726ms) : 698, 754
master - mean (717ms) : 690, 743
HttpMessageHandler (.NET Framework 4.8)gantt
title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7819) - mean (196ms) : 189, 203
master - mean (194ms) : 187, 200
section Bailout
This PR (7819) - mean (200ms) : 196, 205
master - mean (196ms) : 193, 200
section CallTarget+Inlining+NGEN
This PR (7819) - mean (1,190ms) : 1115, 1265
master - mean (1,177ms) : 1113, 1240
HttpMessageHandler (.NET Core 3.1)gantt
title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7819) - mean (282ms) : 275, 290
master - mean (276ms) : 271, 281
section Bailout
This PR (7819) - mean (282ms) : 274, 290
master - mean (277ms) : 273, 281
section CallTarget+Inlining+NGEN
This PR (7819) - mean (949ms) : 895, 1003
master - mean (943ms) : 895, 992
HttpMessageHandler (.NET 6)gantt
title Execution time (ms) HttpMessageHandler (.NET 6)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7819) - mean (271ms) : 267, 276
master - mean (270ms) : 264, 276
section Bailout
This PR (7819) - mean (270ms) : 267, 274
master - mean (270ms) : 266, 274
section CallTarget+Inlining+NGEN
This PR (7819) - mean (928ms) : 872, 984
master - mean (926ms) : 872, 981
HttpMessageHandler (.NET 8)gantt
title Execution time (ms) HttpMessageHandler (.NET 8)
dateFormat x
axisFormat %Q
todayMarker off
section Baseline
This PR (7819) - mean (270ms) : 264, 276
master - mean (268ms) : 264, 273
section Bailout
This PR (7819) - mean (268ms) : 265, 272
master - mean (268ms) : 265, 271
section CallTarget+Inlining+NGEN
This PR (7819) - mean (854ms) : 829, 879
master - mean (860ms) : 838, 882
|
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
BenchmarksBenchmarks Report for benchmark platform 🐌Benchmarks for #7819 compared to master:
The following thresholds were used for comparing the benchmark speeds:
Allocation changes below 0.5% are ignored. Benchmark detailsBenchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations
|
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1 | 5.64 KB | 5.73 KB | 93 B | 1.65% |
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 | 6.06 KB | 6 KB | -61 B | -1.01% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartStopWithChild |
net6.0 | 10.8μs | 56.8ns | 284ns | 0 | 0 | 0 | 5.5 KB |
| master | StartStopWithChild |
netcoreapp3.1 | 14.1μs | 68.2ns | 273ns | 0 | 0 | 0 | 5.64 KB |
| master | StartStopWithChild |
net472 | 22.8μs | 124ns | 688ns | 1.03 | 0.31 | 0.103 | 6.06 KB |
| #7819 | StartStopWithChild |
net6.0 | 10.4μs | 55ns | 275ns | 0 | 0 | 0 | 5.51 KB |
| #7819 | StartStopWithChild |
netcoreapp3.1 | 13.5μs | 72.2ns | 361ns | 0 | 0 | 0 | 5.73 KB |
| #7819 | StartStopWithChild |
net472 | 21.7μs | 119ns | 711ns | 0.982 | 0.327 | 0.109 | 6 KB |
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ More allocations ⚠️
More allocations ⚠️ in #7819
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472
3.31 KB
3.35 KB
46 B
1.39%
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 | 3.31 KB | 3.35 KB | 46 B | 1.39% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | WriteAndFlushEnrichedTraces |
net6.0 | 931μs | 137ns | 493ns | 0 | 0 | 0 | 2.71 KB |
| master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 1.04ms | 292ns | 1.09μs | 0 | 0 | 0 | 2.7 KB |
| master | WriteAndFlushEnrichedTraces |
net472 | 1.21ms | 836ns | 3.13μs | 0 | 0 | 0 | 3.31 KB |
| #7819 | WriteAndFlushEnrichedTraces |
net6.0 | 964μs | 183ns | 710ns | 0 | 0 | 0 | 2.71 KB |
| #7819 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 1.03ms | 173ns | 669ns | 0 | 0 | 0 | 2.7 KB |
| #7819 | WriteAndFlushEnrichedTraces |
net472 | 1.22ms | 1.14μs | 4.41μs | 0 | 0 | 0 | 3.35 KB |
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | AllCycleSimpleBody |
net6.0 | 1.07μs | 6ns | 38.4ns | 0 | 0 | 0 | 1.22 KB |
| master | AllCycleSimpleBody |
netcoreapp3.1 | 1.4μs | 7.78ns | 44ns | 0 | 0 | 0 | 1.2 KB |
| master | AllCycleSimpleBody |
net472 | 1.04μs | 0.458ns | 1.71ns | 0.191 | 0 | 0 | 1.23 KB |
| master | AllCycleMoreComplexBody |
net6.0 | 7.24μs | 35.2ns | 149ns | 0 | 0 | 0 | 4.72 KB |
| master | AllCycleMoreComplexBody |
netcoreapp3.1 | 8.87μs | 45.4ns | 208ns | 0 | 0 | 0 | 4.62 KB |
| master | AllCycleMoreComplexBody |
net472 | 7.76μs | 2.61ns | 9.77ns | 0.741 | 0 | 0 | 4.74 KB |
| master | ObjectExtractorSimpleBody |
net6.0 | 319ns | 1.58ns | 7.26ns | 0 | 0 | 0 | 280 B |
| master | ObjectExtractorSimpleBody |
netcoreapp3.1 | 400ns | 2.08ns | 10ns | 0 | 0 | 0 | 272 B |
| master | ObjectExtractorSimpleBody |
net472 | 297ns | 0.0475ns | 0.184ns | 0.044 | 0 | 0 | 281 B |
| master | ObjectExtractorMoreComplexBody |
net6.0 | 6.37μs | 7.77ns | 30.1ns | 0 | 0 | 0 | 3.78 KB |
| master | ObjectExtractorMoreComplexBody |
netcoreapp3.1 | 7.83μs | 37.6ns | 145ns | 0 | 0 | 0 | 3.69 KB |
| master | ObjectExtractorMoreComplexBody |
net472 | 6.68μs | 2.07ns | 8ns | 0.603 | 0 | 0 | 3.8 KB |
| #7819 | AllCycleSimpleBody |
net6.0 | 1.05μs | 6.13ns | 57.2ns | 0 | 0 | 0 | 1.22 KB |
| #7819 | AllCycleSimpleBody |
netcoreapp3.1 | 1.44μs | 7.82ns | 47.6ns | 0 | 0 | 0 | 1.2 KB |
| #7819 | AllCycleSimpleBody |
net472 | 1.03μs | 0.639ns | 2.48ns | 0.192 | 0 | 0 | 1.23 KB |
| #7819 | AllCycleMoreComplexBody |
net6.0 | 7.07μs | 32.7ns | 127ns | 0 | 0 | 0 | 4.72 KB |
| #7819 | AllCycleMoreComplexBody |
netcoreapp3.1 | 9.04μs | 35.6ns | 138ns | 0 | 0 | 0 | 4.62 KB |
| #7819 | AllCycleMoreComplexBody |
net472 | 7.64μs | 3.49ns | 13.5ns | 0.727 | 0 | 0 | 4.74 KB |
| #7819 | ObjectExtractorSimpleBody |
net6.0 | 318ns | 0.0869ns | 0.301ns | 0 | 0 | 0 | 280 B |
| #7819 | ObjectExtractorSimpleBody |
netcoreapp3.1 | 400ns | 2.19ns | 12.4ns | 0 | 0 | 0 | 272 B |
| #7819 | ObjectExtractorSimpleBody |
net472 | 301ns | 0.0358ns | 0.134ns | 0.044 | 0 | 0 | 281 B |
| #7819 | ObjectExtractorMoreComplexBody |
net6.0 | 6.31μs | 30.2ns | 117ns | 0 | 0 | 0 | 3.78 KB |
| #7819 | ObjectExtractorMoreComplexBody |
netcoreapp3.1 | 7.89μs | 27.7ns | 107ns | 0 | 0 | 0 | 3.69 KB |
| #7819 | ObjectExtractorMoreComplexBody |
net472 | 6.73μs | 7.82ns | 30.3ns | 0.601 | 0 | 0 | 3.8 KB |
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EncodeArgs |
net6.0 | 76.6μs | 162ns | 628ns | 0 | 0 | 0 | 32.4 KB |
| master | EncodeArgs |
netcoreapp3.1 | 96.9μs | 283ns | 1.1μs | 0 | 0 | 0 | 32.4 KB |
| master | EncodeArgs |
net472 | 109μs | 49.3ns | 191ns | 4.92 | 0 | 0 | 32.51 KB |
| master | EncodeLegacyArgs |
net6.0 | 146μs | 53.5ns | 207ns | 0 | 0 | 0 | 2.15 KB |
| master | EncodeLegacyArgs |
netcoreapp3.1 | 197μs | 208ns | 806ns | 0 | 0 | 0 | 2.14 KB |
| master | EncodeLegacyArgs |
net472 | 262μs | 119ns | 460ns | 0 | 0 | 0 | 2.16 KB |
| #7819 | EncodeArgs |
net6.0 | 77.2μs | 23ns | 89ns | 0 | 0 | 0 | 32.4 KB |
| #7819 | EncodeArgs |
netcoreapp3.1 | 98.3μs | 311ns | 1.2μs | 0 | 0 | 0 | 32.4 KB |
| #7819 | EncodeArgs |
net472 | 109μs | 16.4ns | 63.5ns | 4.89 | 0 | 0 | 32.51 KB |
| #7819 | EncodeLegacyArgs |
net6.0 | 146μs | 48.8ns | 176ns | 0 | 0 | 0 | 2.14 KB |
| #7819 | EncodeLegacyArgs |
netcoreapp3.1 | 197μs | 99.2ns | 371ns | 0 | 0 | 0 | 2.14 KB |
| #7819 | EncodeLegacyArgs |
net472 | 263μs | 19.1ns | 71.3ns | 0 | 0 | 0 | 2.17 KB |
Benchmarks.Trace.Asm.AppSecWafBenchmark - Faster 🎉 Same allocations ✔️
Faster 🎉 in #7819
Benchmark
base/diff
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark‑netcoreapp3.1
2.026
853,385.94
421,204.17
| Benchmark | base/diff | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmark‑netcoreapp3.1 | 2.026 | 853,385.94 | 421,204.17 |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunWafRealisticBenchmark |
net6.0 | 392μs | 38.3ns | 148ns | 0 | 0 | 0 | 4.55 KB |
| master | RunWafRealisticBenchmark |
netcoreapp3.1 | 850μs | 3.84μs | 14.9μs | 0 | 0 | 0 | 4.48 KB |
| master | RunWafRealisticBenchmark |
net472 | 433μs | 64.1ns | 240ns | 0 | 0 | 0 | 4.66 KB |
| master | RunWafRealisticBenchmarkWithAttack |
net6.0 | 283μs | 11ns | 41.3ns | 0 | 0 | 0 | 2.24 KB |
| master | RunWafRealisticBenchmarkWithAttack |
netcoreapp3.1 | 295μs | 124ns | 429ns | 0 | 0 | 0 | 2.22 KB |
| master | RunWafRealisticBenchmarkWithAttack |
net472 | 309μs | 29.6ns | 115ns | 0 | 0 | 0 | 2.29 KB |
| #7819 | RunWafRealisticBenchmark |
net6.0 | 392μs | 40.6ns | 152ns | 0 | 0 | 0 | 4.55 KB |
| #7819 | RunWafRealisticBenchmark |
netcoreapp3.1 | 421μs | 323ns | 1.17μs | 0 | 0 | 0 | 4.48 KB |
| #7819 | RunWafRealisticBenchmark |
net472 | 429μs | 64.7ns | 250ns | 0 | 0 | 0 | 4.66 KB |
| #7819 | RunWafRealisticBenchmarkWithAttack |
net6.0 | 285μs | 29.6ns | 115ns | 0 | 0 | 0 | 2.24 KB |
| #7819 | RunWafRealisticBenchmarkWithAttack |
netcoreapp3.1 | 299μs | 30.4ns | 105ns | 0 | 0 | 0 | 2.22 KB |
| #7819 | RunWafRealisticBenchmarkWithAttack |
net472 | 311μs | 63.2ns | 237ns | 0 | 0 | 0 | 2.29 KB |
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendRequest |
net6.0 | 61.5μs | 71.3ns | 267ns | 0 | 0 | 0 | 14.52 KB |
| master | SendRequest |
netcoreapp3.1 | 72.3μs | 67.8ns | 254ns | 0 | 0 | 0 | 17.42 KB |
| master | SendRequest |
net472 | 0.00183ns | 0.00127ns | 0.00491ns | 0 | 0 | 0 | 0 b |
| #7819 | SendRequest |
net6.0 | 61.6μs | 133ns | 514ns | 0 | 0 | 0 | 14.52 KB |
| #7819 | SendRequest |
netcoreapp3.1 | 73μs | 328ns | 1.27μs | 0 | 0 | 0 | 17.42 KB |
| #7819 | SendRequest |
net472 | 0.0045ns | 0.00121ns | 0.00467ns | 0 | 0 | 0 | 0 b |
Benchmarks.Trace.CharSliceBenchmark - Same speed ✔️ More allocations ⚠️
More allocations ⚠️ in #7819
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net6.0
1 B
3 B
2 B
200.00%
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net6.0 | 1 B | 3 B | 2 B | 200.00% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | OriginalCharSlice |
net6.0 | 1.88ms | 3.05μs | 11.8μs | 0 | 0 | 0 | 640.01 KB |
| master | OriginalCharSlice |
netcoreapp3.1 | 2.11ms | 4.21μs | 16.3μs | 0 | 0 | 0 | 640 KB |
| master | OriginalCharSlice |
net472 | 2.68ms | 272ns | 1.05μs | 100 | 0 | 0 | 641.95 KB |
| master | OptimizedCharSlice |
net6.0 | 1.37ms | 135ns | 507ns | 0 | 0 | 0 | 4 B |
| master | OptimizedCharSlice |
netcoreapp3.1 | 1.69ms | 399ns | 1.54μs | 0 | 0 | 0 | 1 B |
| master | OptimizedCharSlice |
net472 | 1.95ms | 233ns | 903ns | 0 | 0 | 0 | 0 b |
| master | OptimizedCharSliceWithPool |
net6.0 | 874μs | 69.5ns | 260ns | 0 | 0 | 0 | 1 B |
| master | OptimizedCharSliceWithPool |
netcoreapp3.1 | 829μs | 71.1ns | 256ns | 0 | 0 | 0 | 0 b |
| master | OptimizedCharSliceWithPool |
net472 | 1.2ms | 280ns | 1.08μs | 0 | 0 | 0 | 0 b |
| #7819 | OriginalCharSlice |
net6.0 | 1.92ms | 354ns | 1.37μs | 0 | 0 | 0 | 640 KB |
| #7819 | OriginalCharSlice |
netcoreapp3.1 | 2.1ms | 5.35μs | 20.7μs | 0 | 0 | 0 | 640 KB |
| #7819 | OriginalCharSlice |
net472 | 2.56ms | 108ns | 402ns | 100 | 0 | 0 | 641.95 KB |
| #7819 | OptimizedCharSlice |
net6.0 | 1.44ms | 233ns | 902ns | 0 | 0 | 0 | 4 B |
| #7819 | OptimizedCharSlice |
netcoreapp3.1 | 1.72ms | 183ns | 707ns | 0 | 0 | 0 | 1 B |
| #7819 | OptimizedCharSlice |
net472 | 2ms | 233ns | 903ns | 0 | 0 | 0 | 0 b |
| #7819 | OptimizedCharSliceWithPool |
net6.0 | 832μs | 35.2ns | 136ns | 0 | 0 | 0 | 3 B |
| #7819 | OptimizedCharSliceWithPool |
netcoreapp3.1 | 820μs | 75.6ns | 293ns | 0 | 0 | 0 | 0 b |
| #7819 | OptimizedCharSliceWithPool |
net472 | 1.13ms | 99.7ns | 386ns | 0 | 0 | 0 | 0 b |
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Slower ⚠️ Fewer allocations 🎉
Slower ⚠️ in #7819
Benchmark
diff/base
Base Median (ns)
Diff Median (ns)
Modality
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1
1.131
664,425.59
751,303.96
several?
Fewer allocations 🎉 in #7819
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472
56.3 KB
55.81 KB
-492 B
-0.87%
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0
42.38 KB
41.66 KB
-721 B
-1.70%
| Benchmark | diff/base | Base Median (ns) | Diff Median (ns) | Modality |
|---|---|---|---|---|
| Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 | 1.131 | 664,425.59 | 751,303.96 | several? |
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 | 56.3 KB | 55.81 KB | -492 B | -0.87% |
| Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 | 42.38 KB | 41.66 KB | -721 B | -1.70% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | WriteAndFlushEnrichedTraces |
net6.0 | 765μs | 3.49μs | 13μs | 0 | 0 | 0 | 42.38 KB |
| master | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 676μs | 2.61μs | 19.2μs | 0 | 0 | 0 | 41.88 KB |
| master | WriteAndFlushEnrichedTraces |
net472 | 957μs | 4.68μs | 20.9μs | 4.81 | 0 | 0 | 56.3 KB |
| #7819 | WriteAndFlushEnrichedTraces |
net6.0 | 696μs | 3.14μs | 12.2μs | 0 | 0 | 0 | 41.66 KB |
| #7819 | WriteAndFlushEnrichedTraces |
netcoreapp3.1 | 750μs | 4.28μs | 30.5μs | 0 | 0 | 0 | 41.92 KB |
| #7819 | WriteAndFlushEnrichedTraces |
net472 | 968μs | 5.39μs | 32.8μs | 8.33 | 0 | 0 | 55.81 KB |
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteNonQuery |
net6.0 | 2μs | 2.23ns | 8.33ns | 0 | 0 | 0 | 1.02 KB |
| master | ExecuteNonQuery |
netcoreapp3.1 | 2.7μs | 4.29ns | 16.1ns | 0 | 0 | 0 | 1.02 KB |
| master | ExecuteNonQuery |
net472 | 2.81μs | 3.78ns | 14.6ns | 0.155 | 0.0141 | 0 | 987 B |
| #7819 | ExecuteNonQuery |
net6.0 | 1.89μs | 4.37ns | 16.9ns | 0 | 0 | 0 | 1.02 KB |
| #7819 | ExecuteNonQuery |
netcoreapp3.1 | 2.65μs | 13.4ns | 58.5ns | 0 | 0 | 0 | 1.02 KB |
| #7819 | ExecuteNonQuery |
net472 | 2.79μs | 2.83ns | 11ns | 0.154 | 0.014 | 0 | 987 B |
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | CallElasticsearch |
net6.0 | 1.67μs | 9.01ns | 49.3ns | 0 | 0 | 0 | 1.03 KB |
| master | CallElasticsearch |
netcoreapp3.1 | 2.27μs | 1.34ns | 5ns | 0 | 0 | 0 | 1.03 KB |
| master | CallElasticsearch |
net472 | 3.47μs | 3.16ns | 12.3ns | 0.156 | 0 | 0 | 1.04 KB |
| master | CallElasticsearchAsync |
net6.0 | 1.88μs | 6.98ns | 27ns | 0 | 0 | 0 | 1.01 KB |
| master | CallElasticsearchAsync |
netcoreapp3.1 | 2.45μs | 11ns | 42.6ns | 0 | 0 | 0 | 1.08 KB |
| master | CallElasticsearchAsync |
net472 | 3.69μs | 1.72ns | 6.44ns | 0.166 | 0 | 0 | 1.1 KB |
| #7819 | CallElasticsearch |
net6.0 | 1.7μs | 0.447ns | 1.67ns | 0 | 0 | 0 | 1.03 KB |
| #7819 | CallElasticsearch |
netcoreapp3.1 | 2.22μs | 10.9ns | 46.4ns | 0 | 0 | 0 | 1.03 KB |
| #7819 | CallElasticsearch |
net472 | 3.46μs | 2.2ns | 8.54ns | 0.155 | 0 | 0 | 1.04 KB |
| #7819 | CallElasticsearchAsync |
net6.0 | 1.79μs | 9.54ns | 48.7ns | 0 | 0 | 0 | 1.01 KB |
| #7819 | CallElasticsearchAsync |
netcoreapp3.1 | 2.42μs | 11.1ns | 42.8ns | 0 | 0 | 0 | 1.08 KB |
| #7819 | CallElasticsearchAsync |
net472 | 3.65μs | 6.58ns | 25.5ns | 0.165 | 0 | 0 | 1.1 KB |
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | ExecuteAsync |
net6.0 | 1.85μs | 9.2ns | 41.2ns | 0 | 0 | 0 | 952 B |
| master | ExecuteAsync |
netcoreapp3.1 | 2.5μs | 9.47ns | 36.7ns | 0 | 0 | 0 | 952 B |
| master | ExecuteAsync |
net472 | 2.7μs | 4.34ns | 16.8ns | 0.135 | 0 | 0 | 915 B |
| #7819 | ExecuteAsync |
net6.0 | 1.85μs | 1.35ns | 5.23ns | 0 | 0 | 0 | 952 B |
| #7819 | ExecuteAsync |
netcoreapp3.1 | 2.41μs | 11.5ns | 44.7ns | 0 | 0 | 0 | 952 B |
| #7819 | ExecuteAsync |
net472 | 2.6μs | 1.78ns | 6.65ns | 0.143 | 0 | 0 | 915 B |
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendAsync |
net6.0 | 6.69μs | 14.9ns | 57.8ns | 0 | 0 | 0 | 2.36 KB |
| master | SendAsync |
netcoreapp3.1 | 8.71μs | 23.7ns | 82.2ns | 0 | 0 | 0 | 2.9 KB |
| master | SendAsync |
net472 | 12.5μs | 12.9ns | 49.9ns | 0.498 | 0 | 0 | 3.18 KB |
| #7819 | SendAsync |
net6.0 | 6.9μs | 21.4ns | 80.2ns | 0 | 0 | 0 | 2.36 KB |
| #7819 | SendAsync |
netcoreapp3.1 | 8.64μs | 24.6ns | 95.2ns | 0 | 0 | 0 | 2.9 KB |
| #7819 | SendAsync |
net472 | 12.5μs | 11ns | 42.6ns | 0.5 | 0 | 0 | 3.18 KB |
Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️
More allocations ⚠️ in #7819
Benchmark
Base Allocated
Diff Allocated
Change
Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472
57.34 KB
65.54 KB
8.19 KB
14.29%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472
278.53 KB
286.72 KB
8.19 KB
2.94%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0
43.14 KB
43.44 KB
296 B
0.69%
| Benchmark | Base Allocated | Diff Allocated | Change | Change % |
|---|---|---|---|---|
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 | 57.34 KB | 65.54 KB | 8.19 KB | 14.29% |
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 | 278.53 KB | 286.72 KB | 8.19 KB | 2.94% |
| Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 | 43.14 KB | 43.44 KB | 296 B | 0.69% |
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StringConcatBenchmark |
net6.0 | 42.1μs | 210ns | 864ns | 0 | 0 | 0 | 43.14 KB |
| master | StringConcatBenchmark |
netcoreapp3.1 | 50.3μs | 287ns | 2.58μs | 0 | 0 | 0 | 42.64 KB |
| master | StringConcatBenchmark |
net472 | 56.5μs | 270ns | 1.04μs | 0 | 0 | 0 | 57.34 KB |
| master | StringConcatAspectBenchmark |
net6.0 | 477μs | 1.51μs | 5.46μs | 0 | 0 | 0 | 276.5 KB |
| master | StringConcatAspectBenchmark |
netcoreapp3.1 | 494μs | 1.68μs | 6.04μs | 0 | 0 | 0 | 254.57 KB |
| master | StringConcatAspectBenchmark |
net472 | 407μs | 2.13μs | 10.8μs | 0 | 0 | 0 | 278.53 KB |
| #7819 | StringConcatBenchmark |
net6.0 | 46.4μs | 299ns | 2.82μs | 0 | 0 | 0 | 43.44 KB |
| #7819 | StringConcatBenchmark |
netcoreapp3.1 | 50.5μs | 279ns | 2.01μs | 0 | 0 | 0 | 42.64 KB |
| #7819 | StringConcatBenchmark |
net472 | 57.7μs | 261ns | 1.01μs | 0 | 0 | 0 | 65.54 KB |
| #7819 | StringConcatAspectBenchmark |
net6.0 | 489μs | 2.16μs | 7.78μs | 0 | 0 | 0 | 276.62 KB |
| #7819 | StringConcatAspectBenchmark |
netcoreapp3.1 | 510μs | 1.84μs | 7.35μs | 0 | 0 | 0 | 255.62 KB |
| #7819 | StringConcatAspectBenchmark |
net472 | 402μs | 2.02μs | 8.55μs | 0 | 0 | 0 | 286.72 KB |
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 2.64μs | 10.9ns | 42ns | 0 | 0 | 0 | 1.7 KB |
| master | EnrichedLog |
netcoreapp3.1 | 3.64μs | 16ns | 62.1ns | 0 | 0 | 0 | 1.7 KB |
| master | EnrichedLog |
net472 | 3.87μs | 4.3ns | 16.6ns | 0.252 | 0 | 0 | 1.64 KB |
| #7819 | EnrichedLog |
net6.0 | 2.72μs | 3.58ns | 13.9ns | 0 | 0 | 0 | 1.7 KB |
| #7819 | EnrichedLog |
netcoreapp3.1 | 3.68μs | 18.4ns | 78ns | 0 | 0 | 0 | 1.7 KB |
| #7819 | EnrichedLog |
net472 | 4.03μs | 3.99ns | 15.4ns | 0.241 | 0 | 0 | 1.64 KB |
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 124μs | 70ns | 262ns | 0 | 0 | 0 | 4.31 KB |
| master | EnrichedLog |
netcoreapp3.1 | 127μs | 213ns | 738ns | 0 | 0 | 0 | 4.31 KB |
| master | EnrichedLog |
net472 | 169μs | 55.1ns | 206ns | 0 | 0 | 0 | 4.52 KB |
| #7819 | EnrichedLog |
net6.0 | 122μs | 119ns | 446ns | 0 | 0 | 0 | 4.31 KB |
| #7819 | EnrichedLog |
netcoreapp3.1 | 129μs | 209ns | 783ns | 0 | 0 | 0 | 4.31 KB |
| #7819 | EnrichedLog |
net472 | 168μs | 207ns | 800ns | 0 | 0 | 0 | 4.52 KB |
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 5.09μs | 20ns | 77.3ns | 0 | 0 | 0 | 2.26 KB |
| master | EnrichedLog |
netcoreapp3.1 | 6.88μs | 18.3ns | 71ns | 0 | 0 | 0 | 2.26 KB |
| master | EnrichedLog |
net472 | 7.52μs | 7.8ns | 30.2ns | 0.3 | 0 | 0 | 2.08 KB |
| #7819 | EnrichedLog |
net6.0 | 5.07μs | 1.57ns | 6.09ns | 0 | 0 | 0 | 2.26 KB |
| #7819 | EnrichedLog |
netcoreapp3.1 | 6.94μs | 16.1ns | 62.2ns | 0 | 0 | 0 | 2.26 KB |
| #7819 | EnrichedLog |
net472 | 7.94μs | 12.1ns | 46.9ns | 0.319 | 0 | 0 | 2.08 KB |
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | SendReceive |
net6.0 | 2.08μs | 8.84ns | 34.2ns | 0 | 0 | 0 | 1.2 KB |
| master | SendReceive |
netcoreapp3.1 | 2.62μs | 12.2ns | 47.2ns | 0 | 0 | 0 | 1.2 KB |
| master | SendReceive |
net472 | 3.01μs | 4.12ns | 16ns | 0.181 | 0 | 0 | 1.2 KB |
| #7819 | SendReceive |
net6.0 | 1.95μs | 9ns | 33.7ns | 0 | 0 | 0 | 1.2 KB |
| #7819 | SendReceive |
netcoreapp3.1 | 2.64μs | 10.7ns | 41.3ns | 0 | 0 | 0 | 1.2 KB |
| #7819 | SendReceive |
net472 | 2.97μs | 1.69ns | 5.84ns | 0.179 | 0 | 0 | 1.2 KB |
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | EnrichedLog |
net6.0 | 4.35μs | 11.9ns | 46ns | 0 | 0 | 0 | 1.58 KB |
| master | EnrichedLog |
netcoreapp3.1 | 5.76μs | 3.84ns | 14.4ns | 0 | 0 | 0 | 1.63 KB |
| master | EnrichedLog |
net472 | 6.66μs | 11.9ns | 46ns | 0.301 | 0 | 0 | 2.03 KB |
| #7819 | EnrichedLog |
net6.0 | 4.38μs | 15.4ns | 59.5ns | 0 | 0 | 0 | 1.58 KB |
| #7819 | EnrichedLog |
netcoreapp3.1 | 5.59μs | 21ns | 81.4ns | 0 | 0 | 0 | 1.63 KB |
| #7819 | EnrichedLog |
net472 | 6.73μs | 9.75ns | 37.8ns | 0.301 | 0 | 0 | 2.03 KB |
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | StartFinishSpan |
net6.0 | 777ns | 0.35ns | 1.31ns | 0 | 0 | 0 | 576 B |
| master | StartFinishSpan |
netcoreapp3.1 | 1.01μs | 0.635ns | 2.46ns | 0 | 0 | 0 | 576 B |
| master | StartFinishSpan |
net472 | 964ns | 0.266ns | 1.03ns | 0.0917 | 0 | 0 | 578 B |
| master | StartFinishScope |
net6.0 | 967ns | 0.271ns | 1.05ns | 0 | 0 | 0 | 696 B |
| master | StartFinishScope |
netcoreapp3.1 | 1.21μs | 1.71ns | 6.64ns | 0 | 0 | 0 | 696 B |
| master | StartFinishScope |
net472 | 1.14μs | 0.618ns | 2.39ns | 0.103 | 0 | 0 | 658 B |
| #7819 | StartFinishSpan |
net6.0 | 783ns | 3.9ns | 17.9ns | 0 | 0 | 0 | 576 B |
| #7819 | StartFinishSpan |
netcoreapp3.1 | 958ns | 4.34ns | 16.8ns | 0 | 0 | 0 | 576 B |
| #7819 | StartFinishSpan |
net472 | 941ns | 0.386ns | 1.49ns | 0.0897 | 0 | 0 | 578 B |
| #7819 | StartFinishScope |
net6.0 | 918ns | 3ns | 11.6ns | 0 | 0 | 0 | 696 B |
| #7819 | StartFinishScope |
netcoreapp3.1 | 1.19μs | 6.18ns | 28.3ns | 0 | 0 | 0 | 696 B |
| #7819 | StartFinishScope |
net472 | 1.14μs | 0.241ns | 0.935ns | 0.102 | 0 | 0 | 658 B |
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️
Raw results
| Branch | Method | Toolchain | Mean | StdError | StdDev | Gen 0 | Gen 1 | Gen 2 | Allocated |
|---|---|---|---|---|---|---|---|---|---|
| master | RunOnMethodBegin |
net6.0 | 1.06μs | 4.72ns | 18.3ns | 0 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
netcoreapp3.1 | 1.4μs | 7.28ns | 35.6ns | 0 | 0 | 0 | 696 B |
| master | RunOnMethodBegin |
net472 | 1.49μs | 1.05ns | 4.07ns | 0.104 | 0 | 0 | 658 B |
| #7819 | RunOnMethodBegin |
net6.0 | 1.04μs | 5.18ns | 24.3ns | 0 | 0 | 0 | 696 B |
| #7819 | RunOnMethodBegin |
netcoreapp3.1 | 1.41μs | 4.98ns | 19.3ns | 0 | 0 | 0 | 696 B |
| #7819 | RunOnMethodBegin |
net472 | 1.44μs | 0.582ns | 2.18ns | 0.101 | 0 | 0 | 658 B |
pablomartinezbernardo
left a comment
There was a problem hiding this comment.
Thanks for this
Summary of changes
Fixes the ability to filter to a specific sample/test
Reason for change
When we're doing comprehensive testing for a single integration against all supported versions, we have to filter to building a single sample (to avoid hitting OOM issues) and only running a single test (seeing as the required other samples aren't available).
We recently broke the ability to do that, first by overwriting the Filter using the group ID, and then by introducing an unescaped
&, which is interpreted by the shellImplementation details
Add quotes around the
&filter values so we don't produce invalid commandsTest coverage
Did a basic test here - it failed as expected, but for an unrelated reason, the important thing is that the command ran without errors (like it was here) :D