Add non-blocking SLO check jobs by igoragoli · Pull Request #7976 · DataDog/dd-trace-dotnet

igoragoli · 2025-12-18T19:24:16Z

Revert changes used for testing (e.g., pointing to the dev BP branch) before merging.
Merge https://github.com/DataDog/benchmarking-platform/pull/225 right after this PR is merged.
Change the notification channel to an appropriate one.

Summary of changes

Upload fetched Windows macrobenchmark results to the BP UI.
Add a non-blocking SLO check along with initial auto-generated SLOs. This CI job will be allowed to fail for the moment.
Run macrobenchmarks and SLO checks on hotfix branches as well.

Reason for change

https://datadoghq.atlassian.net/browse/APMSP-2292

Implementation details

Test coverage

SLO Check job: https://gitlab.ddbuild.io/DataDog/apm-reliability/dd-trace-dotnet/-/jobs/1387169134

Other details

dd-trace-dotnet-ci-bot · 2025-12-18T20:14:36Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (7976) and master.

⚠️ Potential regressions detected

FakeDbCommand

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	68.49 ± (68.52 - 68.84) ms	77.52 ± (77.49 - 77.81) ms	+13.2%	❌⬆️
.NET Framework 4.8 - Bailout
duration	72.43 ± (72.33 - 72.57) ms	81.66 ± (81.45 - 81.88) ms	+12.8%	❌⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1023.25 ± (1029.36 - 1039.28) ms	1094.20 ± (1094.53 - 1100.14) ms	+6.9%	❌⬆️

HttpMessageHandler

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	193.95 ± (194.21 - 195.30) ms	209.64 ± (209.52 - 211.62) ms	+8.1%	❌⬆️
.NET Framework 4.8 - Bailout
duration	197.41 ± (197.38 - 197.93) ms	216.39 ± (215.51 - 218.34) ms	+9.6%	❌⬆️

Full Metrics Comparison

FakeDbCommand

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	68.49 ± (68.52 - 68.84) ms	77.52 ± (77.49 - 77.81) ms	+13.2%	❌⬆️
.NET Framework 4.8 - Bailout
duration	72.43 ± (72.33 - 72.57) ms	81.66 ± (81.45 - 81.88) ms	+12.8%	❌⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1023.25 ± (1029.36 - 1039.28) ms	1094.20 ± (1094.53 - 1100.14) ms	+6.9%	❌⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms	21.86 ± (21.83 - 21.88) ms	23.45 ± (23.40 - 23.51) ms	+7.3%	✅⬆️
process.time_to_main_ms	78.76 ± (78.61 - 78.92) ms	89.71 ± (89.53 - 89.89) ms	+13.9%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.90 ± (10.90 - 10.91) MB	10.91 ± (10.90 - 10.91) MB	+0.0%	✅⬆️
runtime.dotnet.threads.count	12 ± (12 - 12)	12 ± (12 - 12)	+0.0%	✅
.NET Core 3.1 - Bailout
process.internal_duration_ms	21.75 ± (21.73 - 21.77) ms	23.73 ± (23.68 - 23.77) ms	+9.1%	✅⬆️
process.time_to_main_ms	80.02 ± (79.91 - 80.12) ms	93.43 ± (93.20 - 93.66) ms	+16.8%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.94 ± (10.94 - 10.94) MB	10.94 ± (10.94 - 10.95) MB	+0.0%	✅⬆️
runtime.dotnet.threads.count	13 ± (13 - 13)	13 ± (13 - 13)	+0.0%	✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	241.72 ± (238.00 - 245.43) ms	244.35 ± (240.33 - 248.38) ms	+1.1%	✅⬆️
process.time_to_main_ms	474.07 ± (473.38 - 474.76) ms	520.40 ± (519.48 - 521.33) ms	+9.8%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	48.56 ± (48.54 - 48.59) MB	48.44 ± (48.42 - 48.47) MB	-0.3%	✅
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	-0.8%	✅
.NET 6 - Baseline
process.internal_duration_ms	20.65 ± (20.62 - 20.69) ms	22.43 ± (22.38 - 22.48) ms	+8.6%	✅⬆️
process.time_to_main_ms	68.36 ± (68.23 - 68.49) ms	80.02 ± (79.81 - 80.22) ms	+17.0%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.62 ± (10.61 - 10.62) MB	10.64 ± (10.63 - 10.64) MB	+0.2%	✅⬆️
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 6 - Bailout
process.internal_duration_ms	20.52 ± (20.49 - 20.54) ms	22.61 ± (22.55 - 22.67) ms	+10.2%	✅⬆️
process.time_to_main_ms	69.27 ± (69.21 - 69.33) ms	81.88 ± (81.67 - 82.08) ms	+18.2%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	10.67 ± (10.67 - 10.67) MB	10.74 ± (10.74 - 10.74) MB	+0.7%	✅⬆️
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	250.13 ± (249.04 - 251.22) ms	244.09 ± (239.64 - 248.53) ms	-2.4%	✅
process.time_to_main_ms	451.92 ± (451.43 - 452.42) ms	498.53 ± (497.74 - 499.31) ms	+10.3%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	49.29 ± (49.26 - 49.32) MB	49.16 ± (49.14 - 49.18) MB	-0.3%	✅
runtime.dotnet.threads.count	28 ± (28 - 28)	28 ± (28 - 28)	+0.0%	✅
.NET 8 - Baseline
process.internal_duration_ms	18.70 ± (18.67 - 18.72) ms	20.63 ± (20.57 - 20.70) ms	+10.4%	✅⬆️
process.time_to_main_ms	67.05 ± (66.95 - 67.16) ms	78.73 ± (78.51 - 78.94) ms	+17.4%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.69 ± (7.68 - 7.69) MB	7.68 ± (7.67 - 7.68) MB	-0.1%	✅
runtime.dotnet.threads.count	10 ± (10 - 10)	10 ± (10 - 10)	+0.0%	✅
.NET 8 - Bailout
process.internal_duration_ms	18.74 ± (18.71 - 18.76) ms	20.69 ± (20.63 - 20.75) ms	+10.4%	✅⬆️
process.time_to_main_ms	68.32 ± (68.25 - 68.38) ms	79.95 ± (79.73 - 80.17) ms	+17.0%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	7.73 ± (7.72 - 7.74) MB	7.71 ± (7.70 - 7.71) MB	-0.3%	✅
runtime.dotnet.threads.count	11 ± (11 - 11)	11 ± (11 - 11)	+0.0%	✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	178.01 ± (177.28 - 178.75) ms	199.01 ± (198.22 - 199.81) ms	+11.8%	✅⬆️
process.time_to_main_ms	435.35 ± (434.71 - 436.00) ms	481.01 ± (480.05 - 481.97) ms	+10.5%	✅⬆️
runtime.dotnet.exceptions.count	0 ± (0 - 0)	0 ± (0 - 0)	+0.0%	✅
runtime.dotnet.mem.committed	36.69 ± (36.66 - 36.72) MB	37.09 ± (37.05 - 37.12) MB	+1.1%	✅⬆️
runtime.dotnet.threads.count	27 ± (27 - 27)	27 ± (27 - 27)	-0.1%	✅

HttpMessageHandler

Metric	Master (Mean ± 95% CI)	Current (Mean ± 95% CI)	Change	Status
.NET Framework 4.8 - Baseline
duration	193.95 ± (194.21 - 195.30) ms	209.64 ± (209.52 - 211.62) ms	+8.1%	❌⬆️
.NET Framework 4.8 - Bailout
duration	197.41 ± (197.38 - 197.93) ms	216.39 ± (215.51 - 218.34) ms	+9.6%	❌⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration	1137.50 ± (1138.97 - 1147.41) ms	1177.32 ± (1177.55 - 1185.93) ms	+3.5%	✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms	188.25 ± (187.91 - 188.59) ms	205.16 ± (204.24 - 206.09) ms	+9.0%	✅⬆️
process.time_to_main_ms	81.73 ± (81.44 - 82.02) ms	87.28 ± (86.84 - 87.72) ms	+6.8%	✅⬆️
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	16.15 ± (16.12 - 16.18) MB	15.94 ± (15.92 - 15.96) MB	-1.3%	✅
runtime.dotnet.threads.count	20 ± (19 - 20)	20 ± (20 - 20)	+1.9%	✅⬆️
.NET Core 3.1 - Bailout
process.internal_duration_ms	187.65 ± (187.35 - 187.95) ms	201.61 ± (200.84 - 202.39) ms	+7.4%	✅⬆️
process.time_to_main_ms	82.80 ± (82.66 - 82.94) ms	87.35 ± (87.03 - 87.68) ms	+5.5%	✅⬆️
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	16.20 ± (16.17 - 16.23) MB	15.97 ± (15.95 - 15.98) MB	-1.4%	✅
runtime.dotnet.threads.count	21 ± (20 - 21)	21 ± (21 - 21)	+1.6%	✅⬆️
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms	431.37 ± (428.25 - 434.49) ms	440.26 ± (436.92 - 443.60) ms	+2.1%	✅⬆️
process.time_to_main_ms	480.75 ± (480.06 - 481.43) ms	501.06 ± (499.89 - 502.22) ms	+4.2%	✅⬆️
runtime.dotnet.exceptions.count	3 ± (3 - 3)	3 ± (3 - 3)	+0.0%	✅
runtime.dotnet.mem.committed	58.93 ± (58.82 - 59.05) MB	58.78 ± (58.66 - 58.91) MB	-0.3%	✅
runtime.dotnet.threads.count	29 ± (29 - 29)	30 ± (29 - 30)	+0.2%	✅⬆️
.NET 6 - Baseline
process.internal_duration_ms	192.54 ± (192.22 - 192.86) ms	207.95 ± (206.89 - 209.00) ms	+8.0%	✅⬆️
process.time_to_main_ms	70.78 ± (70.58 - 70.97) ms	75.20 ± (74.87 - 75.54) ms	+6.3%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	16.34 ± (16.31 - 16.37) MB	16.22 ± (16.20 - 16.23) MB	-0.7%	✅
runtime.dotnet.threads.count	19 ± (19 - 19)	19 ± (19 - 19)	+3.0%	✅⬆️
.NET 6 - Bailout
process.internal_duration_ms	191.96 ± (191.62 - 192.31) ms	205.62 ± (204.86 - 206.37) ms	+7.1%	✅⬆️
process.time_to_main_ms	71.94 ± (71.80 - 72.09) ms	75.77 ± (75.51 - 76.03) ms	+5.3%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	16.28 ± (16.16 - 16.40) MB	16.25 ± (16.23 - 16.26) MB	-0.2%	✅
runtime.dotnet.threads.count	20 ± (20 - 20)	20 ± (20 - 20)	+2.3%	✅⬆️
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms	441.98 ± (438.85 - 445.10) ms	453.31 ± (449.17 - 457.44) ms	+2.6%	✅⬆️
process.time_to_main_ms	458.69 ± (458.22 - 459.15) ms	476.23 ± (475.31 - 477.14) ms	+3.8%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	59.06 ± (58.93 - 59.19) MB	59.20 ± (59.03 - 59.36) MB	+0.2%	✅⬆️
runtime.dotnet.threads.count	29 ± (29 - 29)	29 ± (29 - 30)	+0.4%	✅⬆️
.NET 8 - Baseline
process.internal_duration_ms	190.42 ± (190.04 - 190.81) ms	203.70 ± (202.86 - 204.54) ms	+7.0%	✅⬆️
process.time_to_main_ms	70.12 ± (69.92 - 70.31) ms	73.57 ± (73.30 - 73.83) ms	+4.9%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.75 ± (11.72 - 11.78) MB	11.61 ± (11.59 - 11.63) MB	-1.2%	✅
runtime.dotnet.threads.count	18 ± (18 - 18)	19 ± (18 - 19)	+1.9%	✅⬆️
.NET 8 - Bailout
process.internal_duration_ms	189.30 ± (189.05 - 189.56) ms	201.43 ± (200.68 - 202.18) ms	+6.4%	✅⬆️
process.time_to_main_ms	70.96 ± (70.86 - 71.06) ms	74.35 ± (74.12 - 74.59) ms	+4.8%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	11.82 ± (11.79 - 11.85) MB	11.69 ± (11.68 - 11.71) MB	-1.1%	✅
runtime.dotnet.threads.count	19 ± (19 - 19)	20 ± (19 - 20)	+2.0%	✅⬆️
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms	367.13 ± (365.75 - 368.50) ms	403.94 ± (397.35 - 410.54) ms	+10.0%	✅⬆️
process.time_to_main_ms	442.90 ± (441.99 - 443.81) ms	460.98 ± (460.04 - 461.91) ms	+4.1%	✅⬆️
runtime.dotnet.exceptions.count	4 ± (4 - 4)	4 ± (4 - 4)	+0.0%	✅
runtime.dotnet.mem.committed	48.37 ± (48.33 - 48.40) MB	49.22 ± (49.05 - 49.39) MB	+1.8%	✅⬆️
runtime.dotnet.threads.count	29 ± (29 - 29)	29 ± (29 - 29)	-0.4%	✅

Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts

FakeDbCommand (.NET Framework 4.8)

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7976) - mean (78ms)  : 75, 80
    master - mean (69ms)  : 67, 71

    section Bailout
    This PR (7976) - mean (82ms)  : crit, 79, 84
    master - mean (72ms)  : 71, 74

    section CallTarget+Inlining+NGEN
    This PR (7976) - mean (1,097ms)  : crit, 1056, 1139
    master - mean (1,034ms)  : 962, 1106

FakeDbCommand (.NET Core 3.1)

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7976) - mean (121ms)  : 116, 125
    master - mean (106ms)  : 104, 108

    section Bailout
    This PR (7976) - mean (124ms)  : crit, 121, 128
    master - mean (107ms)  : 106, 108

    section CallTarget+Inlining+NGEN
    This PR (7976) - mean (803ms)  : crit, 747, 860
    master - mean (741ms)  : 683, 799

FakeDbCommand (.NET 6)

gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7976) - mean (109ms)  : 105, 113
    master - mean (94ms)  : 91, 96

    section Bailout
    This PR (7976) - mean (111ms)  : crit, 108, 114
    master - mean (94ms)  : 93, 95

    section CallTarget+Inlining+NGEN
    This PR (7976) - mean (780ms)  : crit, 706, 855
    master - mean (728ms)  : 703, 753

FakeDbCommand (.NET 8)

gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7976) - mean (108ms)  : 103, 112
    master - mean (92ms)  : 89, 95

    section Bailout
    This PR (7976) - mean (109ms)  : crit, 106, 112
    master - mean (93ms)  : 92, 94

    section CallTarget+Inlining+NGEN
    This PR (7976) - mean (725ms)  : crit, 701, 748
    master - mean (644ms)  : 628, 660

HttpMessageHandler (.NET Framework 4.8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7976) - mean (211ms)  : 194, 227
    master - mean (195ms)  : 189, 201

    section Bailout
    This PR (7976) - mean (217ms)  : crit, 196, 238
    master - mean (198ms)  : 195, 200

    section CallTarget+Inlining+NGEN
    This PR (7976) - mean (1,182ms)  : 1122, 1242
    master - mean (1,143ms)  : 1082, 1204

HttpMessageHandler (.NET Core 3.1)

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7976) - mean (303ms)  : 281, 324
    master - mean (278ms)  : 272, 284

    section Bailout
    This PR (7976) - mean (299ms)  : crit, 284, 313
    master - mean (279ms)  : 275, 282

    section CallTarget+Inlining+NGEN
    This PR (7976) - mean (975ms)  : 918, 1032
    master - mean (942ms)  : 894, 990

HttpMessageHandler (.NET 6)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7976) - mean (293ms)  : 272, 314
    master - mean (272ms)  : 266, 278

    section Bailout
    This PR (7976) - mean (291ms)  : crit, 276, 305
    master - mean (272ms)  : 267, 277

    section CallTarget+Inlining+NGEN
    This PR (7976) - mean (958ms)  : 887, 1029
    master - mean (928ms)  : 874, 982

HttpMessageHandler (.NET 8)

gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7976) - mean (288ms)  : 270, 306
    master - mean (270ms)  : 264, 276

    section Bailout
    This PR (7976) - mean (287ms)  : crit, 272, 302
    master - mean (270ms)  : 267, 273

    section CallTarget+Inlining+NGEN
    This PR (7976) - mean (901ms)  : crit, 799, 1003
    master - mean (841ms)  : 825, 857

pr-commenter · 2025-12-18T20:21:13Z

Benchmarks

Benchmark execution time: 2026-01-30 15:39:40

Comparing candidate commit f2a517a in PR branch augusto/apmsp-2292.macrobenchmarks.add-perf-quality-gates with baseline commit 3d020db in branch master.

Found 3 performance improvements and 14 performance regressions! Performance is the same for 165 metrics, 10 unstable metrics.

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces net6.0

🟥 execution_time [+80.673ms; +80.815ms] or [+66.285%; +66.402%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody net6.0

🟥 execution_time [+21.826ms; +27.980ms] or [+11.003%; +14.105%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody netcoreapp3.1

🟥 execution_time [+16.166ms; +20.839ms] or [+8.094%; +10.434%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody net472

🟥 throughput [-245160.287op/s; -235650.378op/s] or [-7.278%; -6.995%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody net6.0

🟥 throughput [-494922.804op/s; -408703.582op/s] or [-15.975%; -13.192%]

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody netcoreapp3.1

🟥 execution_time [+17.779ms; +23.927ms] or [+9.013%; +12.129%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs net6.0

🟥 execution_time [+23.476ms; +23.730ms] or [+13.241%; +13.384%]

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs netcoreapp3.1

🟩 execution_time [-22.716ms; -21.681ms] or [-11.273%; -10.760%]

scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool net6.0

🟥 execution_time [+69.665µs; +73.881µs] or [+6.825%; +7.238%]
🟥 throughput [-66.334op/s; -62.408op/s] or [-6.771%; -6.370%]

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark netcoreapp3.1

🟥 throughput [-466.353op/s; -256.026op/s] or [-21.000%; -11.529%]

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog netcoreapp3.1

🟥 execution_time [+34.559ms; +40.442ms] or [+20.781%; +24.318%]

scenario:Benchmarks.Trace.RedisBenchmark.SendReceive net6.0

🟩 execution_time [-17.389ms; -13.285ms] or [-8.110%; -6.196%]

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore net6.0

🟥 execution_time [+4.819ms; +6.347ms] or [+5.278%; +6.951%]

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore netcoreapp3.1

🟥 throughput [-15096475.520op/s; -13775389.335op/s] or [-6.289%; -5.739%]

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan net6.0

🟥 execution_time [+10.184ms; +15.822ms] or [+5.142%; +7.989%]

scenario:Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin net6.0

🟩 execution_time [-18.448ms; -14.795ms] or [-8.528%; -6.839%]

Warmup: 3m → 30s Normal: 10m → 1m High load: 5m → 30s

This reverts commit a41f6df.

This reverts commit ccb7fa2.

This reverts commit 6692f4f.

Add performance quality gates infrastructure for Windows macrobenchmarks: - Include check-slo-breaches and notify-slo-breaches templates - Add gate and notify stages - Add check-slo-breaches job that runs after all Windows benchmarks - Add notify-slo-breaches job that alerts to Slack channel - Add dummy SLO file with basic throughput and latency thresholds Both jobs have allow_failure: true for testing purposes.

Update scenario regex from catch-all to throughput-* to match the mock benchmark scenarios (throughput-3, throughput-6).

Use throughput.* (dot-star for any characters) instead of throughput-3 to match all throughput scenarios.

Set conservative thresholds with ~20% margin: - throughput > 8000 op/s (observed: ~10k) - p95 latency < 65ms (observed: 26-51ms) These are initial values for testing the SLO gate mechanism. Thresholds will be calibrated once historical data is collected.

…f-quality-gates

NachoEchevarria

Nice, thanks!

This comment has been minimized.

Sign in to view

igoragoli force-pushed the augusto/apmsp-2292.macrobenchmarks.add-perf-quality-gates branch 2 times, most recently from 95e0358 to 67fbe49 Compare January 29, 2026 10:08

igoragoli added 25 commits January 29, 2026 14:54

fetch results from s3

bd18eb1

test: point to dev branch

80e80ff

test: hardcode build ID

08804c4

test: reduce k6 durations for faster testing

d78efd0

Warmup: 3m → 30s Normal: 10m → 1m High load: 5m → 30s

Revert "test: reduce k6 durations for faster testing"

04ca1ac

This reverts commit a41f6df.

Revert "test: hardcode build ID"

98790be

This reverts commit ccb7fa2.

Revert "test: point to dev branch"

6524a2a

This reverts commit 6692f4f.

refactor: use reports as artifacts dir like microbenchmarks do

2d8d372

format: newline

0c9c0dd

feat: add bp-infra env vars

3dd0c67

feat: add bp-infra env vars

795bd13

feat: add bp-infra env vars

2156617

test: hardcode build id

cc33096

feat: bump after script timeout

092d1c4

feat: add fetch and upload results steps

b27bd5d

test: set BP_INFRA_TEST to true

cdf36ff

feat: add "windows" to windows-specific scripts

840eb8b

test: update hardcoded build id

3748542

fix: update SLO scenario name to match mock results

c798b46

Update scenario regex from catch-all to throughput-* to match the mock benchmark scenarios (throughput-3, throughput-6).

test: modify dummy SLO check scenario

63438d2

fix: use correct regex syntax for scenario matching

60dca61

Use throughput.* (dot-star for any characters) instead of throughput-3 to match all throughput scenarios.

tweak: modify comments on SLO file

5e04bc8

test: set BP_INFRA_TEST to false

89e6d2e

igoragoli force-pushed the augusto/apmsp-2292.macrobenchmarks.add-perf-quality-gates branch from 99e9fd2 to 89e6d2e Compare January 29, 2026 13:55

igoragoli added 2 commits January 29, 2026 15:01

refactor: better explain allow_failure on check-slo-breaches

1626f83

tweak: add a TODO to change the notification channel

d4d33a7

igoragoli changed the title ~~Add pre-release performance quality gates~~ Add non-blocking SLO check jobs Jan 29, 2026

igoragoli added 3 commits January 29, 2026 16:49

tweak: update SLOs with auto-generated ones based on pipeline 93531931

e4a6a69

tweak: set appropriate notification channel

38fe413

tweak: revert testing changes

c5838a3

igoragoli marked this pull request as ready for review January 30, 2026 08:27

igoragoli requested a review from a team as a code owner January 30, 2026 08:27

igoragoli added 2 commits January 30, 2026 09:27

Merge branch 'master' into augusto/apmsp-2292.macrobenchmarks.add-per…

c312f6a

…f-quality-gates

feat: run macrobenchmarks on hotfix branches as well

864521f

NachoEchevarria approved these changes Jan 30, 2026

View reviewed changes

igoragoli added 2 commits January 30, 2026 14:25

fix: remove reference to deleted fetch-results.sh

91e2d0a

trigger ci

f2a517a

igoragoli merged commit 2e54cad into master Feb 2, 2026
140 checks passed

igoragoli deleted the augusto/apmsp-2292.macrobenchmarks.add-perf-quality-gates branch February 2, 2026 09:20

github-actions bot added this to the vNext-v3 milestone Feb 2, 2026

NachoEchevarria added the area:benchmarks Benchmarks, throughput tests, Crank, Bombardier, etc label Feb 9, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add non-blocking SLO check jobs#7976

Add non-blocking SLO check jobs#7976
igoragoli merged 34 commits intomasterfrom
augusto/apmsp-2292.macrobenchmarks.add-perf-quality-gates

igoragoli commented Dec 18, 2025 •

edited

Loading

Uh oh!

This comment has been minimized.

dd-trace-dotnet-ci-bot bot commented Dec 18, 2025 •

edited

Loading

FakeDbCommand

HttpMessageHandler

Uh oh!

pr-commenter bot commented Dec 18, 2025 •

edited

Loading

Uh oh!

NachoEchevarria left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

igoragoli commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

Uh oh!

This comment has been minimized.

dd-trace-dotnet-ci-bot bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Execution-Time Benchmarks Report ⏱️

⚠️ Potential regressions detected

FakeDbCommand

HttpMessageHandler

FakeDbCommand

HttpMessageHandler

Uh oh!

pr-commenter bot commented Dec 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

scenario:Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces net6.0

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody net6.0

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody netcoreapp3.1

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody net472

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody net6.0

scenario:Benchmarks.Trace.Asm.AppSecBodyBenchmark.ObjectExtractorSimpleBody netcoreapp3.1

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs net6.0

scenario:Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs netcoreapp3.1

scenario:Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool net6.0

scenario:Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark netcoreapp3.1

scenario:Benchmarks.Trace.Log4netBenchmark.EnrichedLog netcoreapp3.1

scenario:Benchmarks.Trace.RedisBenchmark.SendReceive net6.0

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore net6.0

scenario:Benchmarks.Trace.SingleSpanAspNetCoreBenchmark.SingleSpanAspNetCore netcoreapp3.1

scenario:Benchmarks.Trace.SpanBenchmark.StartFinishSpan net6.0

scenario:Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin net6.0

Uh oh!

NachoEchevarria left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

igoragoli commented Dec 18, 2025 •

edited

Loading

dd-trace-dotnet-ci-bot bot commented Dec 18, 2025 •

edited

Loading

pr-commenter bot commented Dec 18, 2025 •

edited

Loading