Skip to content

allow filtering when running profiler tests#8029

Merged
vandonr merged 1 commit intomasterfrom
vandonr/nit
Jan 6, 2026
Merged

allow filtering when running profiler tests#8029
vandonr merged 1 commit intomasterfrom
vandonr/nit

Conversation

@vandonr
Copy link
Contributor

@vandonr vandonr commented Jan 5, 2026

Summary of changes

tiny change to "listen" to the provided filter on the BuildAndRunProfilerIntegrationTests command, so that it's possible to run a single test (class).

Reason for change

Implementation details

Test coverage

Other details

@vandonr vandonr requested a review from a team as a code owner January 5, 2026 15:36
@github-actions github-actions bot added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Jan 5, 2026
@dd-trace-dotnet-ci-bot
Copy link

dd-trace-dotnet-ci-bot bot commented Jan 5, 2026

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (8029) and master.

✅ No regressions detected - check the details below

Full Metrics Comparison

FakeDbCommand

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration73.28 ± (73.27 - 73.76) ms72.55 ± (72.60 - 73.05) ms-1.0%✅
.NET Framework 4.8 - Bailout
duration77.15 ± (76.87 - 77.29) ms77.33 ± (77.12 - 77.55) ms+0.2%✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1048.43 ± (1047.63 - 1056.46) ms1053.93 ± (1054.73 - 1065.09) ms+0.5%✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms22.58 ± (22.54 - 22.63) ms22.43 ± (22.40 - 22.47) ms-0.7%✅
process.time_to_main_ms83.49 ± (83.26 - 83.72) ms82.04 ± (81.83 - 82.25) ms-1.7%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.93 ± (10.93 - 10.93) MB10.94 ± (10.94 - 10.94) MB+0.1%✅⬆️
runtime.dotnet.threads.count12 ± (12 - 12)12 ± (12 - 12)+0.0%✅
.NET Core 3.1 - Bailout
process.internal_duration_ms22.30 ± (22.27 - 22.33) ms22.54 ± (22.51 - 22.57) ms+1.1%✅⬆️
process.time_to_main_ms83.78 ± (83.61 - 83.96) ms85.22 ± (85.02 - 85.43) ms+1.7%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.96 ± (10.96 - 10.97) MB10.96 ± (10.95 - 10.96) MB-0.1%✅
runtime.dotnet.threads.count13 ± (13 - 13)13 ± (13 - 13)+0.0%✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms251.54 ± (247.40 - 255.68) ms242.34 ± (238.04 - 246.64) ms-3.7%✅
process.time_to_main_ms493.28 ± (491.92 - 494.64) ms493.18 ± (491.90 - 494.47) ms-0.0%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed48.14 ± (48.12 - 48.17) MB48.21 ± (48.18 - 48.23) MB+0.1%✅⬆️
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)+0.6%✅⬆️
.NET 6 - Baseline
process.internal_duration_ms21.20 ± (21.16 - 21.25) ms21.20 ± (21.15 - 21.24) ms-0.0%✅
process.time_to_main_ms72.44 ± (72.25 - 72.62) ms72.97 ± (72.77 - 73.18) ms+0.7%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.61 ± (10.61 - 10.62) MB10.64 ± (10.64 - 10.64) MB+0.2%✅⬆️
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%✅
.NET 6 - Bailout
process.internal_duration_ms21.20 ± (21.16 - 21.24) ms21.10 ± (21.06 - 21.14) ms-0.5%✅
process.time_to_main_ms73.67 ± (73.50 - 73.84) ms73.32 ± (73.17 - 73.48) ms-0.5%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.72 ± (10.71 - 10.72) MB10.75 ± (10.75 - 10.75) MB+0.3%✅⬆️
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms243.37 ± (239.73 - 247.01) ms243.65 ± (240.05 - 247.25) ms+0.1%✅⬆️
process.time_to_main_ms458.19 ± (456.94 - 459.44) ms456.28 ± (455.09 - 457.47) ms-0.4%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed48.59 ± (48.56 - 48.62) MB48.55 ± (48.52 - 48.58) MB-0.1%✅
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)+0.0%✅⬆️
.NET 8 - Baseline
process.internal_duration_ms19.43 ± (19.39 - 19.46) ms19.26 ± (19.23 - 19.29) ms-0.9%✅
process.time_to_main_ms72.29 ± (72.12 - 72.46) ms71.77 ± (71.59 - 71.95) ms-0.7%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed7.66 ± (7.65 - 7.66) MB7.67 ± (7.66 - 7.67) MB+0.1%✅⬆️
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%✅
.NET 8 - Bailout
process.internal_duration_ms19.39 ± (19.36 - 19.42) ms19.50 ± (19.46 - 19.54) ms+0.6%✅⬆️
process.time_to_main_ms73.30 ± (73.15 - 73.45) ms73.48 ± (73.32 - 73.65) ms+0.2%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed7.74 ± (7.73 - 7.75) MB7.72 ± (7.71 - 7.72) MB-0.3%✅
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms188.60 ± (187.63 - 189.58) ms185.98 ± (185.14 - 186.81) ms-1.4%✅
process.time_to_main_ms448.70 ± (447.09 - 450.31) ms443.23 ± (442.05 - 444.40) ms-1.2%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed36.50 ± (36.47 - 36.53) MB36.50 ± (36.47 - 36.54) MB+0.0%✅⬆️
runtime.dotnet.threads.count27 ± (27 - 27)27 ± (27 - 27)-0.0%✅

HttpMessageHandler

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration194.14 ± (194.24 - 195.10) ms195.06 ± (195.40 - 196.47) ms+0.5%✅⬆️
.NET Framework 4.8 - Bailout
duration197.21 ± (197.02 - 197.80) ms200.47 ± (199.83 - 200.92) ms+1.7%✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1111.77 ± (1115.65 - 1123.70) ms1121.15 ± (1121.46 - 1129.13) ms+0.8%✅⬆️
.NET Core 3.1 - Baseline
process.internal_duration_ms188.69 ± (188.33 - 189.05) ms188.50 ± (188.15 - 188.85) ms-0.1%✅
process.time_to_main_ms81.00 ± (80.79 - 81.21) ms81.12 ± (80.86 - 81.38) ms+0.1%✅⬆️
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed16.07 ± (16.04 - 16.09) MB16.12 ± (16.09 - 16.15) MB+0.3%✅⬆️
runtime.dotnet.threads.count20 ± (19 - 20)20 ± (20 - 20)+0.2%✅⬆️
.NET Core 3.1 - Bailout
process.internal_duration_ms189.39 ± (188.99 - 189.79) ms188.45 ± (188.05 - 188.86) ms-0.5%✅
process.time_to_main_ms82.71 ± (82.55 - 82.88) ms82.53 ± (82.36 - 82.70) ms-0.2%✅
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed16.16 ± (16.13 - 16.19) MB16.26 ± (16.23 - 16.29) MB+0.6%✅⬆️
runtime.dotnet.threads.count21 ± (21 - 21)21 ± (21 - 21)-0.6%✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms417.22 ± (413.65 - 420.78) ms419.98 ± (416.37 - 423.58) ms+0.7%✅⬆️
process.time_to_main_ms474.03 ± (473.33 - 474.73) ms481.00 ± (479.96 - 482.04) ms+1.5%✅⬆️
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed58.48 ± (58.36 - 58.61) MB58.71 ± (58.60 - 58.82) MB+0.4%✅⬆️
runtime.dotnet.threads.count30 ± (29 - 30)30 ± (29 - 30)+0.0%✅⬆️
.NET 6 - Baseline
process.internal_duration_ms192.23 ± (191.79 - 192.66) ms195.22 ± (194.78 - 195.66) ms+1.6%✅⬆️
process.time_to_main_ms69.77 ± (69.60 - 69.94) ms70.92 ± (70.73 - 71.10) ms+1.6%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed16.17 ± (16.04 - 16.29) MB16.38 ± (16.35 - 16.42) MB+1.3%✅⬆️
runtime.dotnet.threads.count19 ± (19 - 19)19 ± (19 - 19)+0.7%✅⬆️
.NET 6 - Bailout
process.internal_duration_ms190.81 ± (190.57 - 191.05) ms193.96 ± (193.49 - 194.43) ms+1.6%✅⬆️
process.time_to_main_ms70.74 ± (70.67 - 70.81) ms71.75 ± (71.55 - 71.94) ms+1.4%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed16.07 ± (15.91 - 16.23) MB16.42 ± (16.39 - 16.45) MB+2.2%✅⬆️
runtime.dotnet.threads.count19 ± (19 - 19)20 ± (20 - 20)+5.1%✅⬆️
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms449.65 ± (446.57 - 452.73) ms448.68 ± (445.08 - 452.29) ms-0.2%✅
process.time_to_main_ms445.24 ± (444.64 - 445.85) ms452.96 ± (452.07 - 453.86) ms+1.7%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed58.23 ± (58.12 - 58.33) MB58.47 ± (58.34 - 58.61) MB+0.4%✅⬆️
runtime.dotnet.threads.count30 ± (30 - 30)30 ± (29 - 30)-0.2%✅
.NET 8 - Baseline
process.internal_duration_ms190.37 ± (189.96 - 190.78) ms193.58 ± (193.11 - 194.04) ms+1.7%✅⬆️
process.time_to_main_ms69.42 ± (69.21 - 69.63) ms70.45 ± (70.26 - 70.64) ms+1.5%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed11.69 ± (11.66 - 11.72) MB11.70 ± (11.68 - 11.72) MB+0.1%✅⬆️
runtime.dotnet.threads.count18 ± (18 - 18)18 ± (18 - 18)-0.1%✅
.NET 8 - Bailout
process.internal_duration_ms189.59 ± (189.33 - 189.86) ms194.89 ± (194.44 - 195.35) ms+2.8%✅⬆️
process.time_to_main_ms70.49 ± (70.39 - 70.59) ms72.34 ± (72.15 - 72.52) ms+2.6%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed11.74 ± (11.71 - 11.76) MB11.76 ± (11.74 - 11.78) MB+0.2%✅⬆️
runtime.dotnet.threads.count19 ± (19 - 19)19 ± (19 - 19)+0.6%✅⬆️
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms364.39 ± (362.77 - 366.00) ms366.52 ± (365.17 - 367.86) ms+0.6%✅⬆️
process.time_to_main_ms427.33 ± (426.73 - 427.92) ms430.29 ± (429.58 - 430.99) ms+0.7%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed47.94 ± (47.91 - 47.97) MB47.98 ± (47.95 - 48.01) MB+0.1%✅⬆️
runtime.dotnet.threads.count29 ± (29 - 29)29 ± (29 - 29)-0.0%✅
Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts
FakeDbCommand (.NET Framework 4.8) 
gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8029) - mean (73ms)  : 70, 76
    master - mean (74ms)  : 70, 77

    section Bailout
    This PR (8029) - mean (77ms)  : 74, 81
    master - mean (77ms)  : 74, 80

    section CallTarget+Inlining+NGEN
    This PR (8029) - mean (1,060ms)  : 982, 1138
    master - mean (1,052ms)  : 989, 1115
Loading
FakeDbCommand (.NET Core 3.1) 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8029) - mean (111ms)  : 105, 116
    master - mean (113ms)  : 108, 118

    section Bailout
    This PR (8029) - mean (114ms)  : 110, 119
    master - mean (112ms)  : 109, 116

    section CallTarget+Inlining+NGEN
    This PR (8029) - mean (775ms)  : 701, 848
    master - mean (780ms)  : 723, 838
Loading
FakeDbCommand (.NET 6) 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8029) - mean (101ms)  : 96, 105
    master - mean (100ms)  : 96, 104

    section Bailout
    This PR (8029) - mean (100ms)  : 97, 104
    master - mean (101ms)  : 97, 105

    section CallTarget+Inlining+NGEN
    This PR (8029) - mean (729ms)  : 659, 798
    master - mean (730ms)  : 667, 793
Loading
FakeDbCommand (.NET 8) 
gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8029) - mean (99ms)  : 95, 102
    master - mean (99ms)  : 96, 103

    section Bailout
    This PR (8029) - mean (101ms)  : 97, 104
    master - mean (100ms)  : 96, 104

    section CallTarget+Inlining+NGEN
    This PR (8029) - mean (661ms)  : 630, 692
    master - mean (673ms)  : 626, 719
Loading
HttpMessageHandler (.NET Framework 4.8) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8029) - mean (196ms)  : 189, 203
    master - mean (195ms)  : 189, 200

    section Bailout
    This PR (8029) - mean (200ms)  : 195, 206
    master - mean (197ms)  : 193, 201

    section CallTarget+Inlining+NGEN
    This PR (8029) - mean (1,125ms)  : 1070, 1180
    master - mean (1,120ms)  : 1060, 1179
Loading
HttpMessageHandler (.NET Core 3.1) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8029) - mean (278ms)  : 273, 284
    master - mean (278ms)  : 273, 284

    section Bailout
    This PR (8029) - mean (279ms)  : 274, 285
    master - mean (280ms)  : 275, 286

    section CallTarget+Inlining+NGEN
    This PR (8029) - mean (939ms)  : 889, 988
    master - mean (921ms)  : 860, 981
Loading
HttpMessageHandler (.NET 6) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8029) - mean (275ms)  : 265, 285
    master - mean (270ms)  : 264, 276

    section Bailout
    This PR (8029) - mean (274ms)  : 268, 280
    master - mean (270ms)  : 267, 272

    section CallTarget+Inlining+NGEN
    This PR (8029) - mean (934ms)  : 884, 984
    master - mean (921ms)  : 866, 975
Loading
HttpMessageHandler (.NET 8) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (8029) - mean (274ms)  : 263, 285
    master - mean (269ms)  : 264, 275

    section Bailout
    This PR (8029) - mean (277ms)  : 270, 284
    master - mean (270ms)  : 266, 273

    section CallTarget+Inlining+NGEN
    This PR (8029) - mean (827ms)  : 801, 852
    master - mean (823ms)  : 802, 843
Loading

gleocadie
gleocadie approved these changes Jan 6, 2026
View reviewed changes
Copy link
Collaborator

@gleocadie gleocadie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vandonr vandonr merged commit f0a17cd into master Jan 6, 2026
150 checks passed
@vandonr vandonr deleted the vandonr/nit branch January 6, 2026 09:23
@github-actions github-actions bot added this to the vNext-v3 milestone Jan 6, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gleocadie gleocadie gleocadie approved these changes

Assignees

No one assigned

Labels

area:builds project files, build scripts, pipelines, versioning, releases, packages

Projects

None yet

Milestone

3.35.0

Development

Successfully merging this pull request may close these issues.

2 participants

@vandonr @gleocadie