Skip to content

Increase retries in Stack Exchange Submit traces test#7875

Merged
NachoEchevarria merged 1 commit intomasterfrom
nacho/IncreaseRetrisInStackExchangeTest
Nov 28, 2025
Merged

Increase retries in Stack Exchange Submit traces test#7875
NachoEchevarria merged 1 commit intomasterfrom
nacho/IncreaseRetrisInStackExchangeTest

Conversation

@NachoEchevarria
Copy link
Collaborator

@NachoEchevarria NachoEchevarria commented Nov 27, 2025

Summary of changes

Stack Exchange Submit traces test is still flaky with one retry with the same infraestructure error (pinging the host stackexchangeredis instead of the replica one)

Since this is not a tracer error, increasing the retries would be advisable.

Reason for change

Implementation details

Test coverage

Other details

@NachoEchevarria NachoEchevarria marked this pull request as ready for review November 27, 2025 15:27
@NachoEchevarria NachoEchevarria requested review from a team as code owners November 27, 2025 15:27
@github-actions github-actions bot added the area:tests unit tests, integration tests label Nov 27, 2025
@dd-trace-dotnet-ci-bot
Copy link

dd-trace-dotnet-ci-bot bot commented Nov 27, 2025

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing This PR (7875) and master.

✅ No regressions detected - check the details below

Full Metrics Comparison

FakeDbCommand

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration74.43 ± (74.22 - 75.01) ms75.13 ± (74.96 - 75.81) ms+0.9%✅⬆️
.NET Framework 4.8 - Bailout
duration79.87 ± (79.68 - 80.43) ms79.24 ± (79.15 - 79.89) ms-0.8%✅
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1054.23 ± (1055.13 - 1063.13) ms1052.52 ± (1055.46 - 1065.25) ms-0.2%✅
.NET Core 3.1 - Baseline
process.internal_duration_ms22.91 ± (22.84 - 22.99) ms22.76 ± (22.69 - 22.82) ms-0.7%✅
process.time_to_main_ms87.59 ± (87.17 - 88.00) ms86.82 ± (86.42 - 87.23) ms-0.9%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.93 ± (10.93 - 10.94) MB10.90 ± (10.90 - 10.91) MB-0.3%✅
runtime.dotnet.threads.count12 ± (12 - 12)12 ± (12 - 12)+0.0%✅
.NET Core 3.1 - Bailout
process.internal_duration_ms22.64 ± (22.57 - 22.70) ms22.65 ± (22.59 - 22.71) ms+0.1%✅⬆️
process.time_to_main_ms87.49 ± (87.16 - 87.82) ms87.29 ± (86.94 - 87.63) ms-0.2%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.97 ± (10.96 - 10.97) MB10.96 ± (10.95 - 10.96) MB-0.1%✅
runtime.dotnet.threads.count13 ± (13 - 13)13 ± (13 - 13)+0.0%✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms222.26 ± (221.07 - 223.44) ms222.11 ± (220.50 - 223.73) ms-0.1%✅
process.time_to_main_ms500.28 ± (498.95 - 501.61) ms497.56 ± (496.45 - 498.67) ms-0.5%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed47.86 ± (47.84 - 47.88) MB47.83 ± (47.81 - 47.86) MB-0.1%✅
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)+0.1%✅⬆️
.NET 6 - Baseline
process.internal_duration_ms21.65 ± (21.59 - 21.72) ms21.75 ± (21.67 - 21.83) ms+0.4%✅⬆️
process.time_to_main_ms75.92 ± (75.58 - 76.26) ms75.40 ± (75.05 - 75.74) ms-0.7%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.61 ± (10.60 - 10.61) MB10.62 ± (10.62 - 10.63) MB+0.2%✅⬆️
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%✅
.NET 6 - Bailout
process.internal_duration_ms21.71 ± (21.63 - 21.79) ms21.70 ± (21.62 - 21.77) ms-0.1%✅
process.time_to_main_ms77.61 ± (77.24 - 77.98) ms77.72 ± (77.34 - 78.10) ms+0.1%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed10.65 ± (10.64 - 10.65) MB10.66 ± (10.66 - 10.67) MB+0.2%✅⬆️
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms209.89 ± (208.75 - 211.03) ms211.37 ± (210.15 - 212.58) ms+0.7%✅⬆️
process.time_to_main_ms467.67 ± (466.71 - 468.63) ms466.91 ± (465.99 - 467.84) ms-0.2%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed48.11 ± (48.09 - 48.13) MB48.05 ± (48.02 - 48.07) MB-0.1%✅
runtime.dotnet.threads.count28 ± (28 - 28)28 ± (28 - 28)+0.0%✅⬆️
.NET 8 - Baseline
process.internal_duration_ms19.95 ± (19.89 - 20.01) ms19.88 ± (19.82 - 19.94) ms-0.3%✅
process.time_to_main_ms75.18 ± (74.92 - 75.44) ms75.69 ± (75.33 - 76.05) ms+0.7%✅⬆️
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed7.64 ± (7.64 - 7.64) MB7.66 ± (7.65 - 7.66) MB+0.2%✅⬆️
runtime.dotnet.threads.count10 ± (10 - 10)10 ± (10 - 10)+0.0%✅
.NET 8 - Bailout
process.internal_duration_ms19.90 ± (19.84 - 19.96) ms19.72 ± (19.67 - 19.78) ms-0.9%✅
process.time_to_main_ms76.72 ± (76.44 - 77.01) ms75.96 ± (75.66 - 76.26) ms-1.0%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed7.71 ± (7.70 - 7.72) MB7.70 ± (7.69 - 7.71) MB-0.1%✅
runtime.dotnet.threads.count11 ± (11 - 11)11 ± (11 - 11)+0.0%✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms194.07 ± (193.11 - 195.04) ms192.59 ± (191.57 - 193.60) ms-0.8%✅
process.time_to_main_ms457.58 ± (456.51 - 458.64) ms455.55 ± (454.36 - 456.73) ms-0.4%✅
runtime.dotnet.exceptions.count0 ± (0 - 0)0 ± (0 - 0)+0.0%✅
runtime.dotnet.mem.committed36.42 ± (36.39 - 36.46) MB36.34 ± (36.31 - 36.38) MB-0.2%✅
runtime.dotnet.threads.count27 ± (27 - 27)27 ± (27 - 27)+0.0%✅⬆️

HttpMessageHandler

Metric Master (Mean ± 95% CI) Current (Mean ± 95% CI) Change Status
.NET Framework 4.8 - Baseline
duration193.10 ± (193.10 - 193.93) ms192.50 ± (192.60 - 193.33) ms-0.3%✅
.NET Framework 4.8 - Bailout
duration196.82 ± (196.68 - 197.26) ms197.96 ± (197.91 - 198.79) ms+0.6%✅⬆️
.NET Framework 4.8 - CallTarget+Inlining+NGEN
duration1112.46 ± (1115.34 - 1124.14) ms1108.21 ± (1110.95 - 1118.71) ms-0.4%✅
.NET Core 3.1 - Baseline
process.internal_duration_ms188.36 ± (187.86 - 188.87) ms187.78 ± (187.48 - 188.07) ms-0.3%✅
process.time_to_main_ms80.86 ± (80.66 - 81.07) ms80.45 ± (80.24 - 80.65) ms-0.5%✅
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed16.11 ± (16.08 - 16.14) MB16.09 ± (16.06 - 16.11) MB-0.2%✅
runtime.dotnet.threads.count20 ± (19 - 20)20 ± (20 - 20)+0.2%✅⬆️
.NET Core 3.1 - Bailout
process.internal_duration_ms187.52 ± (187.17 - 187.88) ms187.80 ± (187.56 - 188.05) ms+0.1%✅⬆️
process.time_to_main_ms81.99 ± (81.86 - 82.13) ms81.83 ± (81.69 - 81.97) ms-0.2%✅
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed16.15 ± (16.12 - 16.18) MB16.20 ± (16.17 - 16.23) MB+0.3%✅⬆️
runtime.dotnet.threads.count21 ± (21 - 21)21 ± (21 - 21)-0.9%✅
.NET Core 3.1 - CallTarget+Inlining+NGEN
process.internal_duration_ms397.76 ± (395.17 - 400.36) ms400.87 ± (398.13 - 403.61) ms+0.8%✅⬆️
process.time_to_main_ms473.22 ± (472.60 - 473.83) ms472.04 ± (471.44 - 472.64) ms-0.2%✅
runtime.dotnet.exceptions.count3 ± (3 - 3)3 ± (3 - 3)+0.0%✅
runtime.dotnet.mem.committed58.66 ± (58.53 - 58.79) MB58.69 ± (58.57 - 58.82) MB+0.1%✅⬆️
runtime.dotnet.threads.count29 ± (29 - 29)29 ± (29 - 29)-0.0%✅
.NET 6 - Baseline
process.internal_duration_ms192.86 ± (192.48 - 193.24) ms192.78 ± (192.39 - 193.16) ms-0.0%✅
process.time_to_main_ms70.17 ± (70.04 - 70.30) ms70.29 ± (70.10 - 70.49) ms+0.2%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed16.17 ± (16.05 - 16.29) MB15.91 ± (15.75 - 16.07) MB-1.6%✅
runtime.dotnet.threads.count19 ± (19 - 19)18 ± (18 - 18)-3.4%✅
.NET 6 - Bailout
process.internal_duration_ms191.46 ± (191.13 - 191.80) ms191.65 ± (191.28 - 192.02) ms+0.1%✅⬆️
process.time_to_main_ms70.97 ± (70.87 - 71.07) ms71.16 ± (71.03 - 71.30) ms+0.3%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed16.41 ± (16.33 - 16.49) MB16.09 ± (15.94 - 16.25) MB-1.9%✅
runtime.dotnet.threads.count20 ± (20 - 20)19 ± (19 - 19)-2.5%✅
.NET 6 - CallTarget+Inlining+NGEN
process.internal_duration_ms405.80 ± (403.88 - 407.73) ms404.90 ± (403.20 - 406.60) ms-0.2%✅
process.time_to_main_ms441.21 ± (440.65 - 441.77) ms441.95 ± (441.33 - 442.57) ms+0.2%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed59.10 ± (58.96 - 59.23) MB58.99 ± (58.85 - 59.14) MB-0.2%✅
runtime.dotnet.threads.count30 ± (29 - 30)30 ± (30 - 30)+0.0%✅⬆️
.NET 8 - Baseline
process.internal_duration_ms190.45 ± (190.07 - 190.84) ms191.15 ± (190.77 - 191.53) ms+0.4%✅⬆️
process.time_to_main_ms69.61 ± (69.43 - 69.79) ms69.79 ± (69.60 - 69.98) ms+0.3%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed11.74 ± (11.72 - 11.77) MB11.77 ± (11.74 - 11.80) MB+0.2%✅⬆️
runtime.dotnet.threads.count18 ± (18 - 18)18 ± (18 - 18)+0.4%✅⬆️
.NET 8 - Bailout
process.internal_duration_ms190.12 ± (189.79 - 190.45) ms189.68 ± (189.36 - 190.00) ms-0.2%✅
process.time_to_main_ms70.61 ± (70.52 - 70.71) ms70.58 ± (70.47 - 70.68) ms-0.1%✅
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed11.84 ± (11.81 - 11.87) MB11.85 ± (11.82 - 11.89) MB+0.1%✅⬆️
runtime.dotnet.threads.count19 ± (19 - 19)19 ± (19 - 19)-0.4%✅
.NET 8 - CallTarget+Inlining+NGEN
process.internal_duration_ms367.68 ± (366.30 - 369.07) ms370.17 ± (368.88 - 371.47) ms+0.7%✅⬆️
process.time_to_main_ms431.34 ± (430.55 - 432.12) ms434.36 ± (433.57 - 435.14) ms+0.7%✅⬆️
runtime.dotnet.exceptions.count4 ± (4 - 4)4 ± (4 - 4)+0.0%✅
runtime.dotnet.mem.committed47.93 ± (47.90 - 47.95) MB47.89 ± (47.87 - 47.92) MB-0.1%✅
runtime.dotnet.threads.count29 ± (29 - 29)29 ± (29 - 29)-0.3%✅
Comparison explanation

Execution-time benchmarks measure the whole time it takes to execute a program, and are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are highlighted in **red**. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

Duration charts
FakeDbCommand (.NET Framework 4.8) 
gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7875) - mean (75ms)  : 69, 82
    master - mean (75ms)  : 69, 81

    section Bailout
    This PR (7875) - mean (80ms)  : 74, 85
    master - mean (80ms)  : 75, 85

    section CallTarget+Inlining+NGEN
    This PR (7875) - mean (1,060ms)  : 989, 1132
    master - mean (1,059ms)  : 1000, 1118
Loading
FakeDbCommand (.NET Core 3.1) 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7875) - mean (117ms)  : 108, 125
    master - mean (118ms)  : 111, 126

    section Bailout
    This PR (7875) - mean (117ms)  : 110, 125
    master - mean (117ms)  : 111, 123

    section CallTarget+Inlining+NGEN
    This PR (7875) - mean (758ms)  : 705, 810
    master - mean (762ms)  : 729, 794
Loading
FakeDbCommand (.NET 6) 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7875) - mean (104ms)  : 97, 111
    master - mean (105ms)  : 98, 111

    section Bailout
    This PR (7875) - mean (107ms)  : 99, 115
    master - mean (106ms)  : 99, 114

    section CallTarget+Inlining+NGEN
    This PR (7875) - mean (707ms)  : 676, 738
    master - mean (706ms)  : 674, 739
Loading
FakeDbCommand (.NET 8) 
gantt
    title Execution time (ms) FakeDbCommand (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7875) - mean (104ms)  : 96, 111
    master - mean (104ms)  : 99, 109

    section Bailout
    This PR (7875) - mean (104ms)  : 98, 111
    master - mean (105ms)  : 100, 110

    section CallTarget+Inlining+NGEN
    This PR (7875) - mean (679ms)  : 634, 724
    master - mean (689ms)  : 658, 720
Loading
HttpMessageHandler (.NET Framework 4.8) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7875) - mean (193ms)  : 189, 197
    master - mean (194ms)  : 189, 198

    section Bailout
    This PR (7875) - mean (198ms)  : 193, 203
    master - mean (197ms)  : 194, 200

    section CallTarget+Inlining+NGEN
    This PR (7875) - mean (1,115ms)  : 1058, 1172
    master - mean (1,120ms)  : 1056, 1184
Loading
HttpMessageHandler (.NET Core 3.1) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7875) - mean (277ms)  : 272, 281
    master - mean (278ms)  : 272, 284

    section Bailout
    This PR (7875) - mean (278ms)  : 275, 281
    master - mean (278ms)  : 274, 282

    section CallTarget+Inlining+NGEN
    This PR (7875) - mean (903ms)  : 858, 948
    master - mean (909ms)  : 868, 949
Loading
HttpMessageHandler (.NET 6) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7875) - mean (271ms)  : 266, 277
    master - mean (271ms)  : 267, 276

    section Bailout
    This PR (7875) - mean (271ms)  : 266, 275
    master - mean (271ms)  : 266, 275

    section CallTarget+Inlining+NGEN
    This PR (7875) - mean (879ms)  : 845, 914
    master - mean (882ms)  : 845, 920
Loading
HttpMessageHandler (.NET 8) 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8)
    dateFormat  x
    axisFormat %Q
    todayMarker off
    section Baseline
    This PR (7875) - mean (271ms)  : 266, 275
    master - mean (270ms)  : 266, 275

    section Bailout
    This PR (7875) - mean (270ms)  : 265, 274
    master - mean (270ms)  : 267, 274

    section CallTarget+Inlining+NGEN
    This PR (7875) - mean (837ms)  : 812, 862
    master - mean (829ms)  : 800, 859
Loading

@datadog-official
Copy link

datadog-official bot commented Nov 27, 2025 •
edited by datadog-datadog-prod-us1 bot
Loading

⚠️ Tests

⚠️ Warnings

❄️ 1 New flaky test detected

DisableTelemetry from Datadog.Trace.Tools.dd_dotnet.ArtifactTests.CreatedumpTests (Datadog) 
The target process unexpectedly exited

ℹ️ Info

🧪 All tests passed

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: fbfa662 | Docs | Datadog PR Page | Was this helpful? Give us feedback!

@NachoEchevarria NachoEchevarria merged commit 8697ca8 into master Nov 28, 2025
152 checks passed
@NachoEchevarria NachoEchevarria deleted the nacho/IncreaseRetrisInStackExchangeTest branch November 28, 2025 13:45
@github-actions github-actions bot added this to the vNext-v3 milestone Nov 28, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@andrewlock andrewlock andrewlock approved these changes

Assignees

No one assigned

Labels

area:tests unit tests, integration tests

Projects

None yet

Milestone

3.33.0

Development

Successfully merging this pull request may close these issues.

2 participants

@NachoEchevarria @andrewlock