[Test Optimization] MsTest support updates by tonyredondo · Pull Request #7663 · DataDog/dd-trace-dotnet

tonyredondo · 2025-10-16T09:42:37Z

Summary of changes

Extend the MSTest package matrix (generator definitions, props, and generated test enumerations) to track new 4.x releases and keep the integration samples in sync.
Update the MSTest V2 calltarget instrumentation to support the async ExecuteAsync path introduced in MSTest 4.x, including new duck-typed interfaces, retry-state handling, and regenerated instrumentation metadata/calltarget tables.
Refresh the sample MSTest test projects to default to 4.x, add async test method attribute overrides, and enable additional CI diagnostics for the corresponding integration tests.

Reason for change

MSTest 4.x introduced asynchronous execution APIs that bypassed our existing instrumentation and were not yet part of the supported version set; without these updates we miss spans/metadata for customers adopting the new framework.

Implementation details

Bumped the MSTest package version ranges (up to <5.0.0) and added explicit 4.x entries across generator inputs, props, and generated C# enumerations used by the integration test harness.
Introduced new duck-typing interfaces for MSTest 4.x, split the calltarget implementation to share logic between Execute and ExecuteAsync, and regenerated instrumentation/calltarget metadata files to cover the new methods.
Adjusted the sample MSTest projects and tests (conditional compilation with MSTEST_ASYNC, async overrides, environment configuration) so the integration suite exercises the new async pipeline.

Test coverage

Current tests now runs on the new versions.

Other details

dd-trace-dotnet-ci-bot · 2025-10-20T13:59:14Z

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

Welch test with statistical test for significance of 5%
Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7663) - mean (74ms)  : 72, 76
     .   : milestone, 74,
    master - mean (75ms)  : 73, 76
     .   : milestone, 75,

    section Baseline
    This PR (7663) - mean (70ms)  : 68, 73
     .   : milestone, 70,
    master - mean (70ms)  : 68, 73
     .   : milestone, 70,

    section CallTarget+Inlining+NGEN
    This PR (7663) - mean (1,068ms)  : 1011, 1126
     .   : milestone, 1068,
    master - mean (1,072ms)  : 991, 1152
     .   : milestone, 1072,

gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7663) - mean (110ms)  : 108, 112
     .   : milestone, 110,
    master - mean (110ms)  : 107, 113
     .   : milestone, 110,

    section Baseline
    This PR (7663) - mean (109ms)  : 106, 112
     .   : milestone, 109,
    master - mean (109ms)  : 105, 113
     .   : milestone, 109,

    section CallTarget+Inlining+NGEN
    This PR (7663) - mean (757ms)  : 729, 784
     .   : milestone, 757,
    master - mean (753ms)  : 725, 780
     .   : milestone, 753,

gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7663) - mean (99ms)  : 96, 101
     .   : milestone, 99,
    master - mean (97ms)  : 95, 100
     .   : milestone, 97,

    section Baseline
    This PR (7663) - mean (98ms)  : 96, 100
     .   : milestone, 98,
    master - mean (97ms)  : 93, 101
     .   : milestone, 97,

    section CallTarget+Inlining+NGEN
    This PR (7663) - mean (730ms)  : 665, 794
     .   : milestone, 730,
    master - mean (712ms)  : 687, 737
     .   : milestone, 712,

gantt
    title Execution time (ms) FakeDbCommand (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7663) - mean (98ms)  : 95, 100
     .   : milestone, 98,
    master - mean (97ms)  : 95, 99
     .   : milestone, 97,

    section Baseline
    This PR (7663) - mean (97ms)  : 94, 99
     .   : milestone, 97,
    master - mean (96ms)  : 93, 98
     .   : milestone, 96,

    section CallTarget+Inlining+NGEN
    This PR (7663) - mean (672ms)  : 657, 688
     .   : milestone, 672,
    master - mean (677ms)  : 664, 690
     .   : milestone, 677,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7663) - mean (196ms)  : 193, 198
     .   : milestone, 196,
    master - mean (196ms)  : 193, 198
     .   : milestone, 196,

    section Baseline
    This PR (7663) - mean (194ms)  : 189, 198
     .   : milestone, 194,
    master - mean (192ms)  : 188, 197
     .   : milestone, 192,

    section CallTarget+Inlining+NGEN
    This PR (7663) - mean (1,178ms)  : 1097, 1258
     .   : milestone, 1178,
    master - mean (1,168ms)  : 1109, 1226
     .   : milestone, 1168,

gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7663) - mean (277ms)  : 273, 281
     .   : milestone, 277,
    master - mean (276ms)  : 274, 279
     .   : milestone, 276,

    section Baseline
    This PR (7663) - mean (277ms)  : 272, 282
     .   : milestone, 277,
    master - mean (276ms)  : 270, 283
     .   : milestone, 276,

    section CallTarget+Inlining+NGEN
    This PR (7663) - mean (944ms)  : 899, 988
     .   : milestone, 944,
    master - mean (946ms)  : 905, 986
     .   : milestone, 946,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7663) - mean (271ms)  : 265, 276
     .   : milestone, 271,
    master - mean (270ms)  : 265, 276
     .   : milestone, 270,

    section Baseline
    This PR (7663) - mean (270ms)  : 264, 276
     .   : milestone, 270,
    master - mean (269ms)  : 265, 274
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (7663) - mean (929ms)  : 879, 978
     .   : milestone, 929,
    master - mean (928ms)  : 884, 973
     .   : milestone, 928,

gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7663) - mean (269ms)  : 265, 272
     .   : milestone, 269,
    master - mean (269ms)  : 264, 274
     .   : milestone, 269,

    section Baseline
    This PR (7663) - mean (269ms)  : 265, 273
     .   : milestone, 269,
    master - mean (270ms)  : 266, 273
     .   : milestone, 270,

    section CallTarget+Inlining+NGEN
    This PR (7663) - mean (855ms)  : 834, 876
     .   : milestone, 855,
    master - mean (861ms)  : 844, 878
     .   : milestone, 861,

pr-commenter · 2025-10-20T21:47:41Z

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7663 compared to master:

1 benchmarks are faster, with geometric mean 2.392
1 benchmarks are slower, with geometric mean 1.144
6 benchmarks have fewer allocations
10 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

Mann–Whitney U test with statistical test for significance of 5%
Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7663

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1	5.69 KB	5.73 KB	40 B	0.70%

Fewer allocations 🎉 in #7663

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472	6.03 KB	5.99 KB	-47 B	-0.78%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`StartStopWithChild`	net6.0	10.6μs	59ns	359ns	0	0	0	5.5 KB
master	`StartStopWithChild`	netcoreapp3.1	14.2μs	44.7ns	173ns	0	0	0	5.69 KB
master	`StartStopWithChild`	net472	21.9μs	121ns	754ns	1	0.334	0.111	6.03 KB
#7663	`StartStopWithChild`	net6.0	10.6μs	59.8ns	423ns	0	0	0	5.53 KB
#7663	`StartStopWithChild`	netcoreapp3.1	13.6μs	67.8ns	288ns	0	0	0	5.73 KB
#7663	`StartStopWithChild`	net472	22.8μs	119ns	583ns	1	0.301	0.1	5.99 KB

Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	927μs	136ns	492ns	2.71 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	1.03ms	308ns	1.19μs	2.7 KB
master	`WriteAndFlushEnrichedTraces`	net472	1.19ms	220ns	853ns	3.31 KB
#7663	`WriteAndFlushEnrichedTraces`	net6.0	936μs	235ns	911ns	2.71 KB
#7663	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	1.02ms	713ns	2.67μs	2.7 KB
#7663	`WriteAndFlushEnrichedTraces`	net472	1.2ms	86ns	333ns	3.31 KB

Benchmarks.Trace.Asm.AppSecBodyBenchmark - Unknown 🤷 More allocations ⚠️

More allocations ⚠️ in #7663

Benchmark	Diff Allocated	Change	Change %
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net6.0	179.56 KB	179.56 KB	∞
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑netcoreapp3.1	185.32 KB	185.32 KB	∞
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net472	205.41 KB	205.41 KB	∞
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net6.0	183.08 KB	183.08 KB	∞
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑netcoreapp3.1	188.75 KB	188.75 KB	∞
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net472	208.94 KB	208.94 KB	∞

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Gen 2	Allocated
master	`AllCycleSimpleBody`	net6.0	N/A	N/A	N/A	NaN	NaN	NaN	0 b
master	`AllCycleSimpleBody`	netcoreapp3.1	N/A	N/A	N/A	NaN	NaN	NaN	0 b
master	`AllCycleSimpleBody`	net472	N/A	N/A	N/A	NaN	NaN	NaN	0 b
master	`AllCycleMoreComplexBody`	net6.0	N/A	N/A	N/A	NaN	NaN	NaN	0 b
master	`AllCycleMoreComplexBody`	netcoreapp3.1	N/A	N/A	N/A	NaN	NaN	NaN	0 b
master	`AllCycleMoreComplexBody`	net472	N/A	N/A	N/A	NaN	NaN	NaN	0 b
master	`ObjectExtractorSimpleBody`	net6.0	317ns	1.73ns	10.1ns	0	0	0	280 B
master	`ObjectExtractorSimpleBody`	netcoreapp3.1	396ns	2ns	9.36ns	0	0	0	272 B
master	`ObjectExtractorSimpleBody`	net472	305ns	0.0744ns	0.288ns	0.0437	0	0	281 B
master	`ObjectExtractorMoreComplexBody`	net6.0	6.45μs	24.1ns	90.3ns	0	0	0	3.78 KB
master	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	7.8μs	37.6ns	150ns	0	0	0	3.69 KB
master	`ObjectExtractorMoreComplexBody`	net472	6.71μs	1.95ns	7.54ns	0.603	0	0	3.8 KB
#7663	`AllCycleSimpleBody`	net6.0	355μs	792ns	3.07μs	0	0	0	179.56 KB
#7663	`AllCycleSimpleBody`	netcoreapp3.1	490μs	2.07μs	8μs	0	0	0	185.32 KB
#7663	`AllCycleSimpleBody`	net472	466μs	67.4ns	261ns	32.4	0	0	205.41 KB
#7663	`AllCycleMoreComplexBody`	net6.0	363μs	291ns	1.13μs	0	0	0	183.08 KB
#7663	`AllCycleMoreComplexBody`	netcoreapp3.1	534μs	2.88μs	14.9μs	0	0	0	188.75 KB
#7663	`AllCycleMoreComplexBody`	net472	475μs	136ns	525ns	32.4	0	0	208.94 KB
#7663	`ObjectExtractorSimpleBody`	net6.0	323ns	0.248ns	0.96ns	0	0	0	280 B
#7663	`ObjectExtractorSimpleBody`	netcoreapp3.1	397ns	2.11ns	11ns	0	0	0	272 B
#7663	`ObjectExtractorSimpleBody`	net472	301ns	0.0375ns	0.13ns	0.0441	0	0	281 B
#7663	`ObjectExtractorMoreComplexBody`	net6.0	6.25μs	31.7ns	145ns	0	0	0	3.78 KB
#7663	`ObjectExtractorMoreComplexBody`	netcoreapp3.1	7.91μs	38ns	157ns	0	0	0	3.69 KB
#7663	`ObjectExtractorMoreComplexBody`	net472	6.71μs	2.74ns	9.86ns	0.572	0	0	3.8 KB

Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EncodeArgs`	net6.0	77.5μs	239ns	924ns	0	32.4 KB
master	`EncodeArgs`	netcoreapp3.1	97.2μs	192ns	742ns	0	32.4 KB
master	`EncodeArgs`	net472	109μs	9.69ns	37.5ns	4.93	32.5 KB
master	`EncodeLegacyArgs`	net6.0	147μs	191ns	740ns	0	2.15 KB
master	`EncodeLegacyArgs`	netcoreapp3.1	199μs	157ns	607ns	0	2.14 KB
master	`EncodeLegacyArgs`	net472	261μs	91ns	353ns	0	2.16 KB
#7663	`EncodeArgs`	net6.0	76.6μs	215ns	833ns	0	32.4 KB
#7663	`EncodeArgs`	netcoreapp3.1	96.5μs	199ns	769ns	0	32.4 KB
#7663	`EncodeArgs`	net472	108μs	54.5ns	211ns	4.88	32.51 KB
#7663	`EncodeLegacyArgs`	net6.0	143μs	80ns	299ns	0	2.15 KB
#7663	`EncodeLegacyArgs`	netcoreapp3.1	199μs	84.8ns	328ns	0	2.14 KB
#7663	`EncodeLegacyArgs`	net472	262μs	18.7ns	69.9ns	0	2.16 KB

Benchmarks.Trace.Asm.AppSecWafBenchmark - Faster 🎉 Same allocations ✔️

Faster 🎉 in #7663

Benchmark	base/diff	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.Asm.AppSecWafBenchmark.RunWafRealisticBenchmarkWithAttack‑netcoreapp3.1	2.392	709,710.42	296,690.55

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`RunWafRealisticBenchmark`	net6.0	393μs	42.6ns	154ns	4.55 KB
master	`RunWafRealisticBenchmark`	netcoreapp3.1	849μs	4.43μs	23μs	4.48 KB
master	`RunWafRealisticBenchmark`	net472	427μs	41.5ns	155ns	4.66 KB
master	`RunWafRealisticBenchmarkWithAttack`	net6.0	284μs	47.8ns	179ns	2.24 KB
master	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	705μs	3.34μs	14.2μs	2.22 KB
master	`RunWafRealisticBenchmarkWithAttack`	net472	308μs	32.6ns	126ns	2.29 KB
#7663	`RunWafRealisticBenchmark`	net6.0	399μs	37.2ns	139ns	4.55 KB
#7663	`RunWafRealisticBenchmark`	netcoreapp3.1	850μs	2.7μs	9.35μs	4.48 KB
#7663	`RunWafRealisticBenchmark`	net472	434μs	48.8ns	189ns	4.66 KB
#7663	`RunWafRealisticBenchmarkWithAttack`	net6.0	286μs	39.7ns	154ns	2.24 KB
#7663	`RunWafRealisticBenchmarkWithAttack`	netcoreapp3.1	297μs	58.2ns	226ns	2.22 KB
#7663	`RunWafRealisticBenchmarkWithAttack`	net472	308μs	33.3ns	125ns	2.29 KB

Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`SendRequest`	net6.0	61.6μs	38.3ns	143ns	14.52 KB
master	`SendRequest`	netcoreapp3.1	71.6μs	299ns	1.46μs	17.42 KB
master	`SendRequest`	net472	0.00319ns	0.00111ns	0.00428ns	0 b
#7663	`SendRequest`	net6.0	60.7μs	50.1ns	187ns	14.52 KB
#7663	`SendRequest`	netcoreapp3.1	71.7μs	342ns	1.45μs	17.42 KB
#7663	`SendRequest`	net472	0.000388ns	0.000261ns	0.00101ns	0 b

Benchmarks.Trace.CharSliceBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7663

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net6.0	4 B	5 B	1 B	25.00%

Fewer allocations 🎉 in #7663

Benchmark	Base Allocated	Change	Change %
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSlice‑net472	73 B	-73 B	-100.00%
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑netcoreapp3.1	17 B	-17 B	-100.00%
Benchmarks.Trace.CharSliceBenchmark.OptimizedCharSliceWithPool‑net472	47 B	-47 B	-100.00%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`OriginalCharSlice`	net6.0	1.94ms	2.63μs	9.84μs	0	640.01 KB
master	`OriginalCharSlice`	netcoreapp3.1	2.14ms	4.85μs	18.8μs	0	640 KB
master	`OriginalCharSlice`	net472	2.67ms	128ns	461ns	100	641.95 KB
master	`OptimizedCharSlice`	net6.0	1.39ms	212ns	823ns	0	4 B
master	`OptimizedCharSlice`	netcoreapp3.1	1.73ms	456ns	1.71μs	0	1 B
master	`OptimizedCharSlice`	net472	1.95ms	259ns	1μs	0	73 B
master	`OptimizedCharSliceWithPool`	net6.0	825μs	34.4ns	129ns	0	4 B
master	`OptimizedCharSliceWithPool`	netcoreapp3.1	898μs	96.2ns	360ns	0	17 B
master	`OptimizedCharSliceWithPool`	net472	1.14ms	123ns	475ns	0	47 B
#7663	`OriginalCharSlice`	net6.0	1.91ms	893ns	3.34μs	0	640.01 KB
#7663	`OriginalCharSlice`	netcoreapp3.1	2.15ms	2.86μs	10.7μs	0	640 KB
#7663	`OriginalCharSlice`	net472	2.65ms	147ns	550ns	100	641.95 KB
#7663	`OptimizedCharSlice`	net6.0	1.36ms	775ns	3μs	0	4 B
#7663	`OptimizedCharSlice`	netcoreapp3.1	1.85ms	283ns	1.1μs	0	1 B
#7663	`OptimizedCharSlice`	net472	1.95ms	242ns	936ns	0	0 b
#7663	`OptimizedCharSliceWithPool`	net6.0	829μs	45.2ns	169ns	0	5 B
#7663	`OptimizedCharSliceWithPool`	netcoreapp3.1	859μs	79.4ns	308ns	0	0 b
#7663	`OptimizedCharSliceWithPool`	net472	1.15ms	156ns	604ns	0	0 b

Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7663

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0	42.4 KB	41.78 KB	-616 B	-1.45%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`WriteAndFlushEnrichedTraces`	net6.0	682μs	1.43μs	5.54μs	0	42.4 KB
master	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	639μs	2.59μs	10μs	0	42.03 KB
master	`WriteAndFlushEnrichedTraces`	net472	937μs	4.53μs	19.2μs	8.33	56.12 KB
#7663	`WriteAndFlushEnrichedTraces`	net6.0	650μs	552ns	1.99μs	0	41.78 KB
#7663	`WriteAndFlushEnrichedTraces`	netcoreapp3.1	682μs	3.91μs	32μs	0	42.01 KB
#7663	`WriteAndFlushEnrichedTraces`	net472	871μs	2.32μs	8.98μs	8.33	56.01 KB

Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Gen 1	Allocated
master	`ExecuteNonQuery`	net6.0	1.88μs	4.98ns	17.9ns	0	0	1.02 KB
master	`ExecuteNonQuery`	netcoreapp3.1	2.59μs	7ns	27.1ns	0	0	1.02 KB
master	`ExecuteNonQuery`	net472	2.84μs	5.91ns	22.9ns	0.153	0.0139	987 B
#7663	`ExecuteNonQuery`	net6.0	1.94μs	8.41ns	31.5ns	0	0	1.02 KB
#7663	`ExecuteNonQuery`	netcoreapp3.1	2.66μs	13.3ns	56.3ns	0	0	1.02 KB
#7663	`ExecuteNonQuery`	net472	2.86μs	6.78ns	26.3ns	0.143	0.0143	987 B

Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`CallElasticsearch`	net6.0	1.8μs	6.28ns	24.3ns	0	1.03 KB
master	`CallElasticsearch`	netcoreapp3.1	2.3μs	11.1ns	42.9ns	0	1.03 KB
master	`CallElasticsearch`	net472	3.53μs	2.12ns	7.65ns	0.159	1.04 KB
master	`CallElasticsearchAsync`	net6.0	1.82μs	8.52ns	33ns	0	1.01 KB
master	`CallElasticsearchAsync`	netcoreapp3.1	2.43μs	12ns	44.7ns	0	1.08 KB
master	`CallElasticsearchAsync`	net472	3.78μs	5.15ns	19.9ns	0.169	1.1 KB
#7663	`CallElasticsearch`	net6.0	1.76μs	6.18ns	23.1ns	0	1.03 KB
#7663	`CallElasticsearch`	netcoreapp3.1	2.3μs	7.97ns	30.9ns	0	1.03 KB
#7663	`CallElasticsearch`	net472	3.61μs	4.75ns	18.4ns	0.163	1.04 KB
#7663	`CallElasticsearchAsync`	net6.0	1.82μs	9.33ns	42.8ns	0	1.01 KB
#7663	`CallElasticsearchAsync`	netcoreapp3.1	2.4μs	11.2ns	43.4ns	0	1.08 KB
#7663	`CallElasticsearchAsync`	net472	3.82μs	4.25ns	16.4ns	0.171	1.1 KB

Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`ExecuteAsync`	net6.0	1.82μs	9.71ns	53.2ns	0	952 B
master	`ExecuteAsync`	netcoreapp3.1	2.34μs	8.72ns	32.6ns	0	952 B
master	`ExecuteAsync`	net472	2.63μs	3.1ns	12ns	0.144	915 B
#7663	`ExecuteAsync`	net6.0	1.77μs	8.44ns	32.7ns	0	952 B
#7663	`ExecuteAsync`	netcoreapp3.1	2.43μs	10.6ns	41ns	0	952 B
#7663	`ExecuteAsync`	net472	2.61μs	1.12ns	4.35ns	0.145	915 B

Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendAsync`	net6.0	6.85μs	9.67ns	37.5ns	0	2.36 KB
master	`SendAsync`	netcoreapp3.1	8.64μs	23.2ns	86.9ns	0	2.9 KB
master	`SendAsync`	net472	12.5μs	11.9ns	46.2ns	0.498	3.18 KB
#7663	`SendAsync`	net6.0	6.75μs	8.58ns	32.1ns	0	2.36 KB
#7663	`SendAsync`	netcoreapp3.1	8.38μs	13.5ns	52.2ns	0	2.9 KB
#7663	`SendAsync`	net472	12.2μs	11.4ns	44.3ns	0.49	3.18 KB

Benchmarks.Trace.Iast.StringAspectsBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7663

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0	43.78 KB	72.77 KB	28.99 KB	66.23%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1	274.43 KB	275.93 KB	1.5 KB	0.55%

Fewer allocations 🎉 in #7663

Benchmark	Base Allocated	Diff Allocated	Change	Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0	285.99 KB	248.22 KB	-37.77 KB	-13.21%

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`StringConcatBenchmark`	net6.0	42.1μs	207ns	1.19μs	43.78 KB
master	`StringConcatBenchmark`	netcoreapp3.1	49.2μs	289ns	2.43μs	42.83 KB
master	`StringConcatBenchmark`	net472	57.2μs	271ns	1.12μs	57.34 KB
master	`StringConcatAspectBenchmark`	net6.0	494μs	2.27μs	8.51μs	285.99 KB
master	`StringConcatAspectBenchmark`	netcoreapp3.1	502μs	779ns	2.81μs	274.43 KB
master	`StringConcatAspectBenchmark`	net472	403μs	1.87μs	8.59μs	288.14 KB
#7663	`StringConcatBenchmark`	net6.0	45μs	264ns	2.5μs	72.77 KB
#7663	`StringConcatBenchmark`	netcoreapp3.1	49.4μs	279ns	1.74μs	42.9 KB
#7663	`StringConcatBenchmark`	net472	57.2μs	284ns	1.24μs	57.34 KB
#7663	`StringConcatAspectBenchmark`	net6.0	447μs	1.95μs	7.04μs	248.22 KB
#7663	`StringConcatAspectBenchmark`	netcoreapp3.1	510μs	1.43μs	5.17μs	275.93 KB
#7663	`StringConcatAspectBenchmark`	net472	405μs	2.01μs	8.75μs	286.72 KB

Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	2.59μs	12.9ns	54.7ns	0	1.7 KB
master	`EnrichedLog`	netcoreapp3.1	3.64μs	17.4ns	67.2ns	0	1.7 KB
master	`EnrichedLog`	net472	3.86μs	5.9ns	22.9ns	0.249	1.64 KB
#7663	`EnrichedLog`	net6.0	2.61μs	1.04ns	3.61ns	0	1.7 KB
#7663	`EnrichedLog`	netcoreapp3.1	3.71μs	12.3ns	47.6ns	0	1.7 KB
#7663	`EnrichedLog`	net472	3.85μs	2.53ns	9.81ns	0.249	1.64 KB

Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Allocated
master	`EnrichedLog`	net6.0	122μs	59.4ns	230ns	4.31 KB
master	`EnrichedLog`	netcoreapp3.1	128μs	213ns	769ns	4.31 KB
master	`EnrichedLog`	net472	165μs	42.3ns	164ns	4.51 KB
#7663	`EnrichedLog`	net6.0	125μs	41ns	154ns	4.31 KB
#7663	`EnrichedLog`	netcoreapp3.1	129μs	47.7ns	185ns	4.31 KB
#7663	`EnrichedLog`	net472	169μs	53.6ns	208ns	4.52 KB

Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	5.1μs	4.77ns	18.5ns	0	2.26 KB
master	`EnrichedLog`	netcoreapp3.1	6.86μs	29.2ns	113ns	0	2.26 KB
master	`EnrichedLog`	net472	7.75μs	4.82ns	18.7ns	0.309	2.08 KB
#7663	`EnrichedLog`	net6.0	4.9μs	7.52ns	29.1ns	0	2.26 KB
#7663	`EnrichedLog`	netcoreapp3.1	6.88μs	14.3ns	55.3ns	0	2.26 KB
#7663	`EnrichedLog`	net472	7.48μs	5.57ns	20.8ns	0.299	2.08 KB

Benchmarks.Trace.RedisBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #7663

Benchmark	diff/base	Base Median (ns)	Diff Median (ns)	Modality
Benchmarks.Trace.RedisBenchmark.SendReceive‑net472	1.144	3,091.84	3,538.00

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`SendReceive`	net6.0	2.06μs	9.32ns	36.1ns	0	1.2 KB
master	`SendReceive`	netcoreapp3.1	2.65μs	7.15ns	27.7ns	0	1.2 KB
master	`SendReceive`	net472	3.09μs	2.99ns	11.6ns	0.185	1.2 KB
#7663	`SendReceive`	net6.0	2.06μs	3.78ns	14.6ns	0	1.2 KB
#7663	`SendReceive`	netcoreapp3.1	2.62μs	7ns	27.1ns	0	1.2 KB
#7663	`SendReceive`	net472	3.53μs	5.07ns	19.6ns	0.177	1.2 KB

Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`EnrichedLog`	net6.0	4.24μs	4.35ns	15.7ns	0	1.58 KB
master	`EnrichedLog`	netcoreapp3.1	5.71μs	17.4ns	67.3ns	0	1.63 KB
master	`EnrichedLog`	net472	6.75μs	7.58ns	29.4ns	0.303	2.03 KB
#7663	`EnrichedLog`	net6.0	4.35μs	4.7ns	18.2ns	0	1.58 KB
#7663	`EnrichedLog`	netcoreapp3.1	5.68μs	13.6ns	52.6ns	0	1.63 KB
#7663	`EnrichedLog`	net472	6.95μs	10.1ns	39.2ns	0.312	2.03 KB

Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`StartFinishSpan`	net6.0	759ns	3.9ns	19.1ns	0	576 B
master	`StartFinishSpan`	netcoreapp3.1	958ns	3.86ns	14.9ns	0	576 B
master	`StartFinishSpan`	net472	967ns	0.0575ns	0.207ns	0.0871	578 B
master	`StartFinishScope`	net6.0	915ns	4.62ns	20.7ns	0	696 B
master	`StartFinishScope`	netcoreapp3.1	1.19μs	6.05ns	27ns	0	696 B
master	`StartFinishScope`	net472	1.21μs	1.09ns	4.09ns	0.103	658 B
#7663	`StartFinishSpan`	net6.0	753ns	0.325ns	1.17ns	0	576 B
#7663	`StartFinishSpan`	netcoreapp3.1	980ns	4.5ns	17.4ns	0	576 B
#7663	`StartFinishSpan`	net472	899ns	0.397ns	1.54ns	0.0902	578 B
#7663	`StartFinishScope`	net6.0	888ns	4.61ns	23ns	0	696 B
#7663	`StartFinishScope`	netcoreapp3.1	1.17μs	5.96ns	26ns	0	696 B
#7663	`StartFinishScope`	net472	1.11μs	0.303ns	1.17ns	0.101	658 B

Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch	Method	Toolchain	Mean	StdError	StdDev	Gen 0	Allocated
master	`RunOnMethodBegin`	net6.0	1.07μs	5.06ns	19.6ns	0	696 B
master	`RunOnMethodBegin`	netcoreapp3.1	1.44μs	6.72ns	26.9ns	0	696 B
master	`RunOnMethodBegin`	net472	1.44μs	0.835ns	3.23ns	0.101	658 B
#7663	`RunOnMethodBegin`	net6.0	1.06μs	5.49ns	25.2ns	0	696 B
#7663	`RunOnMethodBegin`	netcoreapp3.1	1.41μs	6.34ns	24.6ns	0	696 B
#7663	`RunOnMethodBegin`	net472	1.43μs	0.822ns	2.96ns	0.101	658 B

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

tracer/test/test-applications/integrations/Samples.MSTestTests/Samples.MSTestTests.csproj

andrewlock

Awesome, thanks!

andrewlock · 2025-10-22T09:06:28Z

...ce/ClrProfiler/AutoInstrumentation/Testing/MsTestV2/TestMethodAttributeExecuteIntegration.cs

+        }
+
+        var testRunnerState = new TestRunnerState(testMethodProxy, MsTestIntegration.OnMethodBegin(testMethodProxy, testMethodProxy.Type, isRetry: false));
+        return new CallTargetState(Tracer.Instance.InternalActiveScope, testRunnerState);


You previously were returning null as the scope here, and you already grabbed it and confirmed it was not a test scope previously, so this seems like a behaviour change. Is it intentional? 🤔

Totally, this code was in a sync integration before with the OnMethodEnd call closing the span and restoring the scope. Now due to the latest change we had to move this as an async counterpart meaning that we now have OnAsyncMethodEnd the issue with this is that the OnMethodBegin change the active scope with the new Span, but we need a way to restore it synchronously (because the async version contains a copy on write version of the scope, any restoration in that execution context doesn't affect the previous execution context), that's also the reason we call IntegrationOptions.RestoreScopeFromAsyncExecution(in state); in the Sync version. We don't need that line in the Async version because the CallTarget machinery is aware of this issue and that restoration is done automatically for all async integrations.

...rProfiler/AutoInstrumentation/Testing/MsTestV2/UnitTestDiscovererSendTestCasesIntegration.cs

tracer/test/test-applications/integrations/Samples.MSTestTests/Samples.MSTestTests.csproj

andrewlock · 2025-10-22T09:13:37Z

tracer/test/test-applications/integrations/Samples.MSTestTests/TestSuite.cs

+#if MSTEST_ASYNC
+    public override async Task<TestResult[]> ExecuteAsync(ITestMethod testMethod)
+    {
+        var results = await base.ExecuteAsync(testMethod).ConfigureAwait(false);


Silly question - I assume MSTest doesn't have a sync context right (e.g. like XUnit does)

Not yet, I'm waiting for 4.0.2 for them to do that change 🤣

[Test Optimization] MsTest support updates

3cc1d9a

github-actions bot added the area:ci-visibility label Oct 16, 2025

tonyredondo added 5 commits October 16, 2025 14:56

Bump MsTest up-to 3.11.0

b56c9cb

enable debug

e422991

fix for 3.11.0

2afd4e8

fix

ce61a67

Add support for MSTest v4

a7f64e2

DataDog deleted a comment from dd-trace-dotnet-ci-bot bot Oct 20, 2025

DataDog deleted a comment from pr-commenter bot Oct 20, 2025

DataDog deleted a comment from datadog-datadog-prod-us1 bot Oct 20, 2025

fix

9ffa18a

tonyredondo added 2 commits October 20, 2025 17:05

fixes

d88d5b3

restore scope because async call

be45272

tonyredondo added 4 commits October 21, 2025 10:29

fixes

4cddc91

fix

80714f1

fixes

36892b1

fix

bd31867

tonyredondo force-pushed the tony/mstest-support-updates branch from 47ffbe8 to bd31867 Compare October 21, 2025 12:15

tonyredondo marked this pull request as ready for review October 21, 2025 13:35

tonyredondo requested review from a team as code owners October 21, 2025 13:35

Merge branch 'master' into tony/mstest-support-updates

408fc4b

chatgpt-codex-connector bot reviewed Oct 21, 2025

View reviewed changes

tracer/test/test-applications/integrations/Samples.MSTestTests/Samples.MSTestTests.csproj Show resolved Hide resolved

andrewlock approved these changes Oct 22, 2025

View reviewed changes

changes from review

dd5f0ab

tonyredondo merged commit fc2be35 into master Oct 22, 2025
154 checks passed

tonyredondo deleted the tony/mstest-support-updates branch October 22, 2025 16:20

github-actions bot added this to the vNext-v3 milestone Oct 22, 2025

andrewlock added the type:enhancement Improvement to an existing feature label Nov 3, 2025

Conversation

tonyredondo commented Oct 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary of changes

Reason for change

Implementation details

Test coverage

Other details

Uh oh!

dd-trace-dotnet-ci-bot bot commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Execution-Time Benchmarks Report ⏱️

Uh oh!

pr-commenter bot commented Oct 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmark details

More allocations ⚠️ in #7663

Fewer allocations 🎉 in #7663

Raw results

Raw results

More allocations ⚠️ in #7663

Raw results

Raw results

Faster 🎉 in #7663

Raw results

Raw results

More allocations ⚠️ in #7663

Fewer allocations 🎉 in #7663

Raw results

Fewer allocations 🎉 in #7663

Raw results

Raw results

Raw results

Raw results

Raw results

More allocations ⚠️ in #7663

Fewer allocations 🎉 in #7663

Raw results

Raw results

Raw results

Raw results

Slower ⚠️ in #7663

Raw results

Raw results

Raw results

Raw results

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

andrewlock left a comment

Choose a reason for hiding this comment

Uh oh!

andrewlock Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

tonyredondo Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

andrewlock Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

tonyredondo Oct 22, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

tonyredondo commented Oct 16, 2025 •

edited

Loading

dd-trace-dotnet-ci-bot bot commented Oct 20, 2025 •

edited

Loading

pr-commenter bot commented Oct 20, 2025 •

edited

Loading