Skip to content

Add SSI Appsec System tests#7153

Closed
e-n-0 wants to merge 2 commits into
masterfrom
flavien/add-ssi-appsec-systemtests
Closed

Add SSI Appsec System tests#7153
e-n-0 wants to merge 2 commits into
masterfrom
flavien/add-ssi-appsec-systemtests

Conversation

@e-n-0

@e-n-0 e-n-0 commented Jun 27, 2025

Copy link
Copy Markdown
Member

Summary of changes

Add the simple_onboarding_appsec to the gitlab workflow.

Reason for change

Add system tests scenario for SSI Appsec.

Implementation details

Test coverage

Other details

@e-n-0 e-n-0 requested a review from a team as a code owner June 27, 2025 13:32
@github-actions github-actions Bot added the area:builds project files, build scripts, pipelines, versioning, releases, packages label Jun 27, 2025
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jun 27, 2025 •
edited
Loading

Copy link
Copy Markdown

Datadog Report

Branch report: flavien/add-ssi-appsec-systemtests
Commit report: ea7b3ae
Test service: dd-trace-dotnet

✅ 0 Failed, 242157 Passed, 2365 Skipped, 15h 15m 24.96s Total Time

@dd-trace-dotnet-ci-bot

dd-trace-dotnet-ci-bot Bot commented Jun 27, 2025

Copy link
Copy Markdown

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7153) - mean (75ms)  : 73, 77
     .   : milestone, 75,
    master - mean (75ms)  : 73, 78
     .   : milestone, 75,

    section Baseline
    This PR (7153) - mean (71ms)  : 68, 75
     .   : milestone, 71,
    master - mean (71ms)  : 67, 76
     .   : milestone, 71,

    section CallTarget+Inlining+NGEN
    This PR (7153) - mean (1,032ms)  : 999, 1066
     .   : milestone, 1032,
    master - mean (1,025ms)  : 1004, 1045
     .   : milestone, 1025,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7153) - mean (109ms)  : 107, 112
     .   : milestone, 109,
    master - mean (108ms)  : 106, 111
     .   : milestone, 108,

    section Baseline
    This PR (7153) - mean (108ms)  : 105, 110
     .   : milestone, 108,
    master - mean (107ms)  : 104, 111
     .   : milestone, 107,

    section CallTarget+Inlining+NGEN
    This PR (7153) - mean (711ms)  : 689, 733
     .   : milestone, 711,
    master - mean (711ms)  : 683, 738
     .   : milestone, 711,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7153) - mean (96ms)  : 95, 98
     .   : milestone, 96,
    master - mean (96ms)  : 95, 98
     .   : milestone, 96,

    section Baseline
    This PR (7153) - mean (95ms)  : 93, 97
     .   : milestone, 95,
    master - mean (95ms)  : 92, 98
     .   : milestone, 95,

    section CallTarget+Inlining+NGEN
    This PR (7153) - mean (684ms)  : 656, 713
     .   : milestone, 684,
    master - mean (679ms)  : 657, 701
     .   : milestone, 679,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7153) - mean (192ms)  : 190, 195
     .   : milestone, 192,
    master - mean (193ms)  : 191, 195
     .   : milestone, 193,

    section Baseline
    This PR (7153) - mean (189ms)  : 184, 194
     .   : milestone, 189,
    master - mean (190ms)  : 186, 194
     .   : milestone, 190,

    section CallTarget+Inlining+NGEN
    This PR (7153) - mean (1,114ms)  : 1082, 1147
     .   : milestone, 1114,
    master - mean (1,114ms)  : 1079, 1149
     .   : milestone, 1114,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7153) - mean (270ms)  : 266, 275
     .   : milestone, 270,
    master - mean (270ms)  : 266, 273
     .   : milestone, 270,

    section Baseline
    This PR (7153) - mean (269ms)  : 266, 273
     .   : milestone, 269,
    master - mean (269ms)  : 266, 273
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (7153) - mean (891ms)  : 862, 921
     .   : milestone, 891,
    master - mean (884ms)  : 858, 910
     .   : milestone, 884,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7153) - mean (263ms)  : 260, 266
     .   : milestone, 263,
    master - mean (264ms)  : 261, 267
     .   : milestone, 264,

    section Baseline
    This PR (7153) - mean (263ms)  : 260, 266
     .   : milestone, 263,
    master - mean (263ms)  : 260, 266
     .   : milestone, 263,

    section CallTarget+Inlining+NGEN
    This PR (7153) - mean (879ms)  : 842, 916
     .   : milestone, 879,
    master - mean (878ms)  : 848, 909
     .   : milestone, 878,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7153) - mean (262ms)  : 259, 266
     .   : milestone, 262,
    master - mean (262ms)  : 258, 265
     .   : milestone, 262,

    section Baseline
    This PR (7153) - mean (262ms)  : 258, 266
     .   : milestone, 262,
    master - mean (262ms)  : 258, 266
     .   : milestone, 262,

    section CallTarget+Inlining+NGEN
    This PR (7153) - mean (779ms)  : 757, 801
     .   : milestone, 779,
    master - mean (779ms)  : 754, 804
     .   : milestone, 779,
Loading

@e-n-0 e-n-0 force-pushed the flavien/add-ssi-appsec-systemtests branch from ea7b3ae to 68e57b3 Compare July 1, 2025 11:04
@pr-commenter

pr-commenter Bot commented Jul 1, 2025

Copy link
Copy Markdown

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7153 compared to master:

  • 2 benchmarks are slower, with geometric mean 1.173
  • 43 benchmarks have fewer allocations
  • 4 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1 5.75 KB 5.7 KB -59 B -1.03%
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net6.0 5.58 KB 5.51 KB -71 B -1.27%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 11.2μs 59.7ns 322ns 0 0 0 5.58 KB
master StartStopWithChild netcoreapp3.1 14.1μs 67.5ns 286ns 0 0 0 5.75 KB
master StartStopWithChild net472 22.1μs 119ns 651ns 1.03 0.411 0.103 6.09 KB
#7153 StartStopWithChild net6.0 10.8μs 59ns 339ns 0 0 0 5.51 KB
#7153 StartStopWithChild netcoreapp3.1 13.6μs 66.2ns 273ns 0 0 0 5.7 KB
#7153 StartStopWithChild net472 22.7μs 91.9ns 344ns 0.891 0.223 0 6.1 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 3.33 KB 3.31 KB -23 B -0.69%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 927μs 26.6ns 103ns 0 0 0 2.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.04ms 404ns 1.56μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.25ms 357ns 1.38μs 0 0 0 3.33 KB
#7153 WriteAndFlushEnrichedTraces net6.0 925μs 105ns 405ns 0 0 0 2.71 KB
#7153 WriteAndFlushEnrichedTraces netcoreapp3.1 1.02ms 96.3ns 347ns 0 0 0 2.71 KB
#7153 WriteAndFlushEnrichedTraces net472 1.22ms 522ns 1.95μs 0 0 0 3.31 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net472 236.35 KB 239.66 KB 3.3 KB 1.40%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net472 239.87 KB 243.17 KB 3.3 KB 1.38%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 331μs 1.76μs 8.42μs 0 0 0 197.06 KB
master AllCycleSimpleBody netcoreapp3.1 510μs 1.45μs 5.61μs 0 0 0 204.77 KB
master AllCycleSimpleBody net472 436μs 119ns 460ns 36.6 2.16 0 236.35 KB
master AllCycleMoreComplexBody net6.0 338μs 1.76μs 8.8μs 0 0 0 200.56 KB
master AllCycleMoreComplexBody netcoreapp3.1 495μs 987ns 3.56μs 0 0 0 208.18 KB
master AllCycleMoreComplexBody net472 446μs 106ns 412ns 36.6 2.16 0 239.87 KB
master ObjectExtractorSimpleBody net6.0 311ns 1.77ns 12.3ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 409ns 1.93ns 8.2ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 303ns 0.175ns 0.676ns 0.0442 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.52μs 29.5ns 110ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.76μs 36.2ns 140ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.66μs 0.89ns 3.33ns 0.599 0 0 3.8 KB
#7153 AllCycleSimpleBody net6.0 334μs 1.73μs 8.28μs 0 0 0 197.61 KB
#7153 AllCycleSimpleBody netcoreapp3.1 507μs 2.42μs 10.3μs 0 0 0 205.37 KB
#7153 AllCycleSimpleBody net472 468μs 107ns 416ns 37 2.31 0 239.66 KB
#7153 AllCycleMoreComplexBody net6.0 344μs 1.71μs 7.47μs 0 0 0 201.11 KB
#7153 AllCycleMoreComplexBody netcoreapp3.1 510μs 1.59μs 6.17μs 0 0 0 208.78 KB
#7153 AllCycleMoreComplexBody net472 476μs 442ns 1.71μs 37 0 0 243.17 KB
#7153 ObjectExtractorSimpleBody net6.0 322ns 1.8ns 10.8ns 0 0 0 280 B
#7153 ObjectExtractorSimpleBody netcoreapp3.1 399ns 2.07ns 10.1ns 0 0 0 272 B
#7153 ObjectExtractorSimpleBody net472 304ns 0.0369ns 0.143ns 0.0436 0 0 281 B
#7153 ObjectExtractorMoreComplexBody net6.0 6.46μs 5.4ns 20.9ns 0 0 0 3.78 KB
#7153 ObjectExtractorMoreComplexBody netcoreapp3.1 7.71μs 29.3ns 113ns 0 0 0 3.69 KB
#7153 ObjectExtractorMoreComplexBody net472 6.81μs 3.26ns 11.8ns 0.58 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs‑net6.0 2.16 KB 2.14 KB -12 B -0.56%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 73.3μs 263ns 984ns 0 0 0 32.41 KB
master EncodeArgs netcoreapp3.1 95.7μs 36.2ns 130ns 0 0 0 32.4 KB
master EncodeArgs net472 107μs 17.6ns 65.8ns 4.82 0 0 32.51 KB
master EncodeLegacyArgs net6.0 143μs 122ns 472ns 0 0 0 2.16 KB
master EncodeLegacyArgs netcoreapp3.1 197μs 42.5ns 147ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 261μs 66.8ns 250ns 0 0 0 2.16 KB
#7153 EncodeArgs net6.0 78.7μs 32.8ns 123ns 0 0 0 32.4 KB
#7153 EncodeArgs netcoreapp3.1 96.7μs 225ns 870ns 0 0 0 32.4 KB
#7153 EncodeArgs net472 110μs 30.4ns 118ns 4.96 0 0 32.51 KB
#7153 EncodeLegacyArgs net6.0 145μs 275ns 1.07μs 0 0 0 2.14 KB
#7153 EncodeLegacyArgs netcoreapp3.1 197μs 107ns 399ns 0 0 0 2.14 KB
#7153 EncodeLegacyArgs net472 262μs 32.5ns 126ns 0 0 0 2.15 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 271μs 133ns 499ns 0 0 0 4.55 KB
master RunWafRealisticBenchmark netcoreapp3.1 294μs 264ns 989ns 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 307μs 38.1ns 147ns 0 0 0 4.66 KB
master RunWafRealisticBenchmarkWithAttack net6.0 181μs 80.1ns 300ns 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 198μs 104ns 404ns 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 207μs 45.3ns 169ns 0 0 0 2.28 KB
#7153 RunWafRealisticBenchmark net6.0 271μs 58.4ns 226ns 0 0 0 4.55 KB
#7153 RunWafRealisticBenchmark netcoreapp3.1 291μs 355ns 1.28μs 0 0 0 4.48 KB
#7153 RunWafRealisticBenchmark net472 312μs 43.7ns 164ns 0 0 0 4.65 KB
#7153 RunWafRealisticBenchmarkWithAttack net6.0 182μs 66.2ns 256ns 0 0 0 2.24 KB
#7153 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 194μs 216ns 837ns 0 0 0 2.22 KB
#7153 RunWafRealisticBenchmarkWithAttack net472 210μs 30.7ns 119ns 0 0 0 2.29 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 61.1μs 29ns 104ns 0 0 0 14.53 KB
master SendRequest netcoreapp3.1 69.9μs 118ns 440ns 0 0 0 17.42 KB
master SendRequest net472 0.0171ns 0.00147ns 0.00551ns 0 0 0 0 b
#7153 SendRequest net6.0 62μs 24ns 86.4ns 0 0 0 14.52 KB
#7153 SendRequest netcoreapp3.1 71.2μs 116ns 403ns 0 0 0 17.42 KB
#7153 SendRequest net472 0.0184ns 0.00201ns 0.00777ns 0 0 0 0 b
Benchmarks.Trace.CharSliceBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
#7153 OriginalCharSlice net6.0 1.91ms 658ns 2.37μs 0 0 0 640.01 KB
#7153 OriginalCharSlice netcoreapp3.1 2.09ms 952ns 3.43μs 0 0 0 640 KB
#7153 OriginalCharSlice net472 2.64ms 141ns 546ns 100 0 0 641.95 KB
#7153 OptimizedCharSlice net6.0 1.4ms 266ns 1.03μs 0 0 0 4 B
#7153 OptimizedCharSlice netcoreapp3.1 1.69ms 537ns 2.08μs 0 0 0 1 B
#7153 OptimizedCharSlice net472 2.02ms 207ns 802ns 0 0 0 73 B
#7153 OptimizedCharSliceWithPool net6.0 905μs 60.4ns 234ns 0 0 0 5 B
#7153 OptimizedCharSliceWithPool netcoreapp3.1 804μs 81.1ns 303ns 0 0 0 1 B
#7153 OptimizedCharSliceWithPool net472 1.14ms 52.2ns 202ns 0 0 0 47 B
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Slower ⚠️ Same allocations ✔️

Slower ⚠️ in #7153

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 1.174 645,405.00 757,701.67

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 645μs 512ns 1.98μs 0 0 0 41.73 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 631μs 1.67μs 6.67μs 0 0 0 41.89 KB
master WriteAndFlushEnrichedTraces net472 923μs 1.93μs 7.2μs 4.46 0 0 55.75 KB
#7153 WriteAndFlushEnrichedTraces net6.0 748μs 4.05μs 23.3μs 0 0 0 41.63 KB
#7153 WriteAndFlushEnrichedTraces netcoreapp3.1 674μs 905ns 3.26μs 0 0 0 41.86 KB
#7153 WriteAndFlushEnrichedTraces net472 900μs 3.66μs 14.2μs 4.46 0 0 55.93 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net6.0 1.03 KB 1.02 KB -8 B -0.78%
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑netcoreapp3.1 1.02 KB 1.02 KB -8 B -0.78%
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net472 995 B 987 B -8 B -0.80%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.99μs 10.2ns 46.8ns 0 0 0 1.03 KB
master ExecuteNonQuery netcoreapp3.1 2.53μs 3.43ns 13.3ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.7μs 2.41ns 9.34ns 0.147 0.0134 0 995 B
#7153 ExecuteNonQuery net6.0 1.92μs 9.26ns 33.4ns 0 0 0 1.02 KB
#7153 ExecuteNonQuery netcoreapp3.1 2.57μs 10.6ns 40.9ns 0 0 0 1.02 KB
#7153 ExecuteNonQuery net472 2.81μs 3.81ns 14.3ns 0.153 0.0139 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net472 1.11 KB 1.1 KB -8 B -0.72%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑netcoreapp3.1 1.09 KB 1.08 KB -8 B -0.74%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net472 1.05 KB 1.04 KB -8 B -0.76%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0 1.04 KB 1.03 KB -8 B -0.77%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑netcoreapp3.1 1.04 KB 1.03 KB -8 B -0.77%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net6.0 1.02 KB 1.01 KB -8 B -0.79%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.83μs 5.85ns 22.7ns 0 0 0 1.04 KB
master CallElasticsearch netcoreapp3.1 2.3μs 11.8ns 56.6ns 0 0 0 1.04 KB
master CallElasticsearch net472 3.55μs 3.05ns 11.8ns 0.159 0 0 1.05 KB
master CallElasticsearchAsync net6.0 1.81μs 3.89ns 14.5ns 0 0 0 1.02 KB
master CallElasticsearchAsync netcoreapp3.1 2.35μs 7.26ns 28.1ns 0 0 0 1.09 KB
master CallElasticsearchAsync net472 3.84μs 3.19ns 12.4ns 0.169 0 0 1.11 KB
#7153 CallElasticsearch net6.0 1.84μs 8.67ns 33.6ns 0 0 0 1.03 KB
#7153 CallElasticsearch netcoreapp3.1 2.27μs 10.9ns 46.3ns 0 0 0 1.03 KB
#7153 CallElasticsearch net472 3.52μs 7.35ns 28.4ns 0.16 0 0 1.04 KB
#7153 CallElasticsearchAsync net6.0 1.79μs 7.85ns 30.4ns 0 0 0 1.01 KB
#7153 CallElasticsearchAsync netcoreapp3.1 2.41μs 9.19ns 35.6ns 0 0 0 1.08 KB
#7153 CallElasticsearchAsync net472 3.7μs 5.35ns 20.7ns 0.165 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0 960 B 952 B -8 B -0.83%
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑netcoreapp3.1 960 B 952 B -8 B -0.83%
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net472 923 B 915 B -8 B -0.87%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.78μs 9.27ns 43.5ns 0 0 0 960 B
master ExecuteAsync netcoreapp3.1 2.29μs 7.45ns 25.8ns 0 0 0 960 B
master ExecuteAsync net472 2.58μs 1.78ns 6.88ns 0.143 0 0 923 B
#7153 ExecuteAsync net6.0 1.8μs 7.73ns 29.9ns 0 0 0 952 B
#7153 ExecuteAsync netcoreapp3.1 2.4μs 9.33ns 36.1ns 0 0 0 952 B
#7153 ExecuteAsync net472 2.46μs 0.443ns 1.53ns 0.136 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 7.1μs 10.4ns 39ns 0 0 0 2.37 KB
master SendAsync netcoreapp3.1 8.69μs 14.1ns 54.6ns 0 0 0 2.9 KB
master SendAsync net472 12.5μs 10.3ns 38.5ns 0.498 0 0 3.19 KB
#7153 SendAsync net6.0 6.9μs 5.2ns 19.5ns 0 0 0 2.36 KB
#7153 SendAsync netcoreapp3.1 8.73μs 22.3ns 86.2ns 0 0 0 2.9 KB
#7153 SendAsync net472 12.4μs 13ns 50.4ns 0.497 0 0 3.18 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7153

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 1.171 413,700.00 484,600.00

More allocations ⚠️ in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net472 57.34 KB 65.54 KB 8.19 KB 14.29%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 42.64 KB 43.83 KB 1.19 KB 2.80%

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 43.83 KB 43.19 KB -640 B -1.46%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net6.0 259.96 KB 250.67 KB -9.29 KB -3.57%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 274.93 KB 254.41 KB -20.52 KB -7.46%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 44.6μs 212ns 1.6μs 0 0 0 43.83 KB
master StringConcatBenchmark netcoreapp3.1 47.2μs 224ns 838ns 0 0 0 42.64 KB
master StringConcatBenchmark net472 56.8μs 259ns 968ns 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 458μs 1.08μs 3.89μs 0 0 0 259.96 KB
master StringConcatAspectBenchmark netcoreapp3.1 447μs 6.44μs 63.7μs 0 0 0 274.93 KB
master StringConcatAspectBenchmark net472 410μs 2.07μs 9.27μs 0 0 0 286.72 KB
#7153 StringConcatBenchmark net6.0 44.1μs 243ns 1.5μs 0 0 0 43.19 KB
#7153 StringConcatBenchmark netcoreapp3.1 50.3μs 318ns 3.06μs 0 0 0 43.83 KB
#7153 StringConcatBenchmark net472 57.2μs 228ns 855ns 0 0 0 65.54 KB
#7153 StringConcatAspectBenchmark net6.0 461μs 853ns 2.96μs 0 0 0 250.67 KB
#7153 StringConcatAspectBenchmark netcoreapp3.1 485μs 2.13μs 7.98μs 0 0 0 254.41 KB
#7153 StringConcatAspectBenchmark net472 411μs 2.09μs 10.2μs 0 0 0 286.72 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net6.0 1.76 KB 1.7 KB -56 B -3.18%
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1 1.76 KB 1.7 KB -56 B -3.18%
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net472 1.69 KB 1.64 KB -56 B -3.31%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.63μs 2.6ns 10.1ns 0 0 0 1.76 KB
master EnrichedLog netcoreapp3.1 3.44μs 4.38ns 17ns 0 0 0 1.76 KB
master EnrichedLog net472 4.05μs 4.83ns 18.7ns 0.265 0 0 1.69 KB
#7153 EnrichedLog net6.0 2.64μs 2.9ns 11.2ns 0 0 0 1.7 KB
#7153 EnrichedLog netcoreapp3.1 3.54μs 13ns 50.4ns 0 0 0 1.7 KB
#7153 EnrichedLog net472 4.1μs 3.31ns 12.4ns 0.246 0 0 1.64 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑net472 4.57 KB 4.52 KB -55 B -1.20%
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑net6.0 4.37 KB 4.31 KB -56 B -1.28%
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑netcoreapp3.1 4.37 KB 4.31 KB -56 B -1.28%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 123μs 145ns 563ns 0 0 0 4.37 KB
master EnrichedLog netcoreapp3.1 126μs 330ns 1.24μs 0 0 0 4.37 KB
master EnrichedLog net472 167μs 161ns 603ns 0 0 0 4.57 KB
#7153 EnrichedLog net6.0 122μs 61.2ns 237ns 0 0 0 4.31 KB
#7153 EnrichedLog netcoreapp3.1 127μs 230ns 830ns 0 0 0 4.31 KB
#7153 EnrichedLog net472 167μs 114ns 426ns 0 0 0 4.52 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑net6.0 2.32 KB 2.26 KB -56 B -2.41%
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑netcoreapp3.1 2.32 KB 2.26 KB -56 B -2.41%
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑net472 2.14 KB 2.08 KB -56 B -2.62%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.94μs 19.3ns 74.6ns 0 0 0 2.32 KB
master EnrichedLog netcoreapp3.1 6.77μs 22.2ns 79.9ns 0 0 0 2.32 KB
master EnrichedLog net472 7.45μs 8.04ns 31.1ns 0.335 0 0 2.14 KB
#7153 EnrichedLog net6.0 5.07μs 16.9ns 65.5ns 0 0 0 2.26 KB
#7153 EnrichedLog netcoreapp3.1 6.85μs 15.5ns 60.1ns 0 0 0 2.26 KB
#7153 EnrichedLog net472 7.39μs 8.2ns 31.8ns 0.295 0 0 2.08 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.RedisBenchmark.SendReceive‑net472 1.21 KB 1.2 KB -8 B -0.66%
Benchmarks.Trace.RedisBenchmark.SendReceive‑net6.0 1.21 KB 1.2 KB -8 B -0.66%
Benchmarks.Trace.RedisBenchmark.SendReceive‑netcoreapp3.1 1.21 KB 1.2 KB -8 B -0.66%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2.04μs 10.5ns 49.4ns 0 0 0 1.21 KB
master SendReceive netcoreapp3.1 2.53μs 11.8ns 47.4ns 0 0 0 1.21 KB
master SendReceive net472 3.28μs 2.71ns 10.5ns 0.178 0 0 1.21 KB
#7153 SendReceive net6.0 1.97μs 3.29ns 12.7ns 0 0 0 1.2 KB
#7153 SendReceive netcoreapp3.1 2.61μs 7.19ns 27.8ns 0 0 0 1.2 KB
#7153 SendReceive net472 3.23μs 2.69ns 10.4ns 0.178 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑net472 2.08 KB 2.03 KB -56 B -2.69%
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑netcoreapp3.1 1.69 KB 1.63 KB -56 B -3.32%
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑net6.0 1.64 KB 1.58 KB -56 B -3.41%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.18μs 0.92ns 3.44ns 0 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 5.64μs 23ns 89.3ns 0 0 0 1.69 KB
master EnrichedLog net472 6.67μs 7.28ns 27.3ns 0.298 0 0 2.08 KB
#7153 EnrichedLog net6.0 4.18μs 13ns 50.4ns 0 0 0 1.58 KB
#7153 EnrichedLog netcoreapp3.1 5.45μs 8.93ns 34.6ns 0 0 0 1.63 KB
#7153 EnrichedLog net472 6.67μs 7.59ns 29.4ns 0.3 0 0 2.03 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 704 B 696 B -8 B -1.14%
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 704 B 696 B -8 B -1.14%
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 666 B 658 B -8 B -1.20%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472 586 B 578 B -8 B -1.37%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 584 B 576 B -8 B -1.37%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 584 B 576 B -8 B -1.37%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 751ns 3.97ns 22.4ns 0 0 0 584 B
master StartFinishSpan netcoreapp3.1 953ns 4.42ns 17.7ns 0 0 0 584 B
master StartFinishSpan net472 914ns 0.809ns 3.13ns 0.0912 0 0 586 B
master StartFinishScope net6.0 918ns 0.479ns 1.73ns 0 0 0 704 B
master StartFinishScope netcoreapp3.1 1.15μs 6.24ns 34.2ns 0 0 0 704 B
master StartFinishScope net472 1.09μs 0.174ns 0.652ns 0.104 0 0 666 B
#7153 StartFinishSpan net6.0 740ns 4.13ns 24.4ns 0 0 0 576 B
#7153 StartFinishSpan netcoreapp3.1 956ns 4.61ns 17.9ns 0 0 0 576 B
#7153 StartFinishSpan net472 904ns 0.252ns 0.977ns 0.091 0 0 578 B
#7153 StartFinishScope net6.0 906ns 4.38ns 19.1ns 0 0 0 696 B
#7153 StartFinishScope netcoreapp3.1 1.16μs 6.09ns 30.4ns 0 0 0 696 B
#7153 StartFinishScope net472 1.1μs 0.533ns 1.99ns 0.105 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7153

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 704 B 696 B -8 B -1.14%
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1 704 B 696 B -8 B -1.14%
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472 666 B 658 B -8 B -1.20%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.02μs 5.36ns 26.8ns 0 0 0 704 B
master RunOnMethodBegin netcoreapp3.1 1.38μs 2.28ns 8.82ns 0 0 0 704 B
master RunOnMethodBegin net472 1.36μs 0.141ns 0.544ns 0.102 0 0 666 B
#7153 RunOnMethodBegin net6.0 1.06μs 5ns 20ns 0 0 0 696 B
#7153 RunOnMethodBegin netcoreapp3.1 1.44μs 3.92ns 14.7ns 0 0 0 696 B
#7153 RunOnMethodBegin net472 1.42μs 0.694ns 2.69ns 0.0991 0 0 658 B

@andrewlock andrewlock mentioned this pull request Jul 2, 2025
Merged
andrewlock added a commit that referenced this pull request Jul 4, 2025
@andrewlock @e-n-0
Don't load the profiler on arm64 (#7169)
df69689 
## Summary of changes

- Don't try to load the profiler on Linux arm64 at all
- Cherry pick #7153 to see if it resolves the issues

## Reason for change

#7153 ran into what we think is glibc `dlclose` issue (i.e. #7125).
Given the profiler always bails out on arm64 as it's not yet supported,
this is the easiest way to ensure that we don't load (and therefore,
more importantly, we don't _unload_ the profiler on arm64).

## Implementation details

Comment out the profiler from the loader.conf for arm64. That ensures we
don't _try_ to load it on arm64, while making it easy to reenable for
testing etc.

## Test coverage

Cherry picked the #7153 smoke tests, to confirm it resolves the issues

## Other details

Supersedes #7153 given that it enables the tests to verify the fix

---------

Co-authored-by: Flavien Darche <11708575+e-n-0@users.noreply.github.com>
andrewlock added a commit that referenced this pull request Jul 7, 2025
@andrewlock @e-n-0
Don't load the profiler on arm64 (#7169)
cea9aa1 
## Summary of changes

- Don't try to load the profiler on Linux arm64 at all
- Cherry pick #7153 to see if it resolves the issues

## Reason for change

#7153 ran into what we think is glibc `dlclose` issue (i.e. #7125).
Given the profiler always bails out on arm64 as it's not yet supported,
this is the easiest way to ensure that we don't load (and therefore,
more importantly, we don't _unload_ the profiler on arm64).

## Implementation details

Comment out the profiler from the loader.conf for arm64. That ensures we
don't _try_ to load it on arm64, while making it easy to reenable for
testing etc.

## Test coverage

Cherry picked the #7153 smoke tests, to confirm it resolves the issues

## Other details

Supersedes #7153 given that it enables the tests to verify the fix

---------

Co-authored-by: Flavien Darche <11708575+e-n-0@users.noreply.github.com>
@github-actions github-actions Bot added area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) area:docs area:tests unit tests, integration tests area:opentracing area:installers area:vendors Code from other vendors area:shared-components area:profiler Issues related to the continous-profiler area:debugger area:integrations labels Jul 10, 2025
@e-n-0 e-n-0 closed this Jul 10, 2025
@e-n-0 e-n-0 deleted the flavien/add-ssi-appsec-systemtests branch July 10, 2025 11:30
chojomok pushed a commit that referenced this pull request Jul 15, 2025
@andrewlock @e-n-0 @chojomok
Don't load the profiler on arm64 (#7169)
cd0a978 
## Summary of changes

- Don't try to load the profiler on Linux arm64 at all
- Cherry pick #7153 to see if it resolves the issues

## Reason for change

#7153 ran into what we think is glibc `dlclose` issue (i.e. #7125).
Given the profiler always bails out on arm64 as it's not yet supported,
this is the easiest way to ensure that we don't load (and therefore,
more importantly, we don't _unload_ the profiler on arm64).

## Implementation details

Comment out the profiler from the loader.conf for arm64. That ensures we
don't _try_ to load it on arm64, while making it easy to reenable for
testing etc.

## Test coverage

Cherry picked the #7153 smoke tests, to confirm it resolves the issues

## Other details

Supersedes #7153 given that it enables the tests to verify the fix

---------

Co-authored-by: Flavien Darche <11708575+e-n-0@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@daniel-romano-DD daniel-romano-DD daniel-romano-DD approved these changes

Assignees

No one assigned

Labels

area:builds project files, build scripts, pipelines, versioning, releases, packages area:debugger area:docs area:installers area:integrations area:opentracing area:profiler Issues related to the continous-profiler area:shared-components area:tests unit tests, integration tests area:tracer The core tracer library (Datadog.Trace, does not include OpenTracing, native code, or integrations) area:vendors Code from other vendors

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@e-n-0 @daniel-romano-DD