Skip to content

Add snapshot tests to document the symbols we depend on in alpine and native loader#7224

Merged
andrewlock merged 6 commits into
masterfrom
andrew/snapshot-alpine-symbols
Jul 16, 2025
Merged

Add snapshot tests to document the symbols we depend on in alpine and native loader#7224
andrewlock merged 6 commits into
masterfrom
andrew/snapshot-alpine-symbols

Conversation

@andrewlock

@andrewlock andrewlock commented Jul 15, 2025

Copy link
Copy Markdown
Member

Summary of changes

Adds more "native symbol" snapshot tests for the symbols we depend on in our native libraries

Background

As part of the initial work on "universal binaries" we had to fix an issue where we accidentally started depending on a new symbol, which was not always guaranteed to be present and could break customers. As a consequence, we subsequently set-up snapshot tests that use nm to dump the symbols required by our "native wrapper library" AKA Datadog.Linux.ApiWrapper.x64.so. We also added verification checks that we were building against the correct glibc version so we didn't accidentally change our supported distros.

Reason for change

.NET 10 has bumped the minimum supported version of Alpine from 3.13 (.NET 9<) to 3.17 (.NET 10+). My previous understanding was that musl is essentially unversioned, so they're essentially also forward compatible.

Unfortunately, that is apparently not true. Attempting to run .NET 10 on Alpine 3.13 does not work, and it specifically fails with a symbol not found error:

Failed to load /usr/share/dotnet/host/fxr/10.0.0-preview.5.25277.114/libhostfxr.so, error: 
Error relocating /usr/share/dotnet/host/fxr/10.0.0-preview.5.25277.114/libhostfxr.so: 
_ZSt28__throw_bad_array_new_lengthv: symbol not found

So this leaves us with a dilemma; we need to update to alpine 3.17 so we can build and test with .NET 10, but doing so might end up introducing breaking dependencies on new symbols.

This PR is about documenting more fully what we actually depend on. Currently we have

  • Symbol tests for Datadog.Linux.ApiWrapper.x64.so on linux-x64 and linux-arm64
  • Maximum GLibc usages verification for linux-x64 and linux-arm64 for
    • Datadog.Tracer.Native.so
    • Datadog.Profiler.Native.so
    • libdatadog_profiling.so

This PR adds additional tests for the missing libraries and targets

  • Symbol tests for Datadog.Trace.ClrProfiler.Native.so on linux-x64 and linux-arm64
  • Symbol tests for linux-musl-x64 and linux-musl-arm64 for
    • Datadog.Tracer.Native.so
    • Datadog.Profiler.Native.so
    • libdatadog_profiling.so

Note that Datadog.Linux.ApiWrapper.x64.so and Datadog.Trace.ClrProfiler.Native.so are "universal" binaries, so the same binary is used for both Glibc and musl.

Implementation details

  • Extracted the existing symbol test to a reusable function (1st commit, feel free to skip).
  • Call the reusable function in the missing scenarios.
  • Run the tests and save the snapshots.

Test coverage

More now!

Note that I'm not really sure how much value there is in documenting the libdatadog_profiling.so symbols. It's not something we control in this repository and it means needing to update the snapshot when something changes. I've included it now to be safe, but we can always address later.

Other details

Investigating the alpine:3.17 changes is out of scope for this PR, I'll investigate that separately.

The one notably not-documented native library here is ddwaf. Should we document the symbols in that one too?

@andrewlock andrewlock added area:builds project files, build scripts, pipelines, versioning, releases, packages area:tests unit tests, integration tests area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) labels Jul 15, 2025
@andrewlock andrewlock force-pushed the andrew/snapshot-alpine-symbols branch from 36a8151 to 0d7aa62 Compare July 15, 2025 10:54
$"If the new symbols are safe to add, update the snapshot file at {verifiedPath} with the " +
"new values");
var snapshotName = $"native-wrapper-symbols-{UnixArchitectureIdentifier}";
CompareNativeSymbolsSnapshot(libraryPath, snapshotName);

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This change is purely a copy-paste move to a helper function in Build.Shared.Steps

// For information, those symbols comes from libgcc and are exposed for compatibility
var libdatadogAllowedSymbols = IsArm64 && IsAlpine ? new[] { "__register_frame_info@GLIBC_2.0", "__deregister_frame_info@GLIBC_2.0" } : null;
var filesAndVersion = new List<Tuple<string, Version, IEnumerable<string>>>
var filesAndVersion = new []

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

tiny refactor to use value tuples and inference for brevity

.Max();
}

void CompareNativeSymbolsSnapshot(AbsolutePath libraryPath, string snapshotNamePrefix)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is where the code was copy-pasted to, from Build.Profiler.Steps.cs so doesn't really need reviewing (unless you really want to!)

return;
}

// Print the expected values, so it's easier to copy-paste them into the snapshot file as required

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was the only change, I tweaked the logging

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These snapshots are what we depend on today. We don't necessarily care too much what they contain at this point (unless we are investigating escalations later). For our purposes, we're mostly interested in changes, particularly additions of new symbols.

clock_gettime
close
closedir
ddog_crasht_CrashInfo_upload_to_endpoint

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I considered removing all these ddog_ symbols, as they're all provided by libdatadog, but that would have been a bit of a pain so, meh?

@dd-trace-dotnet-ci-bot

dd-trace-dotnet-ci-bot Bot commented Jul 15, 2025

Copy link
Copy Markdown

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7224) - mean (72ms)  : 71, 73
     .   : milestone, 72,
    master - mean (72ms)  : 71, 73
     .   : milestone, 72,

    section Baseline
    This PR (7224) - mean (68ms)  : 66, 70
     .   : milestone, 68,
    master - mean (68ms)  : 66, 70
     .   : milestone, 68,

    section CallTarget+Inlining+NGEN
    This PR (7224) - mean (1,022ms)  : 1000, 1044
     .   : milestone, 1022,
    master - mean (1,021ms)  : 1000, 1043
     .   : milestone, 1021,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7224) - mean (105ms)  : 104, 107
     .   : milestone, 105,
    master - mean (105ms)  : 104, 107
     .   : milestone, 105,

    section Baseline
    This PR (7224) - mean (105ms)  : 103, 107
     .   : milestone, 105,
    master - mean (105ms)  : 102, 107
     .   : milestone, 105,

    section CallTarget+Inlining+NGEN
    This PR (7224) - mean (705ms)  : 687, 722
     .   : milestone, 705,
    master - mean (707ms)  : 693, 722
     .   : milestone, 707,

Loading
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7224) - mean (93ms)  : 92, 94
     .   : milestone, 93,
    master - mean (93ms)  : 92, 94
     .   : milestone, 93,

    section Baseline
    This PR (7224) - mean (92ms)  : 90, 94
     .   : milestone, 92,
    master - mean (92ms)  : 90, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (7224) - mean (663ms)  : 639, 687
     .   : milestone, 663,
    master - mean (667ms)  : 632, 703
     .   : milestone, 667,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7224) - mean (199ms)  : 195, 204
     .   : milestone, 199,
    master - mean (196ms)  : 191, 202
     .   : milestone, 196,

    section Baseline
    This PR (7224) - mean (195ms)  : 189, 202
     .   : milestone, 195,
    master - mean (193ms)  : 187, 198
     .   : milestone, 193,

    section CallTarget+Inlining+NGEN
    This PR (7224) - mean (1,150ms)  : 1125, 1176
     .   : milestone, 1150,
    master - mean (1,143ms)  : 1117, 1170
     .   : milestone, 1143,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7224) - mean (299ms)  : crit, 293, 304
     .   : crit, milestone, 299,
    master - mean (277ms)  : 271, 283
     .   : milestone, 277,

    section Baseline
    This PR (7224) - mean (288ms)  : 274, 302
     .   : milestone, 288,
    master - mean (278ms)  : 271, 286
     .   : milestone, 278,

    section CallTarget+Inlining+NGEN
    This PR (7224) - mean (918ms)  : 889, 948
     .   : milestone, 918,
    master - mean (909ms)  : 883, 934
     .   : milestone, 909,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7224) - mean (275ms)  : 268, 282
     .   : milestone, 275,
    master - mean (271ms)  : 265, 277
     .   : milestone, 271,

    section Baseline
    This PR (7224) - mean (275ms)  : 269, 281
     .   : milestone, 275,
    master - mean (272ms)  : 266, 278
     .   : milestone, 272,

    section CallTarget+Inlining+NGEN
    This PR (7224) - mean (900ms)  : 866, 934
     .   : milestone, 900,
    master - mean (899ms)  : 867, 931
     .   : milestone, 899,

Loading
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7224) - mean (275ms)  : 270, 280
     .   : milestone, 275,
    master - mean (274ms)  : 262, 286
     .   : milestone, 274,

    section Baseline
    This PR (7224) - mean (275ms)  : 268, 283
     .   : milestone, 275,
    master - mean (272ms)  : 264, 279
     .   : milestone, 272,

    section CallTarget+Inlining+NGEN
    This PR (7224) - mean (812ms)  : 786, 838
     .   : milestone, 812,
    master - mean (810ms)  : 786, 835
     .   : milestone, 810,

Loading

@pr-commenter

pr-commenter Bot commented Jul 15, 2025

Copy link
Copy Markdown

Benchmarks

Benchmarks Report for benchmark platform 🐌

Benchmarks for #7224 compared to master:

  • 3 benchmarks are slower, with geometric mean 1.192
  • 43 benchmarks have fewer allocations
  • 5 benchmarks have more allocations

The following thresholds were used for comparing the benchmark speeds:

  • Mann–Whitney U test with statistical test for significance of 5%
  • Only results indicating a difference greater than 10% and 0.3 ns are considered.

Allocation changes below 0.5% are ignored.

Benchmark details

Benchmarks.Trace.ActivityBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net472 6.09 KB 6.06 KB -32 B -0.53%
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑netcoreapp3.1 5.75 KB 5.69 KB -65 B -1.13%
Benchmarks.Trace.ActivityBenchmark.StartStopWithChild‑net6.0 5.58 KB 5.5 KB -81 B -1.45%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartStopWithChild net6.0 11.2μs 59.7ns 322ns 0 0 0 5.58 KB
master StartStopWithChild netcoreapp3.1 14.1μs 67.5ns 286ns 0 0 0 5.75 KB
master StartStopWithChild net472 22.1μs 119ns 651ns 1.03 0.411 0.103 6.09 KB
#7224 StartStopWithChild net6.0 10.5μs 58.1ns 349ns 0 0 0 5.5 KB
#7224 StartStopWithChild netcoreapp3.1 13.7μs 68.5ns 306ns 0 0 0 5.69 KB
#7224 StartStopWithChild net472 23.3μs 125ns 697ns 0.921 0.23 0 6.06 KB
Benchmarks.Trace.AgentWriterBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.AgentWriterBenchmark.WriteAndFlushEnrichedTraces‑net472 3.33 KB 3.31 KB -23 B -0.69%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 927μs 26.6ns 103ns 0 0 0 2.71 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 1.04ms 404ns 1.56μs 0 0 0 2.7 KB
master WriteAndFlushEnrichedTraces net472 1.25ms 357ns 1.38μs 0 0 0 3.33 KB
#7224 WriteAndFlushEnrichedTraces net6.0 924μs 518ns 1.94μs 0 0 0 2.7 KB
#7224 WriteAndFlushEnrichedTraces netcoreapp3.1 1.04ms 119ns 444ns 0 0 0 2.7 KB
#7224 WriteAndFlushEnrichedTraces net472 1.2ms 45.4ns 170ns 0 0 0 3.31 KB
Benchmarks.Trace.Asm.AppSecBodyBenchmark - Same speed ✔️ More allocations ⚠️

More allocations ⚠️ in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleSimpleBody‑net472 236.35 KB 239.66 KB 3.3 KB 1.40%
Benchmarks.Trace.Asm.AppSecBodyBenchmark.AllCycleMoreComplexBody‑net472 239.87 KB 243.17 KB 3.3 KB 1.38%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master AllCycleSimpleBody net6.0 331μs 1.76μs 8.42μs 0 0 0 197.06 KB
master AllCycleSimpleBody netcoreapp3.1 510μs 1.45μs 5.61μs 0 0 0 204.77 KB
master AllCycleSimpleBody net472 436μs 119ns 460ns 36.6 2.16 0 236.35 KB
master AllCycleMoreComplexBody net6.0 338μs 1.76μs 8.8μs 0 0 0 200.56 KB
master AllCycleMoreComplexBody netcoreapp3.1 495μs 987ns 3.56μs 0 0 0 208.18 KB
master AllCycleMoreComplexBody net472 446μs 106ns 412ns 36.6 2.16 0 239.87 KB
master ObjectExtractorSimpleBody net6.0 311ns 1.77ns 12.3ns 0 0 0 280 B
master ObjectExtractorSimpleBody netcoreapp3.1 409ns 1.93ns 8.2ns 0 0 0 272 B
master ObjectExtractorSimpleBody net472 303ns 0.175ns 0.676ns 0.0442 0 0 281 B
master ObjectExtractorMoreComplexBody net6.0 6.52μs 29.5ns 110ns 0 0 0 3.78 KB
master ObjectExtractorMoreComplexBody netcoreapp3.1 7.76μs 36.2ns 140ns 0 0 0 3.69 KB
master ObjectExtractorMoreComplexBody net472 6.66μs 0.89ns 3.33ns 0.599 0 0 3.8 KB
#7224 AllCycleSimpleBody net6.0 331μs 1.52μs 6.09μs 0 0 0 197.61 KB
#7224 AllCycleSimpleBody netcoreapp3.1 506μs 562ns 2.18μs 0 0 0 205.37 KB
#7224 AllCycleSimpleBody net472 442μs 91.3ns 354ns 36.6 2.16 0 239.66 KB
#7224 AllCycleMoreComplexBody net6.0 347μs 1.5μs 5.83μs 0 0 0 201.11 KB
#7224 AllCycleMoreComplexBody netcoreapp3.1 499μs 1.88μs 7.26μs 0 0 0 208.78 KB
#7224 AllCycleMoreComplexBody net472 449μs 169ns 653ns 37.9 2.23 0 243.17 KB
#7224 ObjectExtractorSimpleBody net6.0 322ns 0.0908ns 0.351ns 0 0 0 280 B
#7224 ObjectExtractorSimpleBody netcoreapp3.1 407ns 2.2ns 12ns 0 0 0 272 B
#7224 ObjectExtractorSimpleBody net472 301ns 0.0706ns 0.273ns 0.044 0 0 281 B
#7224 ObjectExtractorMoreComplexBody net6.0 6.26μs 30.4ns 133ns 0 0 0 3.78 KB
#7224 ObjectExtractorMoreComplexBody netcoreapp3.1 7.68μs 39.1ns 161ns 0 0 0 3.69 KB
#7224 ObjectExtractorMoreComplexBody net472 6.8μs 2.47ns 9.56ns 0.578 0 0 3.8 KB
Benchmarks.Trace.Asm.AppSecEncoderBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Asm.AppSecEncoderBenchmark.EncodeLegacyArgs‑net6.0 2.16 KB 2.14 KB -12 B -0.56%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EncodeArgs net6.0 73.3μs 263ns 984ns 0 0 0 32.41 KB
master EncodeArgs netcoreapp3.1 95.7μs 36.2ns 130ns 0 0 0 32.4 KB
master EncodeArgs net472 107μs 17.6ns 65.8ns 4.82 0 0 32.51 KB
master EncodeLegacyArgs net6.0 143μs 122ns 472ns 0 0 0 2.16 KB
master EncodeLegacyArgs netcoreapp3.1 197μs 42.5ns 147ns 0 0 0 2.14 KB
master EncodeLegacyArgs net472 261μs 66.8ns 250ns 0 0 0 2.16 KB
#7224 EncodeArgs net6.0 77.1μs 182ns 703ns 0 0 0 32.4 KB
#7224 EncodeArgs netcoreapp3.1 98.1μs 308ns 1.19μs 0 0 0 32.4 KB
#7224 EncodeArgs net472 109μs 21.1ns 81.9ns 4.96 0 0 32.51 KB
#7224 EncodeLegacyArgs net6.0 145μs 51ns 197ns 0 0 0 2.14 KB
#7224 EncodeLegacyArgs netcoreapp3.1 196μs 101ns 377ns 0 0 0 2.14 KB
#7224 EncodeLegacyArgs net472 262μs 29.3ns 113ns 0 0 0 2.15 KB
Benchmarks.Trace.Asm.AppSecWafBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunWafRealisticBenchmark net6.0 271μs 133ns 499ns 0 0 0 4.55 KB
master RunWafRealisticBenchmark netcoreapp3.1 294μs 264ns 989ns 0 0 0 4.48 KB
master RunWafRealisticBenchmark net472 307μs 38.1ns 147ns 0 0 0 4.66 KB
master RunWafRealisticBenchmarkWithAttack net6.0 181μs 80.1ns 300ns 0 0 0 2.24 KB
master RunWafRealisticBenchmarkWithAttack netcoreapp3.1 198μs 104ns 404ns 0 0 0 2.22 KB
master RunWafRealisticBenchmarkWithAttack net472 207μs 45.3ns 169ns 0 0 0 2.28 KB
#7224 RunWafRealisticBenchmark net6.0 274μs 66ns 256ns 0 0 0 4.55 KB
#7224 RunWafRealisticBenchmark netcoreapp3.1 297μs 896ns 3.47μs 0 0 0 4.48 KB
#7224 RunWafRealisticBenchmark net472 313μs 32.6ns 126ns 0 0 0 4.65 KB
#7224 RunWafRealisticBenchmarkWithAttack net6.0 182μs 51.4ns 185ns 0 0 0 2.24 KB
#7224 RunWafRealisticBenchmarkWithAttack netcoreapp3.1 195μs 63ns 227ns 0 0 0 2.22 KB
#7224 RunWafRealisticBenchmarkWithAttack net472 207μs 33.2ns 128ns 0 0 0 2.29 KB
Benchmarks.Trace.AspNetCoreBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendRequest net6.0 61.1μs 29ns 104ns 0 0 0 14.53 KB
master SendRequest netcoreapp3.1 69.9μs 118ns 440ns 0 0 0 17.42 KB
master SendRequest net472 0.0171ns 0.00147ns 0.00551ns 0 0 0 0 b
#7224 SendRequest net6.0 62.2μs 210ns 787ns 0 0 0 14.52 KB
#7224 SendRequest netcoreapp3.1 71.2μs 119ns 430ns 0 0 0 17.42 KB
#7224 SendRequest net472 0.00987ns 0.00192ns 0.00744ns 0 0 0 0 b
Benchmarks.Trace.CharSliceBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
#7224 OriginalCharSlice net6.0 1.88ms 1.01μs 3.77μs 0 0 0 640.01 KB
#7224 OriginalCharSlice netcoreapp3.1 2.06ms 9.2μs 34.4μs 0 0 0 640 KB
#7224 OriginalCharSlice net472 2.66ms 339ns 1.31μs 100 0 0 641.95 KB
#7224 OptimizedCharSlice net6.0 1.45ms 152ns 589ns 0 0 0 2 B
#7224 OptimizedCharSlice netcoreapp3.1 1.66ms 134ns 517ns 0 0 0 1 B
#7224 OptimizedCharSlice net472 1.98ms 3.92μs 15.2μs 0 0 0 0 b
#7224 OptimizedCharSliceWithPool net6.0 807μs 93.7ns 351ns 0 0 0 3 B
#7224 OptimizedCharSliceWithPool netcoreapp3.1 797μs 95.3ns 369ns 0 0 0 0 b
#7224 OptimizedCharSliceWithPool net472 1.13ms 94.1ns 364ns 0 0 0 0 b
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7224

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑net6.0 1.122 645,405.00 724,367.43

More allocations ⚠️ in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.CIVisibilityProtocolWriterBenchmark.WriteAndFlushEnrichedTraces‑netcoreapp3.1 41.89 KB 42.22 KB 330 B 0.79%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master WriteAndFlushEnrichedTraces net6.0 645μs 512ns 1.98μs 0 0 0 41.73 KB
master WriteAndFlushEnrichedTraces netcoreapp3.1 631μs 1.67μs 6.67μs 0 0 0 41.89 KB
master WriteAndFlushEnrichedTraces net472 923μs 1.93μs 7.2μs 4.46 0 0 55.75 KB
#7224 WriteAndFlushEnrichedTraces net6.0 725μs 1.34μs 5.18μs 0 0 0 41.66 KB
#7224 WriteAndFlushEnrichedTraces netcoreapp3.1 669μs 2.41μs 9.01μs 0 0 0 42.22 KB
#7224 WriteAndFlushEnrichedTraces net472 1ms 3.4μs 12.7μs 4.81 0 0 55.79 KB
Benchmarks.Trace.DbCommandBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net6.0 1.03 KB 1.02 KB -8 B -0.78%
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑netcoreapp3.1 1.02 KB 1.02 KB -8 B -0.78%
Benchmarks.Trace.DbCommandBenchmark.ExecuteNonQuery‑net472 995 B 987 B -8 B -0.80%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteNonQuery net6.0 1.99μs 10.2ns 46.8ns 0 0 0 1.03 KB
master ExecuteNonQuery netcoreapp3.1 2.53μs 3.43ns 13.3ns 0 0 0 1.02 KB
master ExecuteNonQuery net472 2.7μs 2.41ns 9.34ns 0.147 0.0134 0 995 B
#7224 ExecuteNonQuery net6.0 1.93μs 9.43ns 41.1ns 0 0 0 1.02 KB
#7224 ExecuteNonQuery netcoreapp3.1 2.55μs 6.75ns 25.3ns 0 0 0 1.02 KB
#7224 ExecuteNonQuery net472 2.75μs 2.59ns 9.7ns 0.151 0.0137 0 987 B
Benchmarks.Trace.ElasticsearchBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net472 1.11 KB 1.1 KB -8 B -0.72%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑netcoreapp3.1 1.09 KB 1.08 KB -8 B -0.74%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net472 1.05 KB 1.04 KB -8 B -0.76%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑net6.0 1.04 KB 1.03 KB -8 B -0.77%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearch‑netcoreapp3.1 1.04 KB 1.03 KB -8 B -0.77%
Benchmarks.Trace.ElasticsearchBenchmark.CallElasticsearchAsync‑net6.0 1.02 KB 1.01 KB -8 B -0.79%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master CallElasticsearch net6.0 1.83μs 5.85ns 22.7ns 0 0 0 1.04 KB
master CallElasticsearch netcoreapp3.1 2.3μs 11.8ns 56.6ns 0 0 0 1.04 KB
master CallElasticsearch net472 3.55μs 3.05ns 11.8ns 0.159 0 0 1.05 KB
master CallElasticsearchAsync net6.0 1.81μs 3.89ns 14.5ns 0 0 0 1.02 KB
master CallElasticsearchAsync netcoreapp3.1 2.35μs 7.26ns 28.1ns 0 0 0 1.09 KB
master CallElasticsearchAsync net472 3.84μs 3.19ns 12.4ns 0.169 0 0 1.11 KB
#7224 CallElasticsearch net6.0 1.79μs 9.19ns 41.1ns 0 0 0 1.03 KB
#7224 CallElasticsearch netcoreapp3.1 2.35μs 7.61ns 27.4ns 0 0 0 1.03 KB
#7224 CallElasticsearch net472 3.7μs 3.14ns 12.2ns 0.148 0 0 1.04 KB
#7224 CallElasticsearchAsync net6.0 1.86μs 8.2ns 30.7ns 0 0 0 1.01 KB
#7224 CallElasticsearchAsync netcoreapp3.1 2.48μs 4.89ns 18.9ns 0 0 0 1.08 KB
#7224 CallElasticsearchAsync net472 3.76μs 3.35ns 13ns 0.168 0 0 1.1 KB
Benchmarks.Trace.GraphQLBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net6.0 960 B 952 B -8 B -0.83%
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑netcoreapp3.1 960 B 952 B -8 B -0.83%
Benchmarks.Trace.GraphQLBenchmark.ExecuteAsync‑net472 923 B 915 B -8 B -0.87%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master ExecuteAsync net6.0 1.78μs 9.27ns 43.5ns 0 0 0 960 B
master ExecuteAsync netcoreapp3.1 2.29μs 7.45ns 25.8ns 0 0 0 960 B
master ExecuteAsync net472 2.58μs 1.78ns 6.88ns 0.143 0 0 923 B
#7224 ExecuteAsync net6.0 1.99μs 3.3ns 12.4ns 0 0 0 952 B
#7224 ExecuteAsync netcoreapp3.1 2.4μs 6.79ns 26.3ns 0 0 0 952 B
#7224 ExecuteAsync net472 2.57μs 1.82ns 6.82ns 0.141 0 0 915 B
Benchmarks.Trace.HttpClientBenchmark - Same speed ✔️ Same allocations ✔️

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendAsync net6.0 7.1μs 10.4ns 39ns 0 0 0 2.37 KB
master SendAsync netcoreapp3.1 8.69μs 14.1ns 54.6ns 0 0 0 2.9 KB
master SendAsync net472 12.5μs 10.3ns 38.5ns 0.498 0 0 3.19 KB
#7224 SendAsync net6.0 7.07μs 11.3ns 40.8ns 0 0 0 2.36 KB
#7224 SendAsync netcoreapp3.1 8.77μs 27.2ns 105ns 0 0 0 2.9 KB
#7224 SendAsync net472 12.1μs 15.9ns 61.4ns 0.481 0 0 3.18 KB
Benchmarks.Trace.Iast.StringAspectsBenchmark - Slower ⚠️ More allocations ⚠️

Slower ⚠️ in #7224

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 1.329 413,700.00 549,700.00

More allocations ⚠️ in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑netcoreapp3.1 42.64 KB 42.94 KB 304 B 0.71%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑netcoreapp3.1 274.93 KB 276.5 KB 1.58 KB 0.57%

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatBenchmark‑net6.0 43.83 KB 43.48 KB -352 B -0.80%
Benchmarks.Trace.Iast.StringAspectsBenchmark.StringConcatAspectBenchmark‑net472 286.72 KB 278.53 KB -8.19 KB -2.86%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StringConcatBenchmark net6.0 44.6μs 212ns 1.6μs 0 0 0 43.83 KB
master StringConcatBenchmark netcoreapp3.1 47.2μs 224ns 838ns 0 0 0 42.64 KB
master StringConcatBenchmark net472 56.8μs 259ns 968ns 0 0 0 57.34 KB
master StringConcatAspectBenchmark net6.0 458μs 1.08μs 3.89μs 0 0 0 259.96 KB
master StringConcatAspectBenchmark netcoreapp3.1 447μs 6.44μs 63.7μs 0 0 0 274.93 KB
master StringConcatAspectBenchmark net472 410μs 2.07μs 9.27μs 0 0 0 286.72 KB
#7224 StringConcatBenchmark net6.0 48.8μs 260ns 1.27μs 0 0 0 43.48 KB
#7224 StringConcatBenchmark netcoreapp3.1 49.3μs 256ns 1.47μs 0 0 0 42.94 KB
#7224 StringConcatBenchmark net472 57.3μs 235ns 911ns 0 0 0 57.34 KB
#7224 StringConcatAspectBenchmark net6.0 469μs 1.42μs 4.91μs 0 0 0 260.14 KB
#7224 StringConcatAspectBenchmark netcoreapp3.1 549μs 1.43μs 4.96μs 0 0 0 276.5 KB
#7224 StringConcatAspectBenchmark net472 405μs 1.94μs 9.7μs 0 0 0 278.53 KB
Benchmarks.Trace.ILoggerBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net6.0 1.76 KB 1.7 KB -56 B -3.18%
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑netcoreapp3.1 1.76 KB 1.7 KB -56 B -3.18%
Benchmarks.Trace.ILoggerBenchmark.EnrichedLog‑net472 1.69 KB 1.64 KB -56 B -3.31%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 2.63μs 2.6ns 10.1ns 0 0 0 1.76 KB
master EnrichedLog netcoreapp3.1 3.44μs 4.38ns 17ns 0 0 0 1.76 KB
master EnrichedLog net472 4.05μs 4.83ns 18.7ns 0.265 0 0 1.69 KB
#7224 EnrichedLog net6.0 2.58μs 12.2ns 50.2ns 0 0 0 1.7 KB
#7224 EnrichedLog netcoreapp3.1 3.54μs 16.8ns 67.1ns 0 0 0 1.7 KB
#7224 EnrichedLog net472 3.95μs 6.42ns 24.9ns 0.254 0 0 1.64 KB
Benchmarks.Trace.Log4netBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑net472 4.57 KB 4.52 KB -55 B -1.20%
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑net6.0 4.37 KB 4.31 KB -58 B -1.33%
Benchmarks.Trace.Log4netBenchmark.EnrichedLog‑netcoreapp3.1 4.37 KB 4.31 KB -58 B -1.33%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 123μs 145ns 563ns 0 0 0 4.37 KB
master EnrichedLog netcoreapp3.1 126μs 330ns 1.24μs 0 0 0 4.37 KB
master EnrichedLog net472 167μs 161ns 603ns 0 0 0 4.57 KB
#7224 EnrichedLog net6.0 123μs 163ns 629ns 0 0 0 4.31 KB
#7224 EnrichedLog netcoreapp3.1 128μs 199ns 772ns 0 0 0 4.31 KB
#7224 EnrichedLog net472 167μs 133ns 515ns 0 0 0 4.52 KB
Benchmarks.Trace.NLogBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑net6.0 2.32 KB 2.26 KB -56 B -2.41%
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑netcoreapp3.1 2.32 KB 2.26 KB -56 B -2.41%
Benchmarks.Trace.NLogBenchmark.EnrichedLog‑net472 2.14 KB 2.08 KB -56 B -2.62%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.94μs 19.3ns 74.6ns 0 0 0 2.32 KB
master EnrichedLog netcoreapp3.1 6.77μs 22.2ns 79.9ns 0 0 0 2.32 KB
master EnrichedLog net472 7.45μs 8.04ns 31.1ns 0.335 0 0 2.14 KB
#7224 EnrichedLog net6.0 4.9μs 14.3ns 55.3ns 0 0 0 2.26 KB
#7224 EnrichedLog netcoreapp3.1 6.7μs 16.9ns 65.4ns 0 0 0 2.26 KB
#7224 EnrichedLog net472 7.57μs 9.12ns 35.3ns 0.301 0 0 2.08 KB
Benchmarks.Trace.RedisBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.RedisBenchmark.SendReceive‑net472 1.21 KB 1.2 KB -8 B -0.66%
Benchmarks.Trace.RedisBenchmark.SendReceive‑net6.0 1.21 KB 1.2 KB -8 B -0.66%
Benchmarks.Trace.RedisBenchmark.SendReceive‑netcoreapp3.1 1.21 KB 1.2 KB -8 B -0.66%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master SendReceive net6.0 2.04μs 10.5ns 49.4ns 0 0 0 1.21 KB
master SendReceive netcoreapp3.1 2.53μs 11.8ns 47.4ns 0 0 0 1.21 KB
master SendReceive net472 3.28μs 2.71ns 10.5ns 0.178 0 0 1.21 KB
#7224 SendReceive net6.0 2.02μs 1.83ns 6.61ns 0 0 0 1.2 KB
#7224 SendReceive netcoreapp3.1 2.61μs 13.5ns 64.7ns 0 0 0 1.2 KB
#7224 SendReceive net472 3.15μs 3.38ns 12.7ns 0.189 0 0 1.2 KB
Benchmarks.Trace.SerilogBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑net472 2.08 KB 2.03 KB -56 B -2.69%
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑netcoreapp3.1 1.69 KB 1.63 KB -56 B -3.32%
Benchmarks.Trace.SerilogBenchmark.EnrichedLog‑net6.0 1.64 KB 1.58 KB -56 B -3.41%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master EnrichedLog net6.0 4.18μs 0.92ns 3.44ns 0 0 0 1.64 KB
master EnrichedLog netcoreapp3.1 5.64μs 23ns 89.3ns 0 0 0 1.69 KB
master EnrichedLog net472 6.67μs 7.28ns 27.3ns 0.298 0 0 2.08 KB
#7224 EnrichedLog net6.0 4.23μs 11.8ns 42.4ns 0 0 0 1.58 KB
#7224 EnrichedLog netcoreapp3.1 5.6μs 13.5ns 48.6ns 0 0 0 1.63 KB
#7224 EnrichedLog net472 6.59μs 5.38ns 20.1ns 0.295 0 0 2.03 KB
Benchmarks.Trace.SpanBenchmark - Same speed ✔️ Fewer allocations 🎉

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net6.0 704 B 696 B -8 B -1.14%
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑netcoreapp3.1 704 B 696 B -8 B -1.14%
Benchmarks.Trace.SpanBenchmark.StartFinishScope‑net472 666 B 658 B -8 B -1.20%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net472 586 B 578 B -8 B -1.37%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑net6.0 584 B 576 B -8 B -1.37%
Benchmarks.Trace.SpanBenchmark.StartFinishSpan‑netcoreapp3.1 584 B 576 B -8 B -1.37%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master StartFinishSpan net6.0 751ns 3.97ns 22.4ns 0 0 0 584 B
master StartFinishSpan netcoreapp3.1 953ns 4.42ns 17.7ns 0 0 0 584 B
master StartFinishSpan net472 914ns 0.809ns 3.13ns 0.0912 0 0 586 B
master StartFinishScope net6.0 918ns 0.479ns 1.73ns 0 0 0 704 B
master StartFinishScope netcoreapp3.1 1.15μs 6.24ns 34.2ns 0 0 0 704 B
master StartFinishScope net472 1.09μs 0.174ns 0.652ns 0.104 0 0 666 B
#7224 StartFinishSpan net6.0 756ns 3.53ns 13.7ns 0 0 0 576 B
#7224 StartFinishSpan netcoreapp3.1 930ns 5.11ns 28.9ns 0 0 0 576 B
#7224 StartFinishSpan net472 909ns 0.27ns 1.05ns 0.0916 0 0 578 B
#7224 StartFinishScope net6.0 889ns 4.9ns 28.1ns 0 0 0 696 B
#7224 StartFinishScope netcoreapp3.1 1.17μs 5.96ns 28.6ns 0 0 0 696 B
#7224 StartFinishScope net472 1.08μs 0.729ns 2.82ns 0.102 0 0 658 B
Benchmarks.Trace.TraceAnnotationsBenchmark - Slower ⚠️ Fewer allocations 🎉

Slower ⚠️ in #7224

Benchmark diff/base Base Median (ns) Diff Median (ns) Modality
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1 1.136 1,373.36 1,560.58

Fewer allocations 🎉 in #7224

Benchmark Base Allocated Diff Allocated Change Change %
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net6.0 704 B 696 B -8 B -1.14%
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑netcoreapp3.1 704 B 696 B -8 B -1.14%
Benchmarks.Trace.TraceAnnotationsBenchmark.RunOnMethodBegin‑net472 666 B 658 B -8 B -1.20%

Raw results

Branch Method Toolchain Mean StdError StdDev Gen 0 Gen 1 Gen 2 Allocated
master RunOnMethodBegin net6.0 1.02μs 5.36ns 26.8ns 0 0 0 704 B
master RunOnMethodBegin netcoreapp3.1 1.38μs 2.28ns 8.82ns 0 0 0 704 B
master RunOnMethodBegin net472 1.36μs 0.141ns 0.544ns 0.102 0 0 666 B
#7224 RunOnMethodBegin net6.0 1.07μs 0.736ns 2.55ns 0 0 0 696 B
#7224 RunOnMethodBegin netcoreapp3.1 1.56μs 1.06ns 4.12ns 0 0 0 696 B
#7224 RunOnMethodBegin net472 1.38μs 0.551ns 2.14ns 0.103 0 0 658 B

@andrewlock andrewlock force-pushed the andrew/snapshot-alpine-symbols branch from 0d7aa62 to 910290e Compare July 15, 2025 12:18
@andrewlock andrewlock force-pushed the andrew/snapshot-alpine-symbols branch from 910290e to 2aed56a Compare July 15, 2025 12:55
@andrewlock andrewlock changed the title Add snapshot tests to document the symbols we depend on in alpine Add snapshot tests to document the symbols we depend on in alpine and native loader Jul 15, 2025
@andrewlock andrewlock marked this pull request as ready for review July 15, 2025 13:12
@andrewlock andrewlock requested review from a team as code owners July 15, 2025 13:12

@tonyredondo tonyredondo left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@andrewlock andrewlock merged commit bd520d5 into master Jul 16, 2025
150 checks passed
@andrewlock andrewlock deleted the andrew/snapshot-alpine-symbols branch July 16, 2025 15:12
@github-actions github-actions Bot added this to the vNext-v3 milestone Jul 16, 2025
andrewlock added a commit that referenced this pull request Aug 20, 2025
## Summary of changes

Refactors the native snapshot checks and adds explicit validation of the
symbols used by the native libraries against an allowlist

## Reason for change

We are shortly going to start building with `alpine:3.17` instead of
`alpine:3.14`, but we don't want to break customers, so we need to make
sure we only use symbols that are available in the earlier version of
musl.

We previously added snapshots for which symbols we depend on here:
- #7224

That means we can see exactly when we depend on new symbols.

However, we realised that libdatadog is already building with a much
later version of alpine (3.22?) and so wanted to ensure that we're not
_Already_ depending on symbols that are not available in some code
paths.

## Implementation details

There are two parts to this PR

- Extract the existing symbol snapshotting out of `Build.cs` and into a
helper class. There's some minor renaming etc, but otherwise this is
just copy-paste, so you can skip the first commit.
- Extract the symbols defined by musl 1.2.2 and libgcc (which the
profiler/libdatadog use for libunwind stuff). These form the "allow
list" of symbols (libgcc is allowed [because .NET depends on it
too](https://learn.microsoft.com/en-us/dotnet/core/install/linux-alpine?tabs=dotnet9#dependencies)).

Now when we extract the symbols for alpine/universal binaries, we check
that we are only depending on symbols defined in these libraries.

> Note that we ignore `ddog_` and `blaze_` symbols which are defined in
libdatadog and required by the profiler, on the assumption that there
will never be a mismatch here.

## Test coverage

Tested locally with all the native libraries we produce, and all looks
good, so as long as CI passes we're good.

## Other details

We should probably add snapshotting of the WAF too to be safe 🤔
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:builds project files, build scripts, pipelines, versioning, releases, packages area:native-library Automatic instrumentation native C++ code (Datadog.Trace.ClrProfiler.Native) area:tests unit tests, integration tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants