Skip to content

refactor(span)!: use VecMap for meta, metrics and meta_struct for v04 spans#2043

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 17 commits into
mainfrom
yannham/span-vec-fields-integration
Jun 9, 2026
Merged

refactor(span)!: use VecMap for meta, metrics and meta_struct for v04 spans#2043
gh-worker-dd-mergequeue-cf854d[bot] merged 17 commits into
mainfrom
yannham/span-vec-fields-integration

Conversation

@yannham

@yannham yannham commented May 26, 2026

Copy link
Copy Markdown
Contributor

Depends on #2022 and #2049.

What does this PR do?

This PR continues the native span performance work required to land native spans in dd-trace-js. Follow-up of #2022. Actually use VecMap in the Span data structure.

Motivation

Performance improvement following dd-trace-js native span experiments. See #2022.

Additional considerations

There are some deduplication design choices. We deduplicate before serializing to the agent, because while the current agent implementation would support duplicate keys in a map (with the same semantics of "last one wins"), this is not part of the spec/API, so we can't rely on it. We also deduplicate defensively in the msgpack encoder, but it should already be deduped at this stage.

For byte estimate (byte_size()), we make the choice of not deduplicating. This means potentially overestimating the size of a chunk (and reaching the limit sooner), in exchange of delaying the deduplication to serialization time.

How to test the change?

Run tests.

@datadog-prod-us1-3

datadog-prod-us1-3 Bot commented May 26, 2026

Copy link
Copy Markdown

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
Patch Coverage: 93.92%
Overall Coverage: 73.53% (+0.03%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: a2b962e | Docs | Datadog PR Page | Give us feedback!

@github-actions

github-actions Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

Clippy Allow Annotation Report

Comparing clippy allow annotations between branches:

  • Base Branch: origin/main
  • PR Branch: origin/yannham/span-vec-fields-integration

Summary by Rule

Rule Base Branch PR Branch Change
expect_used 3 1 ✅ -2 (-66.7%)
todo 2 2 No change (0%)
unimplemented 1 1 No change (0%)
unwrap_used 3 3 No change (0%)
Total 9 7 ✅ -2 (-22.2%)

Annotation Counts by File

File Base Branch PR Branch Change
libdd-data-pipeline/src/trace_buffer/mod.rs 1 1 No change (0%)
libdd-data-pipeline/src/trace_exporter/mod.rs 2 2 No change (0%)
libdd-trace-utils/src/msgpack_decoder/decode/string.rs 1 0 ✅ -1 (-100.0%)
libdd-trace-utils/src/msgpack_decoder/v04/mod.rs 1 1 No change (0%)
libdd-trace-utils/src/msgpack_decoder/v05/mod.rs 1 0 ✅ -1 (-100.0%)
libdd-trace-utils/src/trace_utils.rs 2 2 No change (0%)
libdd-trace-utils/src/tracer_payload.rs 1 1 No change (0%)

Annotation Stats by Crate

Crate Base Branch PR Branch Change
clippy-annotation-reporter 5 5 No change (0%)
datadog-ffe-ffi 1 1 No change (0%)
datadog-ipc 21 21 No change (0%)
datadog-live-debugger 4 4 No change (0%)
datadog-live-debugger-ffi 10 10 No change (0%)
datadog-profiling-replayer 4 4 No change (0%)
datadog-sidecar 46 46 No change (0%)
libdd-common 13 13 No change (0%)
libdd-common-ffi 12 12 No change (0%)
libdd-data-pipeline 5 5 No change (0%)
libdd-ddsketch 2 2 No change (0%)
libdd-dogstatsd-client 1 1 No change (0%)
libdd-profiling 13 13 No change (0%)
libdd-remote-config 4 4 No change (0%)
libdd-telemetry 20 20 No change (0%)
libdd-tinybytes 4 4 No change (0%)
libdd-trace-normalization 2 2 No change (0%)
libdd-trace-obfuscation 3 3 No change (0%)
libdd-trace-stats 1 1 No change (0%)
libdd-trace-utils 13 11 ✅ -2 (-15.4%)
Total 184 182 ✅ -2 (-1.1%)

About This Report

This report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality.

@yannham yannham changed the title Yannham/span vec fields integration refactor(span): use VecMap for meta, metrics and meta_struct May 26, 2026
@codecov-commenter

codecov-commenter commented May 26, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 93.31104% with 20 lines in your changes missing coverage. Please review.
✅ Project coverage is 72.91%. Comparing base (15e2e0e) to head (e93e908).

Additional details and impacted files
@@                     Coverage Diff                     @@
##           yannham/vecmap-as-dedup    #2043      +/-   ##
===========================================================
- Coverage                    72.92%   72.91%   -0.01%     
===========================================================
  Files                          460      460              
  Lines                        76481    76558      +77     
===========================================================
+ Hits                         55774    55824      +50     
- Misses                       20707    20734      +27     
Components Coverage Δ
libdd-crashtracker 65.21% <ø> (-0.02%) ⬇️
libdd-crashtracker-ffi 36.82% <ø> (ø)
libdd-alloc 98.77% <ø> (ø)
libdd-data-pipeline 85.61% <80.00%> (+<0.01%) ⬆️
libdd-data-pipeline-ffi 75.68% <ø> (-0.02%) ⬇️
libdd-common 79.89% <ø> (ø)
libdd-common-ffi 74.41% <ø> (ø)
libdd-telemetry 73.34% <ø> (ø)
libdd-telemetry-ffi 31.36% <ø> (ø)
libdd-dogstatsd-client 82.64% <ø> (ø)
datadog-ipc 76.22% <ø> (ø)
libdd-profiling 81.69% <ø> (ø)
libdd-profiling-ffi 64.79% <ø> (ø)
libdd-sampling 97.46% <ø> (ø)
datadog-sidecar 28.98% <57.14%> (-0.22%) ⬇️
datdog-sidecar-ffi 9.23% <57.14%> (-0.94%) ⬇️
spawn-worker 48.86% <ø> (ø)
libdd-tinybytes 93.80% <ø> (ø)
libdd-trace-normalization 81.71% <ø> (ø)
libdd-trace-obfuscation 87.30% <ø> (ø)
libdd-trace-protobuf 68.25% <ø> (ø)
libdd-trace-utils 89.03% <96.51%> (+0.06%) ⬆️
libdd-tracer-flare 86.88% <ø> (ø)
libdd-log 74.83% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@yannham yannham force-pushed the yannham/span-vec-fields-integration branch from 1c727ec to a9e644d Compare May 26, 2026 16:21
Comment thread libdd-data-pipeline/src/trace_buffer/mod.rs Outdated
@dd-octo-sts

dd-octo-sts Bot commented May 26, 2026

Copy link
Copy Markdown
Contributor

Artifact Size Benchmark Report

aarch64-alpine-linux-musl
Artifact Baseline Commit Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so 7.70 MB 7.70 MB +0% (+24 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a 83.49 MB 83.67 MB +.21% (+185.36 KB) 🔍
aarch64-unknown-linux-gnu
Artifact Baseline Commit Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a 94.59 MB 94.77 MB +.19% (+184.80 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.34 MB 10.34 MB +.04% (+4.45 KB) 🔍
libdatadog-x64-windows
Artifact Baseline Commit Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll 24.72 MB 24.83 MB +.44% (+111.50 KB) 🔍
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib 86.89 KB 86.89 KB 0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb 179.84 MB 180.80 MB +.53% (+984.00 KB) 🔍
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib 923.31 MB 925.53 MB +.23% (+2.21 MB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll 8.09 MB 8.09 MB --.04% (-3.50 KB) 💪
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib 86.89 KB 86.89 KB 0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb 23.91 MB 23.93 MB +.09% (+24.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib 47.72 MB 47.77 MB +.11% (+58.16 KB) 🔍
libdatadog-x86-windows
Artifact Baseline Commit Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll 21.43 MB 21.52 MB +.42% (+94.00 KB) 🔍
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib 88.26 KB 88.26 KB 0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb 183.87 MB 184.88 MB +.54% (+1.00 MB) 🔍
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib 916.03 MB 918.20 MB +.23% (+2.17 MB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll 6.25 MB 6.24 MB --.03% (-2.50 KB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib 88.26 KB 88.26 KB 0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb 25.63 MB 25.65 MB +.06% (+16.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib 45.36 MB 45.41 MB +.11% (+52.26 KB) 🔍
x86_64-alpine-linux-musl
Artifact Baseline Commit Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a 74.46 MB 74.59 MB +.18% (+141.06 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so 8.58 MB 8.58 MB +.09% (+8.03 KB) 🔍
x86_64-unknown-linux-gnu
Artifact Baseline Commit Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a 89.87 MB 90.01 MB +.15% (+143.10 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so 10.43 MB 10.44 MB +.10% (+11.67 KB) 🔍

@yannham yannham force-pushed the yannham/span-vec-fields-integration branch 2 times, most recently from 681d36f to 95d3d03 Compare May 27, 2026 09:36
@yannham yannham changed the title refactor(span): use VecMap for meta, metrics and meta_struct refactor(span)!: use VecMap for meta, metrics and meta_struct May 27, 2026
@yannham yannham changed the title refactor(span)!: use VecMap for meta, metrics and meta_struct refactor(span)!: use VecMap for meta, metrics and meta_struct for v04 spans May 27, 2026
@yannham yannham marked this pull request as ready for review May 27, 2026 09:54
@yannham yannham requested review from a team as code owners May 27, 2026 09:54

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 95d3d03685

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread datadog-sidecar-ffi/src/span.rs
@yannham yannham force-pushed the yannham/span-vec-fields branch from 74358f8 to 7e38b23 Compare May 27, 2026 13:59
@yannham yannham force-pushed the yannham/span-vec-fields-integration branch 3 times, most recently from 2d45a7f to 8f43f82 Compare May 27, 2026 15:14
Base automatically changed from yannham/span-vec-fields to main May 28, 2026 08:09
@yannham yannham force-pushed the yannham/span-vec-fields-integration branch 2 times, most recently from 3dc15ea to 57b3ec4 Compare May 28, 2026 13:30
@yannham yannham changed the base branch from main to yannham/vecmap-as-dedup May 28, 2026 13:31
@yannham yannham force-pushed the yannham/span-vec-fields-integration branch from 70b40a4 to a2b962e Compare June 9, 2026 15:36
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit 74284ca into main Jun 9, 2026
120 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot deleted the yannham/span-vec-fields-integration branch June 9, 2026 17:05
gh-worker-dd-mergequeue-cf854d Bot pushed a commit that referenced this pull request Jun 10, 2026
# What does this PR do?

Add foundation types for dd-trace-js change buffer: buffer, opcodes and trace. Follow up of #2043.

# Motivation

See #2022. Required for the integration of native spans into dd-trace-js.

# Additional Notes

This PR just introduces the types and doesn't use them yet, hence the `allow(unused)`.

# How to test the change?

Unit tests provided.

Co-authored-by: ekump <edmund.kump@datadoghq.com>
Co-authored-by: yann.hamdaoui <yann.hamdaoui@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants