Skip to content

[APMSVLS-464][APMSVLS-463] fix: Configure span kind and peer tag stats#1189

Open
lucaspimentel wants to merge 6 commits into
mainfrom
lpimentel/fix-stats-span-kind-peer-tags
Open

[APMSVLS-464][APMSVLS-463] fix: Configure span kind and peer tag stats#1189
lucaspimentel wants to merge 6 commits into
mainfrom
lpimentel/fix-stats-span-kind-peer-tags

Conversation

@lucaspimentel

@lucaspimentel lucaspimentel commented Apr 16, 2026

Copy link
Copy Markdown
Member

Overview

Configure SpanConcentrator in bottlecap with the Go agent's default ComputeStatsBySpanKind span kinds and basePeerTags peer tag keys, fixing two gaps in agent-side trace stats computation:

  • APMSVLS-464: span_kinds_stats_computed was empty, so non-top-level, non-measured spans with span.kind = server/client/producer/consumer) were silently excluded from stats.
  • APMSVLS-463: peer_tag_keys was empty, so client/producer/consumer spans had no per-dependency granularity in stats output (e.g., all S3 buckets, DynamoDB tables, Kafka topics lumped together).

Changes

  • Add STATS_ELIGIBLE_SPAN_KINDS constant (4 span kinds matching the Go agent's KindsComputed)
  • Add DEFAULT_PEER_TAG_KEYS constant (43 peer tag keys matching the Go agent's basePeerTags from mappings.json)
  • Pass both to SpanConcentrator::new() instead of empty vecs
  • Add tests that exercise the full StatsConcentratorService pipeline for both features

Follow-up

Both constants are hand-copied from the Go agent (KindsComputed and basePeerTags), which is the source of truth, and the same lists are also duplicated in serverless-components. A future refactor should keep the Rust agents in sync with the Go agent rather than each maintaining its own copy.

Testing

  • test_span_kind_stats_computed: sends a non-root, non-measured client span through the service, verifies stats are produced with span_kind="client"
  • test_peer_tags_populated: sends a client span with db.instance and db.system meta, verifies peer_tags contains both in the stats output
  • Tests were written first and confirmed failing before the fix was applied
  • Full test suite still passes

"Turns out an empty vec is a great way to compute stats on absolutely nothing. Efficient, but not very useful." — Claude 🤖

@lucaspimentel lucaspimentel changed the title [APMSVLS-464][APMSVLS-463] fix: Configure span kind and peer tag stats in bottlecap [APMSVLS-464][APMSVLS-463] fix: Configure span kind and peer tag stats Apr 16, 2026
Tests verify that StatsConcentratorService computes stats
for spans with span.kind and populates peer_tags in output.
Both tests currently fail due to empty span_kinds_stats_computed
and peer_tag_keys vecs passed to SpanConcentrator::new().

🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
Pass STATS_ELIGIBLE_SPAN_KINDS and DEFAULT_PEER_TAG_KEYS to
SpanConcentrator::new() to match the Go agent defaults. This
enables stats for OTel spans with span.kind and adds per-dependency
granularity via peer tags for client/producer/consumer spans.

🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
@lucaspimentel lucaspimentel force-pushed the lpimentel/fix-stats-span-kind-peer-tags branch from c06bc9b to 2eb4403 Compare June 10, 2026 17:58
@datadog-datadog-prod-us1

datadog-datadog-prod-us1 Bot commented Jun 10, 2026

Copy link
Copy Markdown

Pipelines

Fix all issues with BitsAI

⚠️ Warnings

🚦 3 Pipeline jobs failed

DataDog/datadog-lambda-extension | bottlecap (arm64)   View in Datadog   GitLab

DataDog/datadog-lambda-extension | e2e-test-status (amd64)   View in Datadog   GitLab

DataDog/datadog-lambda-extension | e2e-test-status (amd64, fips)   View in Datadog   GitLab

Useful? React with 👍 / 👎

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 9e45de2 | Docs | Datadog PR Page | Give us feedback!

🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
🤖 Co-Authored-By: Claude Code <noreply@anthropic.com>
@lucaspimentel lucaspimentel marked this pull request as ready for review June 10, 2026 21:49
@lucaspimentel lucaspimentel requested review from a team as code owners June 10, 2026 21:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR configures Bottlecap’s SpanConcentrator to match the Go agent defaults for (1) which span.kind values are eligible for stats computation and (2) which peer-related tag keys are used to populate per-dependency peer_tags, closing gaps in agent-side trace stats aggregation.

Changes:

  • Configure SpanConcentrator with the Go agent’s default computed span kinds (client, server, producer, consumer) instead of an empty list.
  • Configure SpanConcentrator with the Go agent’s default peer tag key set (basePeerTags) instead of an empty list.
  • Add end-to-end tests for span-kind-based stats eligibility and peer tag population in emitted stats.

Comment on lines +304 to +306
/// A non-root, non-measured span with `span.kind`="client" should produce stats
/// when `ComputeStatsBySpanKind` is enabled (i.e. `span_kinds_stats_computed` is
/// populated). With the current empty vec, this span is silently dropped.
Comment on lines +339 to +341
/// A client span with peer tag meta keys (`db.instance`, `db.system`) should produce
/// stats with non-empty `peer_tags` when `peer_tag_keys` is configured. With the
/// current empty vec, `peer_tags` in the output will always be empty.
Comment on lines +356 to +360
assert!(
result.is_some(),
"Expected stats for a client span with peer tags, but got None. \
span.kind-based eligibility is not working."
);
/// TODO: The source of truth is the Go agent's `basePeerTags` (derived from
/// pkg/trace/semantics/mappings.json); this list is hand-copied here and in other Rust repos.
/// Refactor so they stay in sync with the Go agent instead of each keeping its own copy.
const DEFAULT_PEER_TAG_KEYS: [&str; 43] = [

@litianningdatadog litianningdatadog Jun 11, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same as above

/// TODO: The source of truth is the Go agent's `KindsComputed`; this list is hand-copied here
/// and in other Rust repos. Refactor so they stay in sync with the Go agent instead of each
/// keeping its own copy.
const STATS_ELIGIBLE_SPAN_KINDS: [&str; 4] = ["client", "consumer", "producer", "server"];

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: prefer &[&str] over [&str; 4] to make future update easier.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants