Skip to content

feat(aiokafka): add partition and offset to produce span#18344

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
feat/aiokafka-produce-partition-offset
Jun 2, 2026
Merged

feat(aiokafka): add partition and offset to produce span#18344
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
feat/aiokafka-produce-partition-offset

Conversation

@piochelepiotr

Copy link
Copy Markdown
Contributor

Summary

  • After the broker acknowledges a send, populate kafka.partition and kafka.message_offset on the producer span from the resulting RecordMetadata.
  • The partition tag now reflects the partition the broker actually assigned, which may differ from any hint the caller passed via send(..., partition=...).
  • Mirrors the equivalent feature added to the Java tracer in Add Kafka cluster ID and offset tags to producer/consumer spans dd-trace-java#11107.

Test plan

  • Verified locally: aiokafka produce snapshots now carry kafka.partition: 0 / kafka.message_offset: 0 instead of -1 for first-message produces.
  • CI: aiokafka contrib suites.

🤖 Generated with Claude Code

@piochelepiotr piochelepiotr requested review from a team as code owners May 29, 2026 16:01
@cit-pr-commenter-54b7da

Copy link
Copy Markdown

Codeowners resolved as

ddtrace/_trace/trace_handlers.py                                        @DataDog/apm-sdk-capabilities-python @DataDog/apm-core-python @DataDog/apm-idm-python
releasenotes/notes/aiokafka-produce-partition-offset-b890906a737d50a6.yaml  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_getmany_multiple_messages_multiple_topics.json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_getmany_multiple_messages_multiple_topics_with_distributed_tracing.json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_getmany_single_message.json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_and_wait_commit_with_offset.json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_and_wait_failure.json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_and_wait_key[None].json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_and_wait_key[test_key].json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_and_wait_value[None].json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_and_wait_value[hueh_hueh_hueh].json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_and_wait_with_distributed_tracing.json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_commit.json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka.test_send_multiple_servers.json  @DataDog/apm-python
tests/snapshots/tests.contrib.aiokafka.test_aiokafka_dsm.test_data_streams_aiokafka_enabled.json  @DataDog/apm-python

@datadog-datadog-prod-us1-2

This comment has been minimized.

@pr-commenter

pr-commenter Bot commented May 29, 2026

Copy link
Copy Markdown

Benchmarks

Benchmark execution time: 2026-05-29 20:21:09

Comparing candidate commit 3d175fe in PR branch feat/aiokafka-produce-partition-offset with baseline commit cc5064e in branch main.

Found 0 performance improvements and 2 performance regressions! Performance is the same for 383 metrics, 9 unstable metrics.

scenario:span-start

  • 🟥 execution_time [+1.276ms; +1.438ms] or [+8.244%; +9.293%]

scenario:telemetryaddmetric-1-count-metric-1-times

  • 🟥 execution_time [+149.551ns; +185.986ns] or [+7.032%; +8.745%]

After the broker acknowledges a send, set the producer span's
kafka.partition and kafka.message_offset to the values from the resulting
RecordMetadata. The partition reflects what the broker actually assigned,
which may differ from any partition hint the caller passed.

Mirrors the equivalent change in dd-trace-java#11107.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@piochelepiotr piochelepiotr force-pushed the feat/aiokafka-produce-partition-offset branch from 37b9529 to 3d175fe Compare May 29, 2026 19:57
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot merged commit f7a3f18 into main Jun 2, 2026
1253 of 1255 checks passed
@gh-worker-dd-mergequeue-cf854d gh-worker-dd-mergequeue-cf854d Bot deleted the feat/aiokafka-produce-partition-offset branch June 2, 2026 20:29
brettlangdon pushed a commit that referenced this pull request Jun 3, 2026
## Summary
- After the broker acknowledges a send, populate `kafka.partition` and `kafka.message_offset` on the producer span from the resulting `RecordMetadata`.
- The partition tag now reflects the partition the broker actually assigned, which may differ from any hint the caller passed via `send(..., partition=...)`.
- Mirrors the equivalent feature added to the Java tracer in DataDog/dd-trace-java#11107.

## Test plan
- [x] Verified locally: aiokafka produce snapshots now carry `kafka.partition: 0` / `kafka.message_offset: 0` instead of `-1` for first-message produces.
- [ ] CI: aiokafka contrib suites.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-authored-by: piotr.wolski <piotr.wolski@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants