Skip to content

Add logfire.instrument_claude_agent_sdk()#1799

Merged
alexmojaki merged 148 commits intomainfrom
instrument-claude-agent-sdk
Mar 27, 2026
Merged

Add logfire.instrument_claude_agent_sdk()#1799
alexmojaki merged 148 commits intomainfrom
instrument-claude-agent-sdk

Conversation

@alexmojaki
Copy link
Copy Markdown
Collaborator

@alexmojaki alexmojaki commented Mar 24, 2026 •
edited by cubic-dev-ai bot
Loading

Summary

  • Adds logfire.instrument_claude_agent_sdk() for native OpenTelemetry instrumentation of the Claude Agent SDK
  • Monkey-patches ClaudeSDKClient to create conversation, turn, and tool spans
  • Injects tracing hooks (PreToolUse, PostToolUse, PostToolUseFailure) for tool call tracing
  • Records usage metrics, cost, session metadata on the root conversation span
  • Uses threading.local() for context propagation (workaround for anyio breaking contextvars)

Test plan

  • 45 tests in tests/otel_integrations/test_claude_agent_sdk.py covering all code paths
  • 100% coverage on logfire/_internal/integrations/claude_agent_sdk.py
  • test_logfire_api.py updated to include the new integration
  • CI green (pyright, ruff, pytest all passing)

🤖 Generated with Claude Code

Summary by cubic

Adds native logfire.instrument_claude_agent_sdk() tracing for the Claude Agent SDK using OTel GenAI semconv. Creates invoke_agent, chat {model}, and execute_tool {tool_name} spans with provider, messages (incl. reasoning), per‑turn usage/cost, conversation ID, and error details.

  • New Features

    • Instruments claude_agent_sdk.ClaudeSDKClient via SDK hooks (scope suffix claude_agent_sdk); returns a context manager to revert; API exposed in logfire and logfire_api (stub uses nullcontext()).
    • Span model: root invoke_agent; sibling chat {model} per assistant turn; sibling execute_tool {tool_name} for tool calls.
    • Records gen_ai.provider.name, gen_ai.input.messages, gen_ai.output.messages, gen_ai.system_instructions, reasoning via ReasoningPart (ThinkingBlock), per‑turn gen_ai.usage.* (incl. cache), gen_ai.response.model, gen_ai.conversation.id, operation.cost, and error.type; sets error level on failures.
    • Captures prompt from .query(); only ClaudeSDKClient is instrumented; instances created after instrumentation get full tool tracing. Docs updated to use logfire.instrument_claude_agent_sdk().

  • Bug Fixes

    • Idempotent instrumentation and clean uninstrument; guard duplicate hook injection (incl. shared/default options).
    • Correct token/cache counts; reset prompt per call; set error level on failed turns/tool calls; cleanup orphaned tool spans.
    • Stability/typing: import claude_agent_sdk at module import; use HookMatcher; isinstance checks for SDK block types; replace getattr with direct attribute access on typed SDK blocks; tighten types; remove dead code; add coverage pragmas; fix sys.modules leak.
    • Tests: cassette‑based integration via a fake claude CLI with --record-claude-cassettes; add dev dependency claude-agent-sdk; CI/resource warning fixes.

Written for commit 09a9ce9. Summary will update on new commits.

alexmojaki added 19 commits March 24, 2026 17:16
@alexmojaki
add claude agent sdk integration with tests
51af258
@alexmojaki
expose instrument_claude_agent_sdk on public API
1e23082
@alexmojaki
update claude agent sdk docs to use native logfire integration
782a2db
@alexmojaki
fix _clear_active_tool_spans to destructure (span, token) tuples and …
0d93e55 
…remove unused collected list
@alexmojaki
add instrument_claude_agent_sdk to logfire-api test
16ff849
@alexmojaki
guard instrument_claude_agent_sdk test on package availability
e937488
@alexmojaki
fix pyright: use importlib.import_module for claude_agent_sdk guard
3b1324a
@alexmojaki
fix cache token counts: use to_int instead of to_float
af36aa8
@alexmojaki
guard against duplicate hook injection on shared options
b6cfe41
@alexmojaki
fix sys.modules leak: save/restore previous module, remove redundant …
b6b4226 
…_logfire_prompt assignments
@alexmojaki
add pragma: no cover for defensive and CI-untestable paths
b5b6cf3
@alexmojaki
add coverage tests for edge cases and utility branches
2908c5f
@alexmojaki
add tests for remaining branch partials: no-content turns, no-usage r…
0d15f39 
…esults, hook edge cases
@alexmojaki
add tests for final branch partials: loop exits, TurnTracker.close wi…
f167e39 
…th no span
@alexmojaki
add pragma: no branch for loop-exit branch partials
a4c344d
@alexmojaki
eliminate branch partials: use setdefault, add test for non-dict non-…
ce6897a 
…text items
@alexmojaki
add test for unknown message type to cover elif-exit branch
515e66d
@alexmojaki
add pragma: no cover to logfire-api claude_agent_sdk stub
4514b23
@alexmojaki
Fix stale _logfire_prompt bug and remove dead code
563a626 
Reset _logfire_prompt at the start of patched_query() so it doesn't
carry over from a previous call when prompt is None.

Also remove unused scaffolding: _logfire_start_time, _logfire_streamed_input,
_next_start_time, and mark_next_start() — all stored but never read.
cubic-dev-ai[bot]

This comment was marked as resolved.

Sign in to view

devin-ai-integration[bot]

This comment was marked as resolved.

Sign in to view

alexmojaki
alexmojaki commented Mar 25, 2026
View reviewed changes
alexmojaki
alexmojaki commented Mar 25, 2026
View reviewed changes
alexmojaki
alexmojaki commented Mar 25, 2026
View reviewed changes
alexmojaki
alexmojaki commented Mar 25, 2026
View reviewed changes
alexmojaki
alexmojaki commented Mar 25, 2026
View reviewed changes
@cloudflare-workers-and-pages
Copy link
Copy Markdown

cloudflare-workers-and-pages bot commented Mar 25, 2026 •
edited
Loading

Deploying logfire-docs with  Cloudflare Pages  Cloudflare Pages

Latest commit: b096c58
Status: ✅  Deploy successful!
Preview URL: https://b46dbff3.logfire-docs.pages.dev
Branch Preview URL: https://instrument-claude-agent-sdk.logfire-docs.pages.dev

View logs

@alexmojaki
fix hook injection: use self.options instead of kwargs extraction
6b62f2e 
original_init sets self.options to ClaudeAgentOptions() when no options
are passed. The kwargs extraction would miss this, skipping hook injection.
alexmojaki
alexmojaki commented Mar 25, 2026
View reviewed changes
alexmojaki
alexmojaki commented Mar 27, 2026
View reviewed changes
devin-ai-integration[bot]

This comment was marked as resolved.

Sign in to view

alexmojaki and others added 2 commits March 27, 2026 15:34
@alexmojaki @claude
stop converting tool responses to str, leave as-is for JSON serializa…
ff940a1 
…tion

Tool responses from the SDK are already structured data. Removing the
str() conversion lets them serialize as proper JSON objects in both the
execute_tool span attribute and conversation history input messages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@alexmojaki @claude
move input_data access inside handle_internal_errors in pre_tool_use_…
0f70428 
…hook

Consistency with post_tool_use_hook and post_tool_use_failure_hook which
already access input_data inside the error handler.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
devin-ai-integration[bot]

This comment was marked as resolved.

Sign in to view

alexmojaki and others added 4 commits March 27, 2026 15:48
@alexmojaki @claude
clear logfire_instance and turn_tracker from thread-local on cleanup
7c1d2d4 
Prevents stale references (including conversation history) from
persisting in thread-local storage after receive_response completes.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@alexmojaki @claude
update spec: tool responses left as-is for JSON serialization
529b66c 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@alexmojaki @claude
add system_instructions to chat spans and update_name for model
184b55d 
Chat spans now include gen_ai.system_instructions so each span is
self-contained. Span name is updated to 'chat {model}' via update_name
when the model is known.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@alexmojaki @claude
set request model equal to response model on chat spans
9ee7f6c 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
devin-ai-integration[bot]

This comment was marked as resolved.

Sign in to view

alexmojaki and others added 4 commits March 27, 2026 16:23
@alexmojaki @claude
use _start/_end for tool spans, attach context only momentarily
532d448 
Cross-async-context detach always fails with OTel error logs, and
keeping context attached doesn't help (tool code runs in yet another
async context). Now: attach root context, _start() the span to capture
the parent link, immediately detach. No context left attached means no
cross-context detach errors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@alexmojaki @claude
add pragma: no branch for clear functions that always have the attr set
9a8ca8a 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@alexmojaki @claude
set request and response model on invoke_agent span from ResultMessage
4ff047c 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@alexmojaki @claude
set request and response model on invoke_agent span from turn tracker
cfae108 
The model comes from AssistantMessage (not ResultMessage), so we store
it on the turn tracker and set it on the root span when the result arrives.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
devin-ai-integration[bot]

This comment was marked as resolved.

Sign in to view

alexmojaki and others added 3 commits March 27, 2026 16:57
@alexmojaki @claude
consolidate all per-conversation state into _ConversationState
7e90a10 
Replaces scattered thread-local attributes and the global
_active_tool_spans dict with a single _ConversationState object.
This fixes cross-thread interference where two conversations would
share the same _active_tool_spans dict.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@alexmojaki @claude
add pragma: no cover for defensive post-hook guards
bcee8ed 
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@alexmojaki
Merge branch 'main' into instrument-claude-agent-sdk
4228f80
@alexmojaki alexmojaki changed the title Add native Claude Agent SDK integration Add logfire.instrument_claude_agent_sdk() Mar 27, 2026
@alexmojaki alexmojaki enabled auto-merge (squash) March 27, 2026 17:26
@alexmojaki alexmojaki merged commit 6de926b into main Mar 27, 2026
17 checks passed
@alexmojaki alexmojaki deleted the instrument-claude-agent-sdk branch March 27, 2026 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@devin-ai-integration devin-ai-integration[bot] devin-ai-integration[bot] left review comments

@cubic-dev-ai cubic-dev-ai[bot] cubic-dev-ai[bot] left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@alexmojaki