Skip to content

fix(anthropic): handle server-side tool blocks without panicking#1750

Merged
looplj merged 1 commit into
looplj:unstablefrom
xdqi:fix/server-tool-use-panic
Jun 1, 2026
Merged

fix(anthropic): handle server-side tool blocks without panicking#1750
looplj merged 1 commit into
looplj:unstablefrom
xdqi:fix/server-tool-use-panic

Conversation

@xdqi

@xdqi xdqi commented May 31, 2026

Copy link
Copy Markdown

Summary

Fix a nil pointer dereference panic in the Anthropic outbound stream transformer when Claude uses server-side tools (e.g. web_search, code_execution).

Problem

When the Anthropic API returns server_tool_use or web_search_tool_result content blocks (triggered by features like Claude's web search), AxonHub panics with:

panic: runtime error: invalid memory address or nil pointer dereference
goroutine ... [running]:
github.com/looplj/axonhub/llm/transformer/anthropic.(*outboundStream).transformStreamChunk(...)
    /build/llm/transformer/anthropic/outbound_stream.go:213

The root cause is that outbound_stream.go only matched "tool_use" exactly for content block start events. When server_tool_use arrived, it was not tracked — so subsequent input_json_delta events attempted to index into an empty/nil tool call slice.

Fix

  • Replace exact == "tool_use" check with isAnthropicToolUseLike() that matches any *_tool_use suffix (covers server_tool_use, mcp_tool_use, etc.)
  • Add nil guard in input_json_delta handler before accessing state.toolCalls
  • Handle *_tool_result content blocks (web_search_tool_result, code_execution_tool_result) in both streaming and non-streaming paths
  • Pass through server tool results in inbound (client→Anthropic) direction for multi-turn conversations
  • Add comprehensive test coverage for server tool use round-trips

@xdqi xdqi force-pushed the fix/server-tool-use-panic branch from 02dedda to 6dc895d Compare May 31, 2026 02:00
@greptile-apps

greptile-apps Bot commented May 31, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes a nil-pointer dereference panic in the Anthropic outbound stream transformer triggered when Claude uses server-side tools (web_search, code_execution). The root cause was that content_block_start events for server_tool_use were not tracked, so subsequent input_json_delta events dereferenced a nil map entry.

  • Panic fix (streaming): isAnthropicToolUseLike replaces the exact \"tool_use\" check throughout the aggregator and outbound stream; a nil guard on state.toolCalls prevents the crash for any untracked block type.
  • Non-streaming path: The previously dead case \"server_tool_use\", \"web_search_tool_result\" branch is removed; default: now routes special blocks through isAnthropicSpecialToolUseBlock / isAnthropicSpecialToolResultBlock so they become proper ToolCall + InlineToolResult entries with block-index metadata for faithful re-ordering.
  • Round-trip fidelity: MessageContent.Raw passes *_tool_result content bytes verbatim through marshal/unmarshal, preserving encrypted_content and other unmodeled fields; a new orderedContentBlock sort restores the original tool/text interleaving on re-emission.

Confidence Score: 5/5

Safe to merge; the nil-pointer panic is correctly fixed in both streaming and non-streaming paths, and the server-tool round-trip is covered by comprehensive new tests.

The primary change — replacing the exact tool_use check with isAnthropicToolUseLike and adding a nil guard — is minimal, targeted, and clearly correct. The non-streaming dead-branch issue flagged in a prior review round has been addressed: the old case "server_tool_use", "web_search_tool_result" is gone and default: now handles those types properly. The block-index ordering logic and Raw byte pass-through are more complex but are exercised by the new stream-round-trip and response-round-trip tests. The one gap found (missing citation flush in the new InlineToolResults handler) is theoretical given current Anthropic API ordering and does not affect any path exercised by the tests.

The inbound_stream.go InlineToolResults handler (lines 723–737) is the only place worth a second look — it omits the flushPendingTextCitations() call that all analogous text-block-close sites include.

Important Files Changed

Filename Overview
llm/transformer/anthropic/outbound_stream.go Core panic fix: replaces exact tool_use check with isAnthropicToolUseLike, adds nil guard on input_json_delta, and routes *_tool_result blocks to InlineToolResult deltas.
llm/transformer/anthropic/inbound_stream.go New InlineToolResults streaming path restores original block type and caller; closes open tool/text blocks before emitting tool_result events, but omits flushPendingTextCitations() before closing a text block.
llm/transformer/anthropic/outbound_convert.go Dead branch (old case "server_tool_use", "web_search_tool_result") removed; default: now properly routes special tool-use and tool-result blocks through isAnthropicSpecialToolUseBlock/isAnthropicSpecialToolResultBlock; block-index tracking added for interleaving.
llm/transformer/anthropic/inbound_convert.go Bidirectional support: inbound path converts special tool-use/result blocks to ToolCall+InlineToolResult with metadata; outbound path re-orders blocks by anthropic_block_index to faithfully restore interleaving.
llm/transformer/anthropic/tool_blocks.go New utility file: predicate functions, metadata helpers, and orderedContentBlock sorting for server-side tool blocks round-tripping.
llm/transformer/anthropic/inline_tool_result.go New file: converts *_tool_result MessageContentBlocks to/from llm.InlineToolResult, preserving original content bytes via TransformerMetadata for byte-identical round-trips.
llm/transformer/anthropic/model.go Adds Raw json.RawMessage to MessageContent for byte-identical content pass-through, Caller field to MessageContentBlock, and SetRaw accessor; UnmarshalJSON now stores raw array bytes.
llm/model.go Adds InlineToolResults []InlineToolResult to llm.Message and the new InlineToolResult type with ToolCallID, Output, IsError, and TransformerMetadata.
llm/transformer/anthropic/aggregator.go Replaces exact tool_use checks with isAnthropicToolUseLike in content block accumulation and validation; *_tool_result blocks preserved as-is during aggregation.
frontend/src/features/requests/utils/response-parser.ts Extends tool_use matching to *_tool_use suffix in both non-streaming and streaming parsing paths for frontend display.

Sequence Diagram

sequenceDiagram
    participant A as Anthropic API
    participant OS as OutboundStream
    participant LLM as llm.Response
    participant IS as InboundStream
    participant C as Client

    A->>OS: content_block_start(server_tool_use)
    OS->>LLM: "ToolCall{id, name, anthropic_type=server_tool_use}"
    A->>OS: input_json_delta x N
    OS->>LLM: "ToolCall{partial args} x N"
    A->>OS: content_block_stop (filtered/dropped)
    A->>OS: content_block_start(web_search_tool_result)
    OS->>LLM: "InlineToolResult{tool_call_id, content Raw bytes}"
    A->>OS: content_block_start(text) + text_delta x N
    OS->>LLM: text content x N
    LLM->>IS: ToolCall chunks
    IS->>C: content_block_start(server_tool_use) + input_json_delta
    LLM->>IS: InlineToolResult chunk
    IS->>C: content_block_stop + content_block_start(web_search_tool_result) + content_block_stop
    LLM->>IS: text chunks
    IS->>C: content_block_start(text) + text_delta x N
Loading

Reviews (2): Last reviewed commit: "fix(anthropic): handle server-side tool ..." | Re-trigger Greptile

Comment thread llm/transformer/anthropic/outbound_convert.go Outdated
Comment on lines 329 to 337
err := json.Unmarshal(data, &blocks)
if err == nil {
c.MultipleContent = blocks
// Preserve the original bytes so unknown nested fields (e.g.
// web_search_result's url/title/encrypted_content) survive a
// round-trip via MarshalJSON.
c.Raw = append(c.Raw[:0], data...)
return nil
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Raw set unconditionally creates a silent mutation hazard

UnmarshalJSON now records Raw for every array-valued MessageContent. Because MarshalJSON checks if len(c.Raw) > 0 first, any modification to MultipleContent (or Content) after deserialization is silently ignored when the object is marshalled. There is no mechanism to invalidate Raw when the struct fields change.

For current code paths the in-memory structs are read-only after deserialization, so this is not immediately broken. However, it is an invisible footgun: a future caller that deserializes a MessageContent, appends a block, and re-marshals it will get the original bytes back with no error or warning. A safer approach is to restrict Raw to the specific content types that need byte-identical round-tripping (e.g. only set it when the block is a *_tool_result), or clear Raw whenever Content or MultipleContent is assigned.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Acknowledged. This is an intentional trade-off: Raw exists specifically so that *_tool_result content blocks (which contain fields like url, title, encrypted_content not modeled on MessageContentBlock) can survive a byte-identical round-trip through the pipeline.

In practice, all code paths that receive a deserialized MessageContent treat it as read-only — the struct is only built for forwarding, never mutated post-unmarshal. Adding a ClearRaw() invalidation hook or scoping Raw to specific block types would add complexity for a scenario that doesn't arise today.

If a future caller needs to mutate and re-serialize, they would need to call SetRaw(nil) explicitly — but that's a bridge to cross when we get there. Happy to revisit if the maintainers prefer defensive invalidation.

Root cause: outbound_stream.go only recognized plain "tool_use" content
blocks, so an Anthropic-native server_tool_use (e.g. web_search) followed
by input_json_delta chunks hit a nil map entry and panicked at
outbound_stream.go:216.

Changes:
- Treat any *_tool_use content block as tool-use-like (plain tool_use
  stays vanilla; anything with a suffix is tagged with anthropic_type +
  optional anthropic_caller in ToolCall.TransformerMetadata so the block
  round-trips byte-compatibly).
- Treat any *_tool_result content block as tool-result-like. Special
  results now travel on the assistant llm.Message via a new
  InlineToolResults slice, carrying original block type, caller, and raw
  content bytes through TransformerMetadata.
- outbound_stream: nil-guard input_json_delta, emit inline tool results
  as assistant deltas.
- aggregator / outbound_convert / inbound_convert / frontend response
  parser: extend matching so *_tool_use suffix variants all participate
  in the existing tool-call flow.
- Preserve content-block ordering: tag each text / tool_use / tool_result
  with its original Anthropic block index via anthropic_block_index, and
  emit in that order on the Anthropic side so server-tool calls stay
  before the assistant text that narrates them.
- inbound_stream: rebuild *_tool_use block type / caller from metadata
  and emit InlineToolResults as content_block_start + content_block_stop
  pairs, closing any open text/tool block first.
- Add MessageContent.Raw escape hatch populated on both UnmarshalJSON and
  the round-trip helpers so unknown nested fields (e.g. web_search_result
  url / title / encrypted_content) survive byte-identical.
- Tests: replay production SSE for no-panic assertion; new dedicated
  round-trip tests for the non-streaming and streaming paths assert
  block ordering (tool use/result precede assistant text) and wire-level
  preservation of nested web_search_result fields.

Spec: specs/20260420-fix-server-tool-use-panic
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@xdqi xdqi force-pushed the fix/server-tool-use-panic branch from 6dc895d to 65e2e97 Compare May 31, 2026 02:11
@xdqi

xdqi commented May 31, 2026

Copy link
Copy Markdown
Author

Thanks for the review! The shadowing issue you identified in outbound_convert.go has been fixed in the latest push (65e2e97):

  • Removed the explicit case "server_tool_use", "web_search_tool_result": that was shadowing the default handler
  • server_tool_use blocks now correctly flow through isAnthropicSpecialToolUseBlocktoolCallFromAnthropicBlock to produce llm.ToolCall
  • web_search_tool_result blocks now correctly flow through isAnthropicSpecialToolResultBlockinlineToolResultFromBlock to produce llm.InlineToolResult
  • The TransformerMetadataKeyAnthropicResponseContent metadata is still populated (for citation round-trip fidelity) alongside the new per-item metadata

All tests pass, including TestConvertToLlmResponse_ServerToolUse and the existing TestAnthropicTransformResponse_WebSearchBlocks_RoundTripIntegration.

@looplj looplj merged commit 54ffbdf into looplj:unstable Jun 1, 2026
4 checks passed
junjiangao pushed a commit to junjiangao/axonhub that referenced this pull request Jun 2, 2026
…plj#1750)

Root cause: outbound_stream.go only recognized plain "tool_use" content
blocks, so an Anthropic-native server_tool_use (e.g. web_search) followed
by input_json_delta chunks hit a nil map entry and panicked at
outbound_stream.go:216.

Changes:
- Treat any *_tool_use content block as tool-use-like (plain tool_use
  stays vanilla; anything with a suffix is tagged with anthropic_type +
  optional anthropic_caller in ToolCall.TransformerMetadata so the block
  round-trips byte-compatibly).
- Treat any *_tool_result content block as tool-result-like. Special
  results now travel on the assistant llm.Message via a new
  InlineToolResults slice, carrying original block type, caller, and raw
  content bytes through TransformerMetadata.
- outbound_stream: nil-guard input_json_delta, emit inline tool results
  as assistant deltas.
- aggregator / outbound_convert / inbound_convert / frontend response
  parser: extend matching so *_tool_use suffix variants all participate
  in the existing tool-call flow.
- Preserve content-block ordering: tag each text / tool_use / tool_result
  with its original Anthropic block index via anthropic_block_index, and
  emit in that order on the Anthropic side so server-tool calls stay
  before the assistant text that narrates them.
- inbound_stream: rebuild *_tool_use block type / caller from metadata
  and emit InlineToolResults as content_block_start + content_block_stop
  pairs, closing any open text/tool block first.
- Add MessageContent.Raw escape hatch populated on both UnmarshalJSON and
  the round-trip helpers so unknown nested fields (e.g. web_search_result
  url / title / encrypted_content) survive byte-identical.
- Tests: replay production SSE for no-panic assertion; new dedicated
  round-trip tests for the non-streaming and streaming paths assert
  block ordering (tool use/result precede assistant text) and wire-level
  preservation of nested web_search_result fields.

Spec: specs/20260420-fix-server-tool-use-panic

Co-authored-by: Yeechan Lu <git@orzfly.com>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants