Skip to content

feat(transformer): add cross-provider search citation and annotation support#1643

Merged
looplj merged 5 commits into
looplj:unstablefrom
ttttmr:worktree-openai-responses-web-search
May 12, 2026
Merged

feat(transformer): add cross-provider search citation and annotation support#1643
looplj merged 5 commits into
looplj:unstablefrom
ttttmr:worktree-openai-responses-web-search

Conversation

@ttttmr

@ttttmr ttttmr commented May 11, 2026

Copy link
Copy Markdown
Contributor

Summary

  • add cross-provider search citation and annotation preservation across OpenAI Responses, OpenAI Chat, Anthropic, and Gemini transformers
  • add integration and stream coverage for citation round-trips, final-event-only annotations, and orchestrator/API end-to-end paths
  • fix rebase drift in citation transformer signatures and keep the validated transformer behavior intact

Test plan

  • go -C "/Users/bytedance/Dev/axonhub/.claude/worktrees/openai-responses-web-search/llm" test ./transformer/openai/responses -run 'Integration|Stream' -count=1
  • go -C "/Users/bytedance/Dev/axonhub/.claude/worktrees/openai-responses-web-search/llm" test ./transformer/openai ./transformer/anthropic ./transformer/gemini -run 'Integration|Stream' -count=1
  • go -C "/Users/bytedance/Dev/axonhub/.claude/worktrees/openai-responses-web-search" test ./internal/server/orchestrator ./internal/server/api -count=1

🤖 Generated with Claude Code

ttttmr and others added 3 commits May 11, 2026 14:35
Preserve unified web_search tools when Anthropic-style requests are routed through the OpenAI Responses outbound transformer, including the documented filters schema and request coverage.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Keep citation and web search metadata intact across OpenAI Responses, Anthropic, and Gemini transforms so streamed and non-streamed outputs stay semantically consistent end to end.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Restore the current branch signatures in the Responses, Anthropic, and Gemini citation paths after rebase conflicts mixed in incompatible scope-aware variants. Update orchestrator citation tests to initialize the channel limiter manager required by the current middleware stack.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@ttttmr

ttttmr commented May 11, 2026

Copy link
Copy Markdown
Contributor Author

#1313
跨api格式搜索转换,并且保留citations

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for web search citations and grounding metadata across multiple LLM providers (Anthropic, Gemini, and OpenAI). It updates the unified LLM schema to include citation annotations (StartIndex, EndIndex) and adds logic to preserve these annotations during streaming and non-streaming transformations. Additionally, it includes comprehensive integration tests to validate citation round-trips and grounding metadata handling. I have kept the review comment regarding indentation, as it points to a readability issue in the aggregator logic.

Comment thread llm/transformer/anthropic/aggregator.go Outdated
Comment on lines 77 to 85
if contentBlocks[index].Type == "text" {
contentBlocks[index].Citations = append(contentBlocks[index].Citations, *event.Delta.Citation)
}
}

if event.Delta.Thinking != nil {
if contentBlocks[index].Type == "thinking" {
if contentBlocks[index].Thinking == nil {
contentBlocks[index].Thinking = lo.ToPtr("")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The indentation of the if blocks for Citation and Thinking deltas is inconsistent with the surrounding code. Please ensure that nested if statements are correctly indented to improve readability.

@greptile-apps

greptile-apps Bot commented May 11, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds end-to-end preservation of search citations and annotations across all four transformer layers (OpenAI Responses, OpenAI Chat, Anthropic, and Gemini), enabling citation round-trips when a request passes through multiple provider adapters.

  • OpenAI Responses: outbound_convert.go and outbound_stream.go extract web_search_call items into TransformerMetadata and track per-content-part annotation offsets using utf8.RuneCountInString; inbound.go converts llm.Annotation back to Responses API Annotation objects and restores web_search_call output items from metadata.
  • Anthropic: outbound_convert.go extracts TextCitation from text blocks into llm.Annotation, and preserves native provider content blocks (including server_tool_use/web_search_tool_result) via TransformerMetadata; inbound_stream.go buffers pending citations and flushes them as citations_delta events just before the text block closes.
  • Gemini: outbound_convert.go derives llm.Annotation from both CitationMetadata and GroundingMetadata; inbound_convert.go converts annotations back to CitationMetadata and maps llm.ToolTypeWebSearch to GoogleSearch.

Confidence Score: 5/5

Safe to merge — the core citation round-trip logic is well-tested across all four transformer layers with no correctness regressions found.

The streaming metadata fallback path in outbound_stream.go properly handles the case where web search call metadata exists but no annotations are emitted, closing the gap flagged in prior review. All other new code paths (Anthropic citation buffering/flushing, Gemini annotation derivation, Responses API annotation offset tracking) have corresponding unit and integration tests, and the logic checks out under manual inspection.

No files require special attention beyond the minor comment wording in llm/transformer/openai/model.go.

Important Files Changed

Filename Overview
llm/model.go Adds StartIndex and EndIndex (*int64) fields to llm.Annotation, documented as Unicode code-point (rune) offsets; straightforward model extension.
llm/transformer/openai/model.go Adds StartIndex/EndIndex fields to the Chat Completions Annotation struct; comments describe them as "byte offset" which is inconsistent with the unified llm.Annotation doc that calls the same field a "Unicode code-point (rune) offset".
llm/transformer/openai/responses/model.go Promotes Annotation from an empty struct to a fully typed struct with custom UnmarshalJSON for flat url/title fields; adds WebSearchAction, WebSearchSource, WebSearchFilters, WebSearchUserLocation; ToolChoice.MarshalJSON fix generalises the plain-string path to any mode string, not just "auto".
llm/transformer/openai/responses/outbound_convert.go Adds annotation tracking via rune-counted offsets, web_search_call metadata extraction, and a refactored convertOutputToMessage that directly reads Content.Items instead of going through GetContentItems(); logic appears correct but the change from GetContentItems() to direct slice access is load-bearing and slightly fragile.
llm/transformer/openai/responses/outbound_stream.go Adds TransformerMetadata tracking state and emits it either with the message's annotation chunk or as a fallback in the response.completed handler; properly addresses the prior concern about metadata being dropped when no annotations are present.
llm/transformer/openai/responses/inbound.go Adds web_search tool parsing, getResponseWebSearchCallsFromMetadata reconstruction with JSON round-trip fallback, and attachAnnotationsToFirstTextItem for non-streaming output; logic is sound and well-tested.
llm/transformer/openai/responses/aggregator.go Adds Annotations []Annotation to aggregatedContentPart, propagates annotations from both content_part_added and output_item_done events, and carries them through buildResponse; new ensureContentPart helper cleanly handles index-based access.
llm/transformer/anthropic/inbound_convert.go Adds citationFromLLMAnnotation, mergeAnthropicResponseContentBlocks, and attachCitationsToFirstAnthropicTextBlock; correctly handles deduplication and the OpenAI→Anthropic type remapping (url_citationweb_search_result_location when web search metadata is present).
llm/transformer/anthropic/inbound_stream.go Adds pendingTextCitations buffer flushed as citations_delta events immediately before text block close; the enqueEvent immediately JSON-marshals events, so the &s.contentIndex pointer aliasing is safe; finish-reason path refactored to use explicit contentClosed flag.
llm/transformer/anthropic/outbound_convert.go Extracts TextCitation from text blocks into llm.Annotation, stores native content blocks (including server_tool_use/web_search_tool_result) in TransformerMetadata for round-trip fidelity; correctly scoped to only trigger metadata storage when native block types appear.
llm/transformer/gemini/outbound_convert.go New deriveGeminiAnnotations prefers CitationMetadata citations over GroundingMetadata grounding supports; the int32→int64 cast for segment indices is correct; ToolTypeWebSearch now maps to GoogleSearch tool.
llm/transformer/gemini/inbound_convert.go Adds citationMetadataFromLLMAnnotations and attaches CitationMetadata or restores GroundingMetadata from TransformerMetadata for round-trips; change to `ToolTypeGoogleSearch

Sequence Diagram

sequenceDiagram
    participant Client
    participant ResponsesOutbound as Responses Outbound (OAI→LLM)
    participant LLMResponse as llm.Response
    participant AnthropicInbound as Anthropic Inbound (LLM→Anthropic)
    participant GeminiInbound as Gemini Inbound (LLM→Gemini)

    Note over ResponsesOutbound: web_search_call OutputItemDone
    ResponsesOutbound->>LLMResponse: TransformerMetadata["openai_responses_web_search_calls"]

    Note over ResponsesOutbound: message OutputItemDone (with annotations)
    ResponsesOutbound->>LLMResponse: Choices[0].Delta.Annotations + TransformerMetadata emitted

    Note over ResponsesOutbound: response.completed (fallback)
    ResponsesOutbound->>LLMResponse: TransformerMetadata emitted if not yet sent

    LLMResponse->>AnthropicInbound: message.Annotations + TransformerMetadata
    AnthropicInbound->>AnthropicInbound: "citationFromLLMAnnotation()<br/>(url_citation to web_search_result_location if OAI web search metadata present)"
    AnthropicInbound->>AnthropicInbound: "mergeAnthropicResponseContentBlocks()<br/>(restore native provider blocks from metadata)"
    AnthropicInbound-->>Client: Anthropic Message with citations

    LLMResponse->>GeminiInbound: message.Annotations
    GeminiInbound->>GeminiInbound: citationMetadataFromLLMAnnotations()
    GeminiInbound-->>Client: Gemini GenerateContentResponse with CitationMetadata
Loading

Reviews (3): Last reviewed commit: "fix: preserve responses web search calls..." | Re-trigger Greptile

Comment thread llm/transformer/anthropic/outbound_stream.go
Comment thread llm/model.go Outdated
Comment thread llm/transformer/anthropic/inbound_stream.go Outdated
Refine Anthropic stream stop handling and align test/docs formatting with CI and review feedback.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread llm/transformer/openai/responses/outbound_stream.go
Keep web_search_call metadata on an existing outbound chunk and restore it into the final responses stream output so annotation-free search responses round-trip correctly.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@ttttmr ttttmr changed the title feat: preserve cross-provider search citations feat(transformer): add cross-provider search citation and annotation support May 11, 2026
@looplj

looplj commented May 12, 2026

Copy link
Copy Markdown
Owner

这个牛逼。

@looplj looplj merged commit c1fd96c into looplj:unstable May 12, 2026
4 checks passed
@ttttmr ttttmr deleted the worktree-openai-responses-web-search branch May 27, 2026 03:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants