Skip to content

feat: cross-dialect LUNAROUTE marker routing (Anthropic→OpenAI)#35

Merged
erans merged 20 commits intomainfrom
feature/cross-dialect-marker-routing
Apr 4, 2026
Merged

feat: cross-dialect LUNAROUTE marker routing (Anthropic→OpenAI)#35
erans merged 20 commits intomainfrom
feature/cross-dialect-marker-routing

Conversation

@erans
Copy link
Copy Markdown
Owner

@erans erans commented Apr 4, 2026

Summary

  • Adds cross-dialect routing: Anthropic-format requests can now be routed to OpenAI-type providers via LUNAROUTE markers (e.g., Claude Code → Kimi K2.5 on Cloudflare)
  • Supports both streaming (SSE with synthetic message_start) and non-streaming paths
  • Normal same-dialect passthrough is unchanged — zero additional cost

How it works

When messages_passthrough() detects a marker targeting an OpenAI-type provider:

  1. Parse raw Anthropic JSON → AnthropicMessagesRequest
  2. Normalize via to_normalized()NormalizedRequest
  3. Send through OpenAIConnector.send()/.stream() via the Provider trait
  4. Convert response back via from_normalized() / stream_event_to_anthropic_events()

Test plan

  • Round-trip test: Anthropic JSON → normalize → Anthropic JSON preserves content
  • Tool use round-trip: tool_use/tool_result survives normalization
  • Stream event conversion: NormalizedStreamEvent → Anthropic SSE events
  • Error SSE event serialization
  • Model override ordering verification
  • Full test suite passes (132 unit + 19 integration)
  • Release build clean
  • Roborev review: no issues found

🤖 Generated with Claude Code

erans and others added 20 commits April 3, 2026 17:51
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Return error when marker targets OpenAI provider but no connector available
  (instead of silently falling through to default)
- Propagate stream errors as SSE error events to client
  (instead of silently dropping them)
- Add tests for error event serialization and model override ordering

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- OpenAI connector doesn't emit NormalizedStreamEvent::Start, but
  Anthropic clients expect a message_start event. Prepend a synthetic
  Start event to the stream.
- Respect sse_keepalive_enabled flag in cross-dialect streaming path
  (matching same-dialect behavior).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Claude Code sends thinking, redacted_thinking, and other block types that
our typed AnthropicContentBlock enum didn't handle. Added custom Deserialize
with Unknown catch-all variant for both AnthropicContentBlock and
AnthropicSystemBlock. Also handles tool_result content as either string or
array of blocks (both are valid in the Anthropic API).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Config values like api_key: "${CLOUDFLARE_API_KEY}" were passed as literal
strings instead of being resolved to their environment variable values.
Added shellexpand::full() to expand both ${VAR} and ~ in the raw config
before parsing. Missing env vars now produce a clear startup error.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Anthropic requires tool_use.id to match ^[a-zA-Z0-9_-]+$. When cross-dialect
routing converts OpenAI provider responses back to Anthropic format, tool call
IDs are now sanitized and prefixed with toolu_ to ensure they pass Anthropic's
validation when Claude Code includes them in subsequent conversation history.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
When a model responds with tool_use, Claude Code sends the tool results
as a new user message without the LUNAROUTE marker. The marker extraction
now skips tool-result-only user messages and looks back to the message
that initiated the tool call chain, ensuring multi-step tool calling
stays on the same provider.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1. Config env expansion: use non-failing mode (env_with_context_no_errors)
   so ${VAR} in comments or unresolved vars don't cause config load failures.

2. Content block deserializer: validate required fields (text, id, name,
   tool_use_id) and return proper deserialization errors instead of silently
   defaulting to empty strings for known block types.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ifecycle

- strip_marker now skips tool-result-only messages like extract_marker does,
  preventing marker text from leaking in conversation history
- Streaming conversion tracks active content block index (Option<u32>) instead
  of a boolean, ensuring content_block_stop is emitted for both text and tool
  blocks at the correct index
- Added regression tests for both fixes

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…valid tool_result content

- Stream state now uses ActiveBlock enum (Text/Tool with index) instead of
  Option<u32>, ensuring type/index transitions always emit proper stop/start
  events even for same-index different-type scenarios
- tool_result.content with unsupported shapes (e.g. object, number) now
  returns a deserialization error instead of silently defaulting to empty
- Added debug logging when unknown content blocks are skipped during
  normalization for easier diagnosis

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Only emit InputJsonDelta when the matching tool block is active,
  preventing invalid SSE ordering if arguments arrive before block start
- to_anthropic_tool_id now sanitizes first then checks for toolu_ prefix,
  avoiding double-prefix (toolu_toolu_*) on upstream IDs that already
  have the prefix but contain invalid characters

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…ination

- to_anthropic_tool_id now replaces invalid chars with _ instead of
  dropping them, preserving position info to avoid collisions between
  distinct upstream IDs (e.g. call.a:1 vs calla1)
- Stream error handlers now close any active content block and send
  message_stop before the error event, ensuring valid SSE sequencing
  for strict Anthropic clients (fixed in both passthrough and
  cross-dialect streaming paths)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- to_anthropic_tool_id appends a 4-char hash suffix when sanitization
  alters the original ID, ensuring distinct upstream IDs produce distinct
  Anthropic IDs even if they sanitize to the same base form
- ToolCallDelta handler now checks if the target tool block is already
  active before opening, preventing spurious stop/start events when a
  provider repeats id/name across chunks

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Increased hash suffix from 16-bit to 32-bit for sanitized tool IDs,
  reducing collision probability from ~1/65K to ~1/4B per ID pair
- Added TODO documenting that cross-dialect path skips response recording
  (request recording works, response/tool-call recording needs design)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…version

Replace simple ActiveBlock tracking with StreamBlockState that maintains a
monotonically increasing index allocator. OpenAI tool_call_index values
(which start at 0) can collide with text delta indices; the allocator
ensures each Anthropic content_block gets a unique index regardless of
the upstream indexing scheme. Includes HashMap mapping OpenAI
tool_call_index to allocated Anthropic indices.

Added regression test for text-then-tool at same OpenAI index 0.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
erans added a commit that referenced this pull request Apr 4, 2026
…35)

Add LUNAROUTE marker-based routing that translates Anthropic-format requests
to OpenAI-format providers and back, enabling cross-dialect provider routing.

Key changes:
- Non-streaming and streaming cross-dialect routing (Anthropic→OpenAI→Anthropic)
- Unknown content block tolerance (thinking, redacted_thinking, etc.)
- Tool ID sanitization with collision-safe hashing for Anthropic API compatibility
- Marker inheritance through tool-calling exchanges
- Config environment variable expansion (shellexpand)
- Proper streaming content block lifecycle with unique index allocation
- Stream error termination with valid SSE sequencing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@erans erans merged commit 98a5925 into main Apr 4, 2026
8 checks passed
@erans erans deleted the feature/cross-dialect-marker-routing branch April 4, 2026 05:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant