Skip to content

StreamFrameFlowBuilder.emitToolCallDelta() incorrectly treats every chunk with non-null id as a new tool call #2002

@yzhengfei

Description

@yzhengfei

Description

When streaming tool call responses from certain OpenAI-compatible LLM providers, the StreamFrameFlowBuilder.emitToolCallDelta() method incorrectly emits a ToolCallComplete frame after every ToolCallDelta chunk, instead of accumulating the chunks and only emitting one ToolCallComplete at the end.

This causes a single logical tool call to be fragmented into multiple "complete" tool calls, each containing only a small piece of the arguments.

Root Cause

In StreamFrameFlowBuilder.kt, line 196:

val new: PendingToolCall = if (id != null || index != previous?.index) {
    tryEmitPendingToolCall()
    PendingToolCall(id, name, args, index)
} else {
    // append to existing pending tool call...
}

The condition id != null assumes that only the first chunk of a tool call carries a non-null id. This is true for the standard OpenAI Chat Completions streaming format (where OpenAIStreamToolCall.id is String? and only present in the first chunk), but it breaks for providers that include id in every streaming chunk.

When id is present in every chunk, each emitToolCallDelta() call is treated as the start of a new tool call, which:

  1. Calls tryEmitPendingToolCall() → emits ToolCallComplete for the previous (incomplete) chunk
  2. Creates a brand-new PendingToolCall, discarding the accumulated state

Observed Behavior

Input stream (3 chunks for one tool call):

ToolCallDelta(id=call_1, name=readFile, content=,         index=0)
ToolCallDelta(id=call_1, name=null,    content={"path": ", index=0)
ToolCallDelta(id=call_1, name=null,    content=/Users,     index=0)

Actual output (broken):

ToolCallDelta(id=call_1, name=readFile, content=,          index=0)
ToolCallComplete(id=call_1, name=readFile, content=)          ← premature!
ToolCallDelta(id=call_1, name=null,    content={"path": ", index=0)
ToolCallComplete(id=call_1, name=,     content={"path": ")    ← premature!
ToolCallDelta(id=call_1, name=null,    content=/Users,      index=0)
ToolCallComplete(id=call_1, name=,     content=/Users)         ← premature!

Expected output:

ToolCallDelta(id=call_1, name=readFile, content=,          index=0)
ToolCallDelta(id=call_1, name=null,    content={"path": ", index=0)
ToolCallDelta(id=call_1, name=null,    content=/Users,     index=0)
ToolCallComplete(id=call_1, name=readFile, content={"path": "/Users"}  index=0)  ← one complete call

Suggested Fix

Change the condition in emitToolCallDelta() from:

if (id != null || index != previous?.index)

to:

if ((id != null && id != previous?.id) || (index != null && index != previous?.index))

This ensures that a chunk with the same id as the previous one is treated as a continuation, not a new tool call. The behavior for null id (continuation chunks) is preserved, and null index is also handled correctly.

Affected Providers

This bug affects any OpenAI-compatible LLM provider that includes the id field in every streaming chunk of a tool call. The standard OpenAI Chat Completions API only sends id in the first chunk (OpenAIStreamToolCall.id is String?), so it is unaffected. However:

  • OpenRouter (OpenRouterLLMClient) uses OpenAIToolCall which has id: String (non-nullable), meaning every chunk carries the id. This is likely affected.
  • Other OpenAI-compatible providers that include id in every chunk will also be affected.

Metadata

Metadata

Labels

bugSomething isn't working

Type

No fields configured for Bug.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions