Summary

Fixes a critical bug where the Qwen3Coder streaming tool parser silently drops parameters when speculative decoding is enabled (e.g. num_speculative_tokens=5). This causes downstream clients to receive tool calls with missing fields — for example a write tool call arrives as {"filePath": "/path/file"} with no content field, which the client sees as content: undefined.

Supersedes #35421, which partially addressed the issue but left the two root causes unfixed.

Why #35421 doesn't work

PR #35421 correctly identified and fixed several real issues:

• streamed_args_for_tool tracking (append "" on header, accumulate "{"/"}")
• The json_started condition that skipped "{" when parameter_prefix was in delta_text
• The serving.py double-serialization of already-JSON-string arguments

However, it did not fix the two structural bugs that actually cause parameter loss under speculative decoding:

Bug 1: Close-before-params ordering (the root cause)

In the current code on main (and unchanged by #35421), the </function> close check runs before the parameter extraction loop:

# Line 531 — runs FIRST
if not self.json_closed and self.function_end_token in tool_text:
    self.json_closed = True
    ...
    return DeltaMessage(arguments="}")  # ← emits "}" and returns

# Line 591 — runs SECOND (but never reached if </function> fired above)
if not self.in_param and self.param_count < len(param_starts):
    # process parameter...

With speculative decoding, a single burst can deliver the final parameter value together with </function>. The close check sees </function> in tool_text, emits "}", sets in_function = False, and returns. The parameter loop never executes. The parameter is silently lost.

This PR fixes it by moving parameter processing before the close check.

Bug 2: Single-pass if + return None early exit

The parameter extraction uses a single if block that processes exactly one parameter per call. When a parameter is incomplete (still streaming), it executes return None:

else:
    # Still streaming, wait for more content
    return None  # ← bails out entirely

With speculative decoding, a burst can contain multiple parameters where the first N are complete but the last is still streaming. The return None discards all the already-complete parameters that were never emitted.

This PR fixes it by using a while loop that accumulates all complete parameter fragments, with break (instead of return None) when one is incomplete. Already-complete fragments are emitted before yielding control.

Bug 3: already_added dedup prevents same-name tool calls

The current code deduplicates prev_tool_call_arr entries by function name:

already_added = any(tool.get("name") == self.current_function_name ...)
if not already_added:
    self.prev_tool_call_arr.append(...)

Two calls to the same function (e.g. two read calls) share a single entry. Then current_tool_index=1 but prev_tool_call_arr has length 1 → IndexError.

This PR fixes it by always appending (each tool call is a separate invocation) and using index-based lookups.

Changes

• vllm/tool_parsers/qwen3coder_tool_parser.py:
  • Reorder: process parameters FIRST (while loop), check </function> AFTER
  • Replace single-pass if with while loop + break + fragment accumulation
  • Remove already_added name-based dedup; always append to prev_tool_call_arr
  • Use index-based prev_tool_call_arr update (companion to always-append)
  • Track streamed_args_for_tool throughout (header, {, fragments, })
  • Make json_started check unconditional (remove parameter_prefix not in delta_text)
  • Replace silent pass with logger.debug on parse errors
  • Remove dead in_param branch (never activated)
• vllm/entrypoints/openai/chat_completion/serving.py:
  • Guard against double-serialization when arguments is already a JSON string

Test plan

• Verify with single-token streaming (no speculative decode) — behavior should be unchanged
• Verify with num_speculative_tokens=5 — multi-parameter tool calls (e.g. write with filePath + content) should have all parameters present
• Verify parallel tool calls with the same function name (e.g. two read calls) work correctly
• Run existing test_qwen3coder_tool_parser.py tests

🤖 Generated with Claude Code