🐛 fix(model-runtime): tolerate null function.name in streaming tool_call deltas#14139
Merged
Merged
Conversation
…all deltas Some providers (NVIDIA NIM with z-ai/glm5 and qwen3.5-MoE, plus some aihubmix-style proxies) open a streaming tool_call with \`function.name = null\` as a start marker and supply the real name in a later delta. The strict MessageToolCallSchema threw ZodError mid-stream and killed the whole operation before any tokens were even recorded. - parseToolCalls: coerce null/undefined name to '' before Zod parse; merge name from subsequent deltas (previously only arguments merged). - RuntimeExecutors: drop tool_calls whose name never resolved to a non-empty string before pushing to state.messages, so they can't poison subsequent history replays on strict providers. Closes LOBE-8199. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## canary #14139 +/- ##
===========================================
- Coverage 86.12% 67.53% -18.60%
===========================================
Files 610 2192 +1582
Lines 51406 188767 +137361
Branches 8689 23211 +14522
===========================================
+ Hits 44273 127480 +83207
- Misses 7005 61158 +54153
- Partials 128 129 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
Merged
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
💻 Change Type
🔗 Related Issue
Closes LOBE-8199.
🔀 Description of Change
Some providers (NVIDIA NIM serving
z-ai/glm5andqwen/qwen3.5-397b-a17b, plus some aihubmix-style proxies) open a streamingtool_calldelta withfunction.name = nullas a start marker, and only supply the real name in a later delta. The strictMessageToolCallSchemathrewZodErrormid-stream and killed the entire operation — in one sampled op the agent had been running for 58 minutes and 113 steps before dying on this.This behavior is the same shape that the
zhipuprovider already preprocesses per-provider (see zhipu/index.ts:104). This PR promotes the same tolerance to the central aggregation layer so it covers any provider exhibiting the same streaming pattern, instead of handling it one provider at a time.Changes:
packages/model-runtime/src/helpers/parseToolCalls.ts— coercefunction.name: null/undefinedto''before the Zod parse so streaming doesn't die; mergefunction.namefrom subsequent deltas (previously onlyargumentsmerged) so the real name patched in later replaces the placeholder. Once-resolved names are not overwritten by later empty-name deltas.src/server/modules/AgentRuntime/RuntimeExecutors.ts— defense-in-depth at the state persist boundary: droptool_callswhosefunction.namenever resolved to a non-empty string before pushing tostate.messages.ToolNameResolveralready filters these out of the dispatch path, but keeping them in replay history would still 400 on strict providers (which is exactly how this bug surfaced). Mirrors the existing LOBE-7761sanitizeToolCallArgumentspattern at the same boundary.🧪 How to Test
Unit tests in
parseToolCalls.test.tscover three new scenarios:name: null+ later delta patches in the real name + final delta appends arguments → full resolution.name: nullis accepted when merging into existing tool_calls.name: ''delta.The existing
"should throw error if incomplete tool calls data"test still passes — we only loosen fornull name; chunks missingfunctionentirely still throw.📝 Additional Information
Observed in 3 production ops (all on
provider: nvidia):op_1777033807076_…op_1777026389129_…op_1776906827308_…Op 3's final step is particularly clarifying: the harness' existing
TRUNCATED_ARGUMENTSpath (builtin.ts:38) was already gracefully catching truncatedargumentsstrings (extended-thinking exhaustingmax_tokens), but the same root cause producingname: nullhad no corresponding graceful path and took down the whole op. This PR closes that asymmetry at the aggregation layer.🤖 Generated with Claude Code