Skip to content

🐛 fix(model-runtime): tolerate null function.name in streaming tool_call deltas#14139

Merged
arvinxx merged 2 commits into
canaryfrom
fix/lobe-8199-streaming-tool-call-name-null
Apr 25, 2026
Merged

🐛 fix(model-runtime): tolerate null function.name in streaming tool_call deltas#14139
arvinxx merged 2 commits into
canaryfrom
fix/lobe-8199-streaming-tool-call-name-null

Conversation

@arvinxx

@arvinxx arvinxx commented Apr 24, 2026

Copy link
Copy Markdown
Member

💻 Change Type

  • 🐛 fix

🔗 Related Issue

Closes LOBE-8199.

🔀 Description of Change

Some providers (NVIDIA NIM serving z-ai/glm5 and qwen/qwen3.5-397b-a17b, plus some aihubmix-style proxies) open a streaming tool_call delta with function.name = null as a start marker, and only supply the real name in a later delta. The strict MessageToolCallSchema threw ZodError mid-stream and killed the entire operation — in one sampled op the agent had been running for 58 minutes and 113 steps before dying on this.

path: ["function", "name"]
expected: "string"  received: "null"

This behavior is the same shape that the zhipu provider already preprocesses per-provider (see zhipu/index.ts:104). This PR promotes the same tolerance to the central aggregation layer so it covers any provider exhibiting the same streaming pattern, instead of handling it one provider at a time.

Changes:

  • packages/model-runtime/src/helpers/parseToolCalls.ts — coerce function.name: null/undefined to '' before the Zod parse so streaming doesn't die; merge function.name from subsequent deltas (previously only arguments merged) so the real name patched in later replaces the placeholder. Once-resolved names are not overwritten by later empty-name deltas.

  • src/server/modules/AgentRuntime/RuntimeExecutors.ts — defense-in-depth at the state persist boundary: drop tool_calls whose function.name never resolved to a non-empty string before pushing to state.messages. ToolNameResolver already filters these out of the dispatch path, but keeping them in replay history would still 400 on strict providers (which is exactly how this bug surfaced). Mirrors the existing LOBE-7761 sanitizeToolCallArguments pattern at the same boundary.

🧪 How to Test

  • Tested locally
  • Added/updated tests
  • No tests needed

Unit tests in parseToolCalls.test.ts cover three new scenarios:

  1. First delta with name: null + later delta patches in the real name + final delta appends arguments → full resolution.
  2. New parallel tool_call opening with name: null is accepted when merging into existing tool_calls.
  3. Already-resolved names are not overwritten by a later name: '' delta.

The existing "should throw error if incomplete tool calls data" test still passes — we only loosen for null name; chunks missing function entirely still throw.

bunx vitest run 'packages/model-runtime/src/helpers/parseToolCalls.test.ts'
  → 10 passed (7 existing + 3 new)

bunx vitest run 'packages/model-runtime/src/providers/zhipu'
  → 83 passed

bunx vitest run 'packages/model-runtime/src/core/streams'
  → 220 passed

bunx vitest run 'src/server/modules/AgentRuntime/__tests__/RuntimeExecutors.test.ts'
  → 93 passed

bun run type-check
  → clean

📝 Additional Information

Observed in 3 production ops (all on provider: nvidia):

OP Model Steps Duration
op_1777033807076_… qwen3.5-397b-a17b 1 1.5s (died on first chunk)
op_1777026389129_… z-ai/glm5 113 58 min
op_1776906827308_… z-ai/glm5 7 10 min

Op 3's final step is particularly clarifying: the harness' existing TRUNCATED_ARGUMENTS path (builtin.ts:38) was already gracefully catching truncated arguments strings (extended-thinking exhausting max_tokens), but the same root cause producing name: null had no corresponding graceful path and took down the whole op. This PR closes that asymmetry at the aggregation layer.

🤖 Generated with Claude Code

…all deltas

Some providers (NVIDIA NIM with z-ai/glm5 and qwen3.5-MoE, plus some
aihubmix-style proxies) open a streaming tool_call with
\`function.name = null\` as a start marker and supply the real name in a
later delta. The strict MessageToolCallSchema threw ZodError mid-stream
and killed the whole operation before any tokens were even recorded.

- parseToolCalls: coerce null/undefined name to '' before Zod parse;
  merge name from subsequent deltas (previously only arguments merged).
- RuntimeExecutors: drop tool_calls whose name never resolved to a
  non-empty string before pushing to state.messages, so they can't
  poison subsequent history replays on strict providers.

Closes LOBE-8199.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@vercel

vercel Bot commented Apr 24, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
lobehub Ready Ready Preview, Comment Apr 24, 2026 4:59pm

Request Review

@dosubot dosubot Bot added the size:M This PR changes 30-99 lines, ignoring generated files. label Apr 24, 2026

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We've reviewed this pull request using the Sourcery rules engine

@dosubot dosubot Bot added the feature:tool Tool calling and function execution label Apr 24, 2026
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@codecov

codecov Bot commented Apr 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.88889% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 67.53%. Comparing base (043d2a8) to head (3b20d9d).
⚠️ Report is 4 commits behind head on canary.

Additional details and impacted files
@@             Coverage Diff             @@
##           canary   #14139       +/-   ##
===========================================
- Coverage   86.12%   67.53%   -18.60%     
===========================================
  Files         610     2192     +1582     
  Lines       51406   188767   +137361     
  Branches     8689    23211    +14522     
===========================================
+ Hits        44273   127480    +83207     
- Misses       7005    61158    +54153     
- Partials      128      129        +1     
Flag Coverage Δ
app 60.57% <100.00%> (?)
database 92.24% <ø> (ø)
packages/agent-runtime 79.82% <ø> (ø)
packages/context-engine 83.23% <ø> (ø)
packages/conversation-flow 92.40% <ø> (ø)
packages/file-loaders 87.02% <ø> (ø)
packages/memory-user-memory 74.74% <ø> (ø)
packages/model-bank 99.89% <ø> (ø)
packages/model-runtime 84.05% <80.00%> (-0.01%) ⬇️
packages/prompts 70.14% <ø> (ø)
packages/python-interpreter 92.90% <ø> (ø)
packages/ssrf-safe-fetch 0.00% <ø> (ø)
packages/utils 88.41% <ø> (ø)
packages/web-crawler 88.66% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
Store 67.16% <ø> (∅)
Services 53.07% <ø> (∅)
Server 66.58% <100.00%> (∅)
Libs 53.30% <ø> (∅)
Utils 80.04% <ø> (-13.43%) ⬇️
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@arvinxx arvinxx merged commit ba05c32 into canary Apr 25, 2026
@arvinxx arvinxx deleted the fix/lobe-8199-streaming-tool-call-name-null branch April 25, 2026 04:17
@arvinxx arvinxx mentioned this pull request Apr 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

feature:tool Tool calling and function execution size:M This PR changes 30-99 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant