fix(ui): live tool call streaming in Control UI webchat#39104
fix(ui): live tool call streaming in Control UI webchat#39104jakepresent wants to merge 4 commits into
Conversation
Tool calls (web_search, exec, etc.) were invisible during streaming in the Control UI webchat and only appeared after a manual page refresh. Six independent issues combined to cause this: 1. Control UI connected with caps: [] so the gateway never sent tool events to it. Fix: register the tool-events capability. (gateway.ts) 2. handleAgentEvent rejected all tool events because the client-generated chatRunId (UUID) never matched the server's engine runId on incoming events. Fix: filter tool events by sessionKey only. (app-tool-stream.ts) 3. Tool cards were gated behind the showThinking toggle, which controls model reasoning tokens. Users with showThinking off saw no tool output. Fix: render tool cards unconditionally via interleaved stream segments and tool messages. (views/chat.ts) 4. On run completion, resetToolStream cleared tool cards but history was not reloaded to repopulate them from persisted state. Fix: detect when tool events were seen during the run and reload history before the streaming state is lost. Return a flag to prevent a duplicate reload from the existing shouldReloadHistoryForFinalEvent path. (app-gateway.ts) 5. No mid-run history refresh after tool results. The server persists tool results as they happen, but the UI never fetched them during a run. Fix: reload history on each tool result event so persisted text and tool output replace streaming fragments. (app-gateway.ts) 6. History reload left stale streaming state (chatToolMessages, stream segments, chatStream, sync timer, toolStreamById/Order) that caused duplicate tool cards and ghost text. Fix: fully reset all streaming state on history load. (controllers/chat.ts) Additional improvements: - Commit in-progress chatStream text as a segment on tool start so text renders above the tool card instead of below it. (app-tool-stream.ts) - Interleave stream segments and tool messages by index for correct visual ordering during streaming. (views/chat.ts) - Cancel pending sync timer in resetToolStream instead of flushing from empty state. (app-tool-stream.ts) Note: brief text fragments may appear in stream segments when a tool event arrives before the server's 150ms delta throttle flushes pending text. These self-correct within ~1s when the tool result triggers a history reload. A server-side fix (flushing the delta buffer before emitting tool events) would eliminate this entirely — tracked in openclaw#39082. Fixes openclaw#38888
Greptile SummaryThis PR fixes six independent issues that together made tool calls ( Key points:
Confidence Score: 3/5
Last reviewed commit: 93cc0d5 |
| if (host.chatStream && host.chatStream.trim().length > 0) { | ||
| host.chatStreamSegments.push({ text: host.chatStream, ts: now }); | ||
| host.chatStream = null; | ||
| host.chatStreamStartedAt = null; | ||
| } |
There was a problem hiding this comment.
Mutating @state array without reassignment
chatStreamSegments.push(...) mutates the array in-place without changing the reference. In Lit, @state-decorated properties only schedule re-renders when the property is assigned (the setter is called). The segment is only visible here because the immediately following host.chatStream = null assignment happens to trigger a re-render that reads the mutated array.
This is a silent coupling: if the chatStream nullification is ever moved, removed, or guarded separately, the pushed segment will be invisible until the next unrelated state change. The safe pattern for Lit is to replace the array:
| if (host.chatStream && host.chatStream.trim().length > 0) { | |
| host.chatStreamSegments.push({ text: host.chatStream, ts: now }); | |
| host.chatStream = null; | |
| host.chatStreamStartedAt = null; | |
| } | |
| if (host.chatStream && host.chatStream.trim().length > 0) { | |
| host.chatStreamSegments = [...host.chatStreamSegments, { text: host.chatStream, ts: now }]; | |
| host.chatStream = null; | |
| host.chatStreamStartedAt = null; | |
| } |
Prompt To Fix With AI
This is a comment left during a code review.
Path: ui/src/ui/app-tool-stream.ts
Line: 440-444
Comment:
**Mutating `@state` array without reassignment**
`chatStreamSegments.push(...)` mutates the array in-place without changing the reference. In Lit, `@state`-decorated properties only schedule re-renders when the property is **assigned** (the setter is called). The segment is only visible here because the immediately following `host.chatStream = null` assignment happens to trigger a re-render that reads the mutated array.
This is a silent coupling: if the `chatStream` nullification is ever moved, removed, or guarded separately, the pushed segment will be invisible until the next unrelated state change. The safe pattern for Lit is to replace the array:
```suggestion
if (host.chatStream && host.chatStream.trim().length > 0) {
host.chatStreamSegments = [...host.chatStreamSegments, { text: host.chatStream, ts: now }];
host.chatStream = null;
host.chatStreamStartedAt = null;
}
```
How can I resolve this? If you propose a fix, please make it concise.| if (!runId || !host.refreshSessionsAfterChat.has(runId)) { | ||
| return; | ||
| // Even without a tracked runId, reload history when tools were used so | ||
| // the persisted tool results replace the now-cleared streaming state. | ||
| if (hadToolEvents && state === "final") { | ||
| void loadChatHistory(host as unknown as OpenClawApp); | ||
| return true; | ||
| } | ||
| return false; | ||
| } |
There was a problem hiding this comment.
hadToolEvents reload guard skips the refreshSessionsAfterChat path
The fix for "history not reloaded after tool runs" (fix #4) only applies when !runId || !host.refreshSessionsAfterChat.has(runId). When a run's server runId IS in refreshSessionsAfterChat and tools were used, execution falls through to the host.refreshSessionsAfterChat.delete(runId) block, which calls loadSessions and returns false. Back in handleChatGatewayEvent, shouldReloadHistoryForFinalEvent returns false for a normal assistant reply (role: "assistant"), so loadChatHistory is never called for that path.
For the common case (!refreshSessionsAfterChat.has(runId)) this is fine because the mid-run loadChatHistory calls on each tool result keep history current, and handleChatEvent appends the final message directly. However, if refreshSessionsAfterChat ends up containing the run ID of a chat that also used tools, the cleared tool-stream state won't be backed by a fresh history fetch. Consider applying the same guard here:
host.refreshSessionsAfterChat.delete(runId);
if (state === "final") {
void loadSessions(host as unknown as OpenClawApp, {
activeMinutes: CHAT_SESSIONS_ACTIVE_MINUTES,
});
// Also reload history if tools were seen, just like the untracked-run path.
if (hadToolEvents) {
void loadChatHistory(host as unknown as OpenClawApp);
return true;
}
}
return false;Prompt To Fix With AI
This is a comment left during a code review.
Path: ui/src/ui/app-gateway.ts
Line: 271-279
Comment:
**`hadToolEvents` reload guard skips the `refreshSessionsAfterChat` path**
The fix for "history not reloaded after tool runs" (fix #4) only applies when `!runId || !host.refreshSessionsAfterChat.has(runId)`. When a run's server `runId` IS in `refreshSessionsAfterChat` and tools were used, execution falls through to the `host.refreshSessionsAfterChat.delete(runId)` block, which calls `loadSessions` and returns `false`. Back in `handleChatGatewayEvent`, `shouldReloadHistoryForFinalEvent` returns `false` for a normal assistant reply (`role: "assistant"`), so `loadChatHistory` is never called for that path.
For the common case (`!refreshSessionsAfterChat.has(runId)`) this is fine because the mid-run `loadChatHistory` calls on each tool `result` keep history current, and `handleChatEvent` appends the final message directly. However, if `refreshSessionsAfterChat` ends up containing the run ID of a chat that also used tools, the cleared tool-stream state won't be backed by a fresh history fetch. Consider applying the same guard here:
```ts
host.refreshSessionsAfterChat.delete(runId);
if (state === "final") {
void loadSessions(host as unknown as OpenClawApp, {
activeMinutes: CHAT_SESSIONS_ACTIVE_MINUTES,
});
// Also reload history if tools were seen, just like the untracked-run path.
if (hadToolEvents) {
void loadChatHistory(host as unknown as OpenClawApp);
return true;
}
}
return false;
```
How can I resolve this? If you propose a fix, please make it concise.| const toolHost = state as unknown as { | ||
| chatToolMessages: unknown[]; | ||
| chatStreamSegments: Array<{ text: string; ts: number }>; | ||
| toolStreamById: Map<string, unknown>; | ||
| toolStreamOrder: string[]; | ||
| toolStreamSyncTimer: number | null; | ||
| }; | ||
| toolHost.chatToolMessages = []; | ||
| toolHost.chatStreamSegments = []; | ||
| toolHost.toolStreamById.clear(); | ||
| toolHost.toolStreamOrder = []; | ||
| if (toolHost.toolStreamSyncTimer != null) { | ||
| clearTimeout(toolHost.toolStreamSyncTimer); | ||
| toolHost.toolStreamSyncTimer = null; | ||
| } | ||
| state.chatStream = null; | ||
| state.chatStreamStartedAt = null; |
There was a problem hiding this comment.
Duplicated streaming-state cleanup across loadChatHistory and resetToolStream
loadChatHistory now manually replicates the internals of resetToolStream (clear toolStreamById, toolStreamOrder, chatToolMessages, chatStreamSegments, cancel timer) plus two extra fields (chatStream, chatStreamStartedAt). resetToolStream was explicitly designed to own this cleanup, so future changes to either location can easily fall out of sync.
Consider exposing a single helper that clears all streaming state — or simply call resetToolStream here and then separately null out the chat stream fields:
// After successfully receiving history:
resetToolStream(state as unknown as Parameters<typeof resetToolStream>[0]);
state.chatStream = null;
state.chatStreamStartedAt = null;This removes the duplicated as unknown as cast and keeps the cleanup in one place.
Prompt To Fix With AI
This is a comment left during a code review.
Path: ui/src/ui/controllers/chat.ts
Line: 74-90
Comment:
**Duplicated streaming-state cleanup across `loadChatHistory` and `resetToolStream`**
`loadChatHistory` now manually replicates the internals of `resetToolStream` (clear `toolStreamById`, `toolStreamOrder`, `chatToolMessages`, `chatStreamSegments`, cancel timer) plus two extra fields (`chatStream`, `chatStreamStartedAt`). `resetToolStream` was explicitly designed to own this cleanup, so future changes to either location can easily fall out of sync.
Consider exposing a single helper that clears all streaming state — or simply call `resetToolStream` here and then separately null out the chat stream fields:
```ts
// After successfully receiving history:
resetToolStream(state as unknown as Parameters<typeof resetToolStream>[0]);
state.chatStream = null;
state.chatStreamStartedAt = null;
```
This removes the duplicated `as unknown as` cast and keeps the cleanup in one place.
How can I resolve this? If you propose a fix, please make it concise.- Use immutable spread for chatStreamSegments instead of in-place push to ensure Lit reactivity without relying on sibling assignment - Call resetToolStream() from loadChatHistory instead of duplicating its internals with manual casting - Move hadToolEvents history reload to common path so it covers both the refreshSessionsAfterChat branch and the early-return branch
Land #39104 by @jakepresent. Co-authored-by: Jake Present <jakepresent@microsoft.com>
|
Landed via maintainer flow. What I did:
SHAs:
Thanks @jakepresent for the contribution. |
Land openclaw#39104 by @jakepresent. Co-authored-by: Jake Present <jakepresent@microsoft.com>
Land openclaw#39104 by @jakepresent. Co-authored-by: Jake Present <jakepresent@microsoft.com>
Land openclaw#39104 by @jakepresent. Co-authored-by: Jake Present <jakepresent@microsoft.com>
Land openclaw#39104 by @jakepresent. Co-authored-by: Jake Present <jakepresent@microsoft.com>
Land openclaw#39104 by @jakepresent. Co-authored-by: Jake Present <jakepresent@microsoft.com>
Land openclaw#39104 by @jakepresent. Co-authored-by: Jake Present <jakepresent@microsoft.com>
Land openclaw#39104 by @jakepresent. Co-authored-by: Jake Present <jakepresent@microsoft.com> (cherry picked from commit de2ccff)
) Land openclaw#39104 by @jakepresent. (cherry picked from commit de2ccff) Co-authored-by: Peter Steinberger <steipete@gmail.com> Co-authored-by: Jake Present <jakepresent@microsoft.com>
Land openclaw#39104 by @jakepresent. Co-authored-by: Jake Present <jakepresent@microsoft.com>
Land openclaw#39104 by @jakepresent. Co-authored-by: Jake Present <jakepresent@microsoft.com>
Land openclaw#39104 by @jakepresent. Co-authored-by: Jake Present <jakepresent@microsoft.com>
Land openclaw#39104 by @jakepresent. Co-authored-by: Jake Present <jakepresent@microsoft.com>
Summary
Tool calls (
web_search,exec, etc.) in the Control UI webchat were invisible during streaming and only appeared after a manual page refresh. This PR fixes six independent issues that combined to cause the problem.Fixes #38888 · Supersedes #39036
Root Causes & Fixes
1. Missing
tool-eventscapability (gateway.ts)Control UI connected with
caps: []so the gateway never registered it as a tool event recipient.Fix: Register
tool-eventsin the hello handshake.2. runId mismatch rejecting tool events (
app-tool-stream.ts)handleAgentEventfiltered tool events bychatRunId, but the client generates a UUID viagenerateUUID()that never matches the server's engine runId.Fix: Filter by
sessionKeyonly.3. Tool cards gated behind
showThinking(views/chat.ts)Tool messages only rendered inside
if (props.showThinking), which controls model reasoning tokens. Users with it off saw nothing.Fix: Render tool cards unconditionally, interleaved with stream segments.
4. History not reloaded after tool runs (
app-gateway.ts)On run completion,
resetToolStreamcleared tool cards butshouldReloadHistoryForFinalEventreturnedfalsefor normal messages, so they were never repopulated.Fix: Check for tool events before reset; reload history when tools were seen. Return a flag to prevent a duplicate reload from the existing
shouldReloadHistoryForFinalEventpath.5. No mid-run history reload on tool results (
app-gateway.ts)The server persists tool results as they happen, but the UI never fetched them during a run.
Fix: Reload history on each tool
resultevent so persisted text replaces streaming fragments.6. Stale streaming state after history reload (
controllers/chat.ts)History reload left
chatToolMessages,toolStreamById,toolStreamOrder,chatStreamSegments,chatStream, and the sync timer intact. The 80ms sync timer rebuiltchatToolMessagesfrom stale data, causing duplicate tool cards.Fix: Fully reset all streaming state on history load.
Additional Improvements
app-tool-stream.ts,views/chat.ts): When a toolstartevent arrives, snapshotchatStreaminto a segment and null outchatStream. Rendering interleaves segments and tool cards by index so text appears above its corresponding tool card, not below.app-tool-stream.ts): Prevents flushing from empty state.Known Limitation
Brief text fragments may appear in stream segments when a tool event arrives before the server's 150ms delta throttle flushes pending text. These self-correct within ~1s when the tool result triggers a history reload. A server-side fix would eliminate this — tracked in #39082.
Changes
gateway.tstool-eventscapabilityapp-tool-stream.tsapp-gateway.tscontrollers/chat.tsviews/chat.tsstreamSegmentsprop, interleaved rendering outsideshowThinkinggateapp-render.tsstreamSegmentsto chat propsapp.tschatStreamSegmentsstateapp-tool-stream.node.test.tsTesting
Tested manually on Windows with OpenClaw 2026.3.2:
showThinkingoff — tool cards visible, reasoning hiddenshowThinkingon — tool cards + raw JSON + reasoning visibleRelated