Skip to content

fix(ui): live tool call streaming in Control UI webchat#39104

Closed
jakepresent wants to merge 4 commits into
openclaw:mainfrom
jakepresent:fix/webchat-tool-streaming
Closed

fix(ui): live tool call streaming in Control UI webchat#39104
jakepresent wants to merge 4 commits into
openclaw:mainfrom
jakepresent:fix/webchat-tool-streaming

Conversation

@jakepresent

Copy link
Copy Markdown
Contributor

Summary

Tool calls (web_search, exec, etc.) in the Control UI webchat were invisible during streaming and only appeared after a manual page refresh. This PR fixes six independent issues that combined to cause the problem.

Fixes #38888 · Supersedes #39036

Root Causes & Fixes

1. Missing tool-events capability (gateway.ts)

Control UI connected with caps: [] so the gateway never registered it as a tool event recipient.

Fix: Register tool-events in the hello handshake.

2. runId mismatch rejecting tool events (app-tool-stream.ts)

handleAgentEvent filtered tool events by chatRunId, but the client generates a UUID via generateUUID() that never matches the server's engine runId.

Fix: Filter by sessionKey only.

3. Tool cards gated behind showThinking (views/chat.ts)

Tool messages only rendered inside if (props.showThinking), which controls model reasoning tokens. Users with it off saw nothing.

Fix: Render tool cards unconditionally, interleaved with stream segments.

4. History not reloaded after tool runs (app-gateway.ts)

On run completion, resetToolStream cleared tool cards but shouldReloadHistoryForFinalEvent returned false for normal messages, so they were never repopulated.

Fix: Check for tool events before reset; reload history when tools were seen. Return a flag to prevent a duplicate reload from the existing shouldReloadHistoryForFinalEvent path.

5. No mid-run history reload on tool results (app-gateway.ts)

The server persists tool results as they happen, but the UI never fetched them during a run.

Fix: Reload history on each tool result event so persisted text replaces streaming fragments.

6. Stale streaming state after history reload (controllers/chat.ts)

History reload left chatToolMessages, toolStreamById, toolStreamOrder, chatStreamSegments, chatStream, and the sync timer intact. The 80ms sync timer rebuilt chatToolMessages from stale data, causing duplicate tool cards.

Fix: Fully reset all streaming state on history load.

Additional Improvements

  • Stream segment ordering (app-tool-stream.ts, views/chat.ts): When a tool start event arrives, snapshot chatStream into a segment and null out chatStream. Rendering interleaves segments and tool cards by index so text appears above its corresponding tool card, not below.
  • Cancel sync timer on reset (app-tool-stream.ts): Prevents flushing from empty state.

Known Limitation

Brief text fragments may appear in stream segments when a tool event arrives before the server's 150ms delta throttle flushes pending text. These self-correct within ~1s when the tool result triggers a history reload. A server-side fix would eliminate this — tracked in #39082.

Changes

File Lines Change
gateway.ts +1 -1 Register tool-events capability
app-tool-stream.ts +25 -4 Session-only filtering, stream segment commit on tool start, cancel sync timer in reset, clear segments in reset
app-gateway.ts +32 -6 Tool-result history reload, terminal event reload with tool detection, double-reload guard
controllers/chat.ts +21 Full streaming state reset on history load
views/chat.ts +18 -2 streamSegments prop, interleaved rendering outside showThinking gate
app-render.ts +1 Pass streamSegments to chat props
app.ts +1 Init chatStreamSegments state
app-tool-stream.node.test.ts +3 Add new fields to test mock

Testing

Tested manually on Windows with OpenClaw 2026.3.2:

  • Single and multiple sequential tool calls — cards render live with correct text ordering
  • History reload on tool result — persisted text replaces streaming fragments
  • showThinking off — tool cards visible, reasoning hidden
  • showThinking on — tool cards + raw JSON + reasoning visible
  • No tool calls — no behavior change
  • Run completion — streaming state fully cleared, history repopulated

Related

Tool calls (web_search, exec, etc.) were invisible during streaming in the
Control UI webchat and only appeared after a manual page refresh. Six
independent issues combined to cause this:

1. Control UI connected with caps: [] so the gateway never sent tool events
   to it. Fix: register the tool-events capability. (gateway.ts)

2. handleAgentEvent rejected all tool events because the client-generated
   chatRunId (UUID) never matched the server's engine runId on incoming
   events. Fix: filter tool events by sessionKey only. (app-tool-stream.ts)

3. Tool cards were gated behind the showThinking toggle, which controls
   model reasoning tokens. Users with showThinking off saw no tool output.
   Fix: render tool cards unconditionally via interleaved stream segments
   and tool messages. (views/chat.ts)

4. On run completion, resetToolStream cleared tool cards but history was
   not reloaded to repopulate them from persisted state. Fix: detect when
   tool events were seen during the run and reload history before the
   streaming state is lost. Return a flag to prevent a duplicate reload
   from the existing shouldReloadHistoryForFinalEvent path. (app-gateway.ts)

5. No mid-run history refresh after tool results. The server persists tool
   results as they happen, but the UI never fetched them during a run.
   Fix: reload history on each tool result event so persisted text and
   tool output replace streaming fragments. (app-gateway.ts)

6. History reload left stale streaming state (chatToolMessages, stream
   segments, chatStream, sync timer, toolStreamById/Order) that caused
   duplicate tool cards and ghost text. Fix: fully reset all streaming
   state on history load. (controllers/chat.ts)

Additional improvements:
- Commit in-progress chatStream text as a segment on tool start so text
  renders above the tool card instead of below it. (app-tool-stream.ts)
- Interleave stream segments and tool messages by index for correct visual
  ordering during streaming. (views/chat.ts)
- Cancel pending sync timer in resetToolStream instead of flushing from
  empty state. (app-tool-stream.ts)

Note: brief text fragments may appear in stream segments when a tool event
arrives before the server's 150ms delta throttle flushes pending text.
These self-correct within ~1s when the tool result triggers a history
reload. A server-side fix (flushing the delta buffer before emitting tool
events) would eliminate this entirely — tracked in openclaw#39082.

Fixes openclaw#38888
@greptile-apps

greptile-apps Bot commented Mar 7, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes six independent issues that together made tool calls (web_search, exec, etc.) invisible during streaming in the Control UI webchat. The fixes cover the full lifecycle: capability registration at connection (gateway.ts), event routing (app-tool-stream.ts), stream segment ordering and history reload on tool results (app-gateway.ts), stale-state cleanup on history load (controllers/chat.ts), and unconditional tool card rendering (views/chat.ts).

Key points:

  • Capability registration (gateway.ts): caps: ["tool-events"] is now sent in the hello handshake — the simplest and most critical fix.
  • Session-only filtering (app-tool-stream.ts): Removes the chatRunId guard that was always rejecting tool events (client UUID vs. server runId). Events without a sessionKey are now accepted unconditionally for the current session, which is intentionally broader.
  • hadToolEvents reload guard (app-gateway.ts): Post-completion history reload is triggered when tools were seen, but the guard only applies in the !refreshSessionsAfterChat.has(runId) branch — the sibling branch for "refresh sessions" runs doesn't receive the same treatment (see inline comment).
  • Streaming state cleanup in loadChatHistory (controllers/chat.ts): Clearing is duplicated from resetToolStream internals via as unknown as casting rather than calling the existing helper; this creates a maintenance risk if the two paths diverge.
  • chatStreamSegments.push() reactivity (app-tool-stream.ts): The segment commit uses an in-place push on a Lit @state array, which only triggers a render because chatStream = null follows immediately in the same block — a fragile coupling worth addressing.

Confidence Score: 3/5

  • The PR correctly fixes the core visibility issues but has a logic gap in the terminal-event reload path and fragile Lit reactivity coupling that should be addressed before merging to reduce regression risk.
  • Score reflects: (1) six clearly-identified root causes are addressed with targeted fixes; (2) manual testing was performed across multiple scenarios; (3) however, the hadToolEvents reload guard doesn't cover the refreshSessionsAfterChat path, meaning tool-heavy runs that opt into session-refresh may not get a final history reload in some edge cases; (4) the chatStreamSegments.push() pattern implicitly relies on a sibling assignment to trigger Lit's re-render scheduler — safe today but fragile under refactoring; (5) automated test coverage for the new behaviour (segment commits, relaxed session filter, timer cancellation, post-tool history reload) is absent.
  • ui/src/ui/app-gateway.ts (incomplete hadToolEvents guard in handleTerminalChatEvent) and ui/src/ui/controllers/chat.ts (duplicated cleanup logic vs. resetToolStream).

Last reviewed commit: 93cc0d5

Comment on lines +440 to +444
if (host.chatStream && host.chatStream.trim().length > 0) {
host.chatStreamSegments.push({ text: host.chatStream, ts: now });
host.chatStream = null;
host.chatStreamStartedAt = null;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mutating @state array without reassignment

chatStreamSegments.push(...) mutates the array in-place without changing the reference. In Lit, @state-decorated properties only schedule re-renders when the property is assigned (the setter is called). The segment is only visible here because the immediately following host.chatStream = null assignment happens to trigger a re-render that reads the mutated array.

This is a silent coupling: if the chatStream nullification is ever moved, removed, or guarded separately, the pushed segment will be invisible until the next unrelated state change. The safe pattern for Lit is to replace the array:

Suggested change
if (host.chatStream && host.chatStream.trim().length > 0) {
host.chatStreamSegments.push({ text: host.chatStream, ts: now });
host.chatStream = null;
host.chatStreamStartedAt = null;
}
if (host.chatStream && host.chatStream.trim().length > 0) {
host.chatStreamSegments = [...host.chatStreamSegments, { text: host.chatStream, ts: now }];
host.chatStream = null;
host.chatStreamStartedAt = null;
}
Prompt To Fix With AI
This is a comment left during a code review.
Path: ui/src/ui/app-tool-stream.ts
Line: 440-444

Comment:
**Mutating `@state` array without reassignment**

`chatStreamSegments.push(...)` mutates the array in-place without changing the reference. In Lit, `@state`-decorated properties only schedule re-renders when the property is **assigned** (the setter is called). The segment is only visible here because the immediately following `host.chatStream = null` assignment happens to trigger a re-render that reads the mutated array.

This is a silent coupling: if the `chatStream` nullification is ever moved, removed, or guarded separately, the pushed segment will be invisible until the next unrelated state change. The safe pattern for Lit is to replace the array:

```suggestion
    if (host.chatStream && host.chatStream.trim().length > 0) {
      host.chatStreamSegments = [...host.chatStreamSegments, { text: host.chatStream, ts: now }];
      host.chatStream = null;
      host.chatStreamStartedAt = null;
    }
```

How can I resolve this? If you propose a fix, please make it concise.

Comment thread ui/src/ui/app-gateway.ts Outdated
Comment on lines 271 to 279
if (!runId || !host.refreshSessionsAfterChat.has(runId)) {
return;
// Even without a tracked runId, reload history when tools were used so
// the persisted tool results replace the now-cleared streaming state.
if (hadToolEvents && state === "final") {
void loadChatHistory(host as unknown as OpenClawApp);
return true;
}
return false;
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hadToolEvents reload guard skips the refreshSessionsAfterChat path

The fix for "history not reloaded after tool runs" (fix #4) only applies when !runId || !host.refreshSessionsAfterChat.has(runId). When a run's server runId IS in refreshSessionsAfterChat and tools were used, execution falls through to the host.refreshSessionsAfterChat.delete(runId) block, which calls loadSessions and returns false. Back in handleChatGatewayEvent, shouldReloadHistoryForFinalEvent returns false for a normal assistant reply (role: "assistant"), so loadChatHistory is never called for that path.

For the common case (!refreshSessionsAfterChat.has(runId)) this is fine because the mid-run loadChatHistory calls on each tool result keep history current, and handleChatEvent appends the final message directly. However, if refreshSessionsAfterChat ends up containing the run ID of a chat that also used tools, the cleared tool-stream state won't be backed by a fresh history fetch. Consider applying the same guard here:

host.refreshSessionsAfterChat.delete(runId);
if (state === "final") {
  void loadSessions(host as unknown as OpenClawApp, {
    activeMinutes: CHAT_SESSIONS_ACTIVE_MINUTES,
  });
  // Also reload history if tools were seen, just like the untracked-run path.
  if (hadToolEvents) {
    void loadChatHistory(host as unknown as OpenClawApp);
    return true;
  }
}
return false;
Prompt To Fix With AI
This is a comment left during a code review.
Path: ui/src/ui/app-gateway.ts
Line: 271-279

Comment:
**`hadToolEvents` reload guard skips the `refreshSessionsAfterChat` path**

The fix for "history not reloaded after tool runs" (fix #4) only applies when `!runId || !host.refreshSessionsAfterChat.has(runId)`. When a run's server `runId` IS in `refreshSessionsAfterChat` and tools were used, execution falls through to the `host.refreshSessionsAfterChat.delete(runId)` block, which calls `loadSessions` and returns `false`. Back in `handleChatGatewayEvent`, `shouldReloadHistoryForFinalEvent` returns `false` for a normal assistant reply (`role: "assistant"`), so `loadChatHistory` is never called for that path.

For the common case (`!refreshSessionsAfterChat.has(runId)`) this is fine because the mid-run `loadChatHistory` calls on each tool `result` keep history current, and `handleChatEvent` appends the final message directly. However, if `refreshSessionsAfterChat` ends up containing the run ID of a chat that also used tools, the cleared tool-stream state won't be backed by a fresh history fetch. Consider applying the same guard here:

```ts
host.refreshSessionsAfterChat.delete(runId);
if (state === "final") {
  void loadSessions(host as unknown as OpenClawApp, {
    activeMinutes: CHAT_SESSIONS_ACTIVE_MINUTES,
  });
  // Also reload history if tools were seen, just like the untracked-run path.
  if (hadToolEvents) {
    void loadChatHistory(host as unknown as OpenClawApp);
    return true;
  }
}
return false;
```

How can I resolve this? If you propose a fix, please make it concise.

Comment thread ui/src/ui/controllers/chat.ts Outdated
Comment on lines +74 to +90
const toolHost = state as unknown as {
chatToolMessages: unknown[];
chatStreamSegments: Array<{ text: string; ts: number }>;
toolStreamById: Map<string, unknown>;
toolStreamOrder: string[];
toolStreamSyncTimer: number | null;
};
toolHost.chatToolMessages = [];
toolHost.chatStreamSegments = [];
toolHost.toolStreamById.clear();
toolHost.toolStreamOrder = [];
if (toolHost.toolStreamSyncTimer != null) {
clearTimeout(toolHost.toolStreamSyncTimer);
toolHost.toolStreamSyncTimer = null;
}
state.chatStream = null;
state.chatStreamStartedAt = null;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Duplicated streaming-state cleanup across loadChatHistory and resetToolStream

loadChatHistory now manually replicates the internals of resetToolStream (clear toolStreamById, toolStreamOrder, chatToolMessages, chatStreamSegments, cancel timer) plus two extra fields (chatStream, chatStreamStartedAt). resetToolStream was explicitly designed to own this cleanup, so future changes to either location can easily fall out of sync.

Consider exposing a single helper that clears all streaming state — or simply call resetToolStream here and then separately null out the chat stream fields:

// After successfully receiving history:
resetToolStream(state as unknown as Parameters<typeof resetToolStream>[0]);
state.chatStream = null;
state.chatStreamStartedAt = null;

This removes the duplicated as unknown as cast and keeps the cleanup in one place.

Prompt To Fix With AI
This is a comment left during a code review.
Path: ui/src/ui/controllers/chat.ts
Line: 74-90

Comment:
**Duplicated streaming-state cleanup across `loadChatHistory` and `resetToolStream`**

`loadChatHistory` now manually replicates the internals of `resetToolStream` (clear `toolStreamById`, `toolStreamOrder`, `chatToolMessages`, `chatStreamSegments`, cancel timer) plus two extra fields (`chatStream`, `chatStreamStartedAt`). `resetToolStream` was explicitly designed to own this cleanup, so future changes to either location can easily fall out of sync.

Consider exposing a single helper that clears all streaming state — or simply call `resetToolStream` here and then separately null out the chat stream fields:

```ts
// After successfully receiving history:
resetToolStream(state as unknown as Parameters<typeof resetToolStream>[0]);
state.chatStream = null;
state.chatStreamStartedAt = null;
```

This removes the duplicated `as unknown as` cast and keeps the cleanup in one place.

How can I resolve this? If you propose a fix, please make it concise.

- Use immutable spread for chatStreamSegments instead of in-place push
  to ensure Lit reactivity without relying on sibling assignment
- Call resetToolStream() from loadChatHistory instead of duplicating
  its internals with manual casting
- Move hadToolEvents history reload to common path so it covers both
  the refreshSessionsAfterChat branch and the early-return branch
steipete added a commit that referenced this pull request Mar 7, 2026
Land #39104 by @jakepresent.

Co-authored-by: Jake Present <jakepresent@microsoft.com>
@steipete

steipete commented Mar 7, 2026

Copy link
Copy Markdown
Contributor

Landed via maintainer flow.

What I did:

  • Rebased onto latest main via temp branch land-pr-39104.
  • Applied/fixed PR changes plus a follow-up regression fix in loadChatHistory so tool-stream reset only runs when tool-stream host fields exist.
  • Added changelog entry crediting @jakepresent and fix(ui): live tool call streaming in Control UI webchat #39104.
  • Ran full gate before commit:
    • pnpm lint
    • pnpm build
    • pnpm test
  • Committed with Conventional Commit + co-author trailer.

SHAs:

  • Original PR head: 639a98d0bfc91f2020d46af2462ce078b86bbe86
  • Landed on main: de2ccffec16610bad7884dbbd6c4942657e84804

Thanks @jakepresent for the contribution.

@steipete steipete closed this Mar 7, 2026
@jakepresent jakepresent deleted the fix/webchat-tool-streaming branch March 7, 2026 19:31
vincentkoc pushed a commit to BryanTegomoh/openclaw-contrib that referenced this pull request Mar 8, 2026
Land openclaw#39104 by @jakepresent.

Co-authored-by: Jake Present <jakepresent@microsoft.com>
openperf pushed a commit to openperf/moltbot that referenced this pull request Mar 8, 2026
Land openclaw#39104 by @jakepresent.

Co-authored-by: Jake Present <jakepresent@microsoft.com>
mcaxtr pushed a commit to mcaxtr/openclaw that referenced this pull request Mar 8, 2026
Land openclaw#39104 by @jakepresent.

Co-authored-by: Jake Present <jakepresent@microsoft.com>
GordonSH-oss pushed a commit to GordonSH-oss/openclaw that referenced this pull request Mar 9, 2026
Land openclaw#39104 by @jakepresent.

Co-authored-by: Jake Present <jakepresent@microsoft.com>
jenawant pushed a commit to jenawant/openclaw that referenced this pull request Mar 10, 2026
Land openclaw#39104 by @jakepresent.

Co-authored-by: Jake Present <jakepresent@microsoft.com>
V-Gutierrez pushed a commit to V-Gutierrez/openclaw-vendor that referenced this pull request Mar 17, 2026
Land openclaw#39104 by @jakepresent.

Co-authored-by: Jake Present <jakepresent@microsoft.com>
alexey-pelykh pushed a commit to remoteclaw/remoteclaw that referenced this pull request Mar 21, 2026
Land openclaw#39104 by @jakepresent.

Co-authored-by: Jake Present <jakepresent@microsoft.com>
(cherry picked from commit de2ccff)
alexey-pelykh added a commit to remoteclaw/remoteclaw that referenced this pull request Mar 21, 2026
)

Land openclaw#39104 by @jakepresent.


(cherry picked from commit de2ccff)

Co-authored-by: Peter Steinberger <steipete@gmail.com>
Co-authored-by: Jake Present <jakepresent@microsoft.com>
lovewanwan pushed a commit to lovewanwan/openclaw that referenced this pull request Apr 28, 2026
Land openclaw#39104 by @jakepresent.

Co-authored-by: Jake Present <jakepresent@microsoft.com>
ogt-redknie pushed a commit to ogt-redknie/OPENX that referenced this pull request May 2, 2026
Land openclaw#39104 by @jakepresent.

Co-authored-by: Jake Present <jakepresent@microsoft.com>
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 9, 2026
Land openclaw#39104 by @jakepresent.

Co-authored-by: Jake Present <jakepresent@microsoft.com>
github-actions Bot pushed a commit to Desicool/openclaw that referenced this pull request May 24, 2026
Land openclaw#39104 by @jakepresent.

Co-authored-by: Jake Present <jakepresent@microsoft.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] WebChat/Control UI: WebSocket messages not pushed to frontend, requires refresh to see output

2 participants