WhatsApp: send commentary updates live by tristanmanchester · Pull Request #42489 · openclaw/openclaw

tristanmanchester · 2026-03-10T20:23:07Z

Summary

Problem: On WhatsApp, GPT-5.4 commentary text was discovered during the run but usually arrived only after completion, bundled with the final reply. In practice users got the commentary messages dumped at the end instead of seeing progress live.
Why it matters: That made long-running WhatsApp tasks feel broken and defeated the point of provider-emitted commentary.
OpenAI phase context: This work relies on the newer assistant phase parameter guidance in the latest-model docs: use phase: "commentary" for interim updates and phase: "final_answer" for the terminal reply.
What changed:
- Preserved and reconciled assistant phase / signature metadata so commentary and final-answer segments can be identified reliably.
- Wired onCommentaryReply through the embedded runner and finalized-message fallback.
- Extracted shared reply-delivery infrastructure in reply-delivery.ts so directive normalization, silent suppression, and commentary/final bookkeeping are reusable instead of being inlined in the WhatsApp monitor.
- Rewired WhatsApp to consume that shared helper while preserving its existing transport and config behavior.
- Hardened the assistant-output pipeline after review: stopReason-only error rewriting, boundary-aware segment extraction, deterministic fallback ids reused across live/final reconciliation, safer compaction-boundary recovery, payload fallback when filtered assistantOutputs become empty, and websocket delta metadata that preserves phase / signature identity before response.completed.
- Added non-WhatsApp regression coverage so Telegram remains unchanged and does not wire commentary delivery by accident.
Relationship to prior phase-preservation work: The lower-level OpenAI Responses/WebSocket phase preservation landed separately in commit cced1e0f7694f8e1c98086370612f5a41e811230 (Preserve Responses API phase param, PR Preserve Responses API phase param #43475). This PR builds on that same phase contract and now also extracts the delivery-side plumbing into shared infrastructure while hardening the downstream reconciliation logic that consumes it.
What did NOT change (scope boundary): No Telegram live commentary support, no LINE/Discord rollout, no heartbeat behavior changes, and no dependency/transport redesign beyond the minimum plumbing needed to make the commentary delivery path reusable.

Change Type (select all)

Scope (select all touched areas)

Linked Issue/PR

OpenAI latest-model docs: New phase parameter
Prior phase-preservation commit: cced1e0f7694f8e1c98086370612f5a41e811230
Prior related PR: Preserve Responses API phase param #43475
Review-driven hardening follow-up is included directly in this PR branch; no separate follow-up PR.
No directly relevant public issues or prior PRs found in openclaw/openclaw after targeted search for WhatsApp commentary / partial-reply / external-chat final-only behavior.

User-visible / Behavior Changes

New WhatsApp config: channels.whatsapp.commentaryDelivery = "off" | "live".
With commentaryDelivery: "live", provider-emitted assistant commentary messages are sent as separate WhatsApp messages during the run.
Before this change, WhatsApp would typically send all commentary messages at the end together with the final reply.
Commentary directives like [[reply_to_current]] are normalized before send and no longer leak into user-facing commentary messages.
Terminal replies exclude only commentary segments that were actually delivered live.
Final replies now fall back to assistantTexts / fallbackAnswerText when all assistantOutputs were stripped as already-delivered commentary, preventing commentary-only terminal payloads.
No new live-commentary behavior was enabled for Telegram, Discord, or LINE in this PR.

Security Impact (required)

New permissions/capabilities? (Yes/No) No
Secrets/tokens handling changed? (Yes/No) No
New/changed network calls? (Yes/No) No
Command/tool execution surface changed? (Yes/No) No
Data access scope changed? (Yes/No) No
If any Yes, explain risk + mitigation:

Repro + Verification

Environment

OS: macOS
Runtime/container: local gateway via pnpm openclaw gateway run
Model/provider: openai-codex / gpt-5.4
Integration/channel (if any): WhatsApp Web
Relevant config (redacted): channels.whatsapp.commentaryDelivery=live

Steps

Start the local gateway from this branch with WhatsApp enabled and channels.whatsapp.commentaryDelivery=live.
Send a long-running WhatsApp prompt that requests three assistant commentary updates and one final answer.
Observe gateway logs and the WhatsApp thread while the run is still in progress.

Expected

Commentary messages arrive during the run.
The final answer arrives once at the end.
Commentary text is not repeated inside the final answer.

Actual

Before this PR, commentary was often discovered mid-run but flushed only after embedded run done, together with the final reply.
After this PR, commentary is sent live during the run and the final answer remains separate.

Evidence

Attach at least one:

Failing test/log before + passing after
Trace/log snippets
Screenshot/recording
Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

Verified scenarios:
- Confirmed the broken behavior from live WhatsApp logs where commentary was discovered during the run but only sent at completion.
- Retested after the fix and saw commentary send during the run at 20:54:37, 20:54:40, and 20:56:40, while the run completed later at 20:57:14.
- Confirmed the final answer still arrived once at 20:57:14.
- Confirmed [[reply_to_current]] no longer leaked into commentary messages.
- Confirmed the hardened reconciliation path does not rewrite successful replies just because errorMessage metadata is present, and it still finds finalized output after both out-of-bounds and in-range compaction cursor shifts.
Edge cases checked:
- Finalized-message fallback still delivers missed commentary.
- Cumulative streamMessage updates do not send only the first partial fragment.
- Delivered commentary is removed from the terminal payload, but undelivered commentary is not.
- Filtered-empty assistantOutputs fall back to assistantTexts instead of dropping the final answer.
- WebSocket text deltas carry enough metadata for live commentary reconciliation before the completed response arrives.
- The shared reply-delivery helper preserves current WhatsApp behavior and Telegram still does not wire commentary callbacks.
What you did not verify:
- Telegram or other channels as live-commentary senders.
- Heartbeat follow-up behavior.

Review Conversations

I replied to or resolved every bot review conversation I addressed in this PR.
I left unresolved only the conversations that still need reviewer or maintainer judgment.

If a bot review conversation is addressed by this PR, resolve that conversation yourself. Do not leave bot review conversation cleanup for maintainers.

Compatibility / Migration

Backward compatible? (Yes/No) Yes
Config/env changes? (Yes/No) Yes
Migration needed? (Yes/No) No
If yes, exact upgrade steps:

Failure Recovery (if this breaks)

How to disable/revert this change quickly: Set channels.whatsapp.commentaryDelivery=off.
Files/config to restore: Revert the shared commentary delivery helper and the WhatsApp commentary plumbing changes.
Known bad symptoms reviewers should watch for: duplicate commentary, commentary leaking directives, commentary still arriving only after embedded run done, non-WhatsApp channels unexpectedly wiring commentary callbacks, or final replies missing after commentary was already delivered live.

Risks and Mitigations

Risk: Cumulative partial commentary could send an early fragment instead of the completed segment.
- Mitigation: Live reconciliation now waits until a commentary segment is no longer the terminal growing block before sending it, and tests cover cumulative streamMessage growth.
Risk: Missed live commentary could disappear from the final answer.
- Mitigation: Final payload filtering removes only commentary segment ids that were actually delivered live, and payload construction falls back to assistantTexts / fallbackAnswerText if filtering empties assistantOutputs.
Risk: Compaction or metadata drift could cause dropped, duplicated, or rewritten assistant output.
- Mitigation: Reconciliation now uses deterministic fallback ids, stable message matching for prompt-boundary recovery, stopReason-only error rewriting, and websocket delta metadata that preserves live/final segment identity.
Risk: Refactoring the send path could accidentally broaden commentary behavior to other channels.
- Mitigation: The shared helper is infrastructure-only; WhatsApp remains the only opted-in channel, and a Telegram regression test verifies onCommentaryReply remains unwired there.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f5efe4ade8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

greptile-apps · 2026-03-10T20:43:55Z

Greptile Summary

This PR adds WhatsApp live commentary delivery for GPT-5.4's commentary-phase text blocks, fixing a user-visible issue where all progress messages arrived in a single burst at the end of long-running tasks instead of appearing incrementally. The implementation introduces a 250 ms background polling loop in runEmbeddedAttempt that reconciles both in-flight streaming messages and finalized messages, delivering commentary-phase segments to WhatsApp via a new onCommentaryReply callback wired through the embedded runner stack. Delivered segment IDs are tracked in deliveredCommentarySegmentIds so the terminal reply only includes commentary that was not already sent live. A new commentaryDelivery: "off" | "live" config option (defaulting to "off") ensures full backward compatibility.

Key findings from review:

Logic bug — missing isSilent guard in sendWhatsAppPayload (process-message.ts lines 401–418): normalizeReplyPayloadDirectives returns a { payload, isSilent } tuple, but isSilent is never checked. If a commentary (or final) payload normalizes to a silent reply, deliverWebReply is still called and didSendReply = true is incorrectly set, causing processMessage to report true (message delivered) when nothing was actually sent. A one-line guard (if (normalized.isSilent) return;) before deliverWebReply would fix this.
Style — nextStartIndex regresses to first in-flight position (assistant-output.ts): When a not-yet-finalized assistant message is encountered, nextStartIndex rewinds to that position even though finalized messages after it are already processed and guarded by seenSegmentIds. This is correct but causes O(n²) re-examination for long agent turns with many interleaved tool calls. Not urgent.
Style — reconciler loop sleeps before first poll (attempt.ts): The 250 ms sleep runs before the first reconcile pass, adding unnecessary latency for the initial commentary segment on fast providers. Moving the sleep to the end of the loop body would preserve the polling interval while delivering the first commentary without delay.
The two-set deduplication strategy (seenLiveCommentaryIds for streaming / seenAssistantOutputIds for finalized) is sound: live segments are delivered once and filtered from the terminal payload; missed segments fall back to the finalized path.
Test coverage is thorough, including cumulative-growth, fallback delivery, and directive-stripping scenarios.

Confidence Score: 3/5

Safe to merge after fixing the isSilent guard in sendWhatsAppPayload; the commentaryDelivery: "off" default makes the live path entirely opt-in so existing deployments are unaffected.
The core two-set deduplication logic is sound and well-tested. The main concern is the missing isSilent check in sendWhatsAppPayload, which could cause processMessage to return true (masking a no-op delivery) when a commentary or final payload normalizes to a silent reply — a subtle but real correctness issue. The 250 ms sleep-first poll and nextStartIndex regression are minor efficiency issues. The feature is gated behind commentaryDelivery: "live", so the blast radius of any latent bugs is small.
Pay close attention to src/web/auto-reply/monitor/process-message.ts (missing isSilent guard in sendWhatsAppPayload) and src/agents/assistant-output.ts (nextStartIndex regression pattern in reconcileAssistantOutputs).

_{Last reviewed commit: f5efe4a}

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c7fc5d726e

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e76dbb1e9d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f8a92b4143

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

This reverts commit 669999a.

This reverts commit 0e97659517b7e3c99c75f08e9a94b8ac89694a26.

…t IDs

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 334398cff7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 37719ae8e8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ea49855566

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-12T05:11:53Z

+  const errorContext =
+    messageRecord?.stopReason === "error" ||
+    (typeof messageRecord?.errorMessage === "string" &&
+      messageRecord.errorMessage.trim().length > 0);


Gate assistant error rewriting on stopReason only

extractAssistantOutputCandidates marks errorContext true whenever errorMessage is present, but non-error assistant turns can legitimately carry errorMessage metadata (see the existing guard in src/agents/pi-embedded-utils.ts:246-251). Because this commit now prefers assistantOutputs in final payload construction, normal replies that merely have background tool failures can be rewritten into generic error text, corrupting user-facing output.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-03-12T05:11:53Z

+  const normalizedStartIndex =
+    params.startIndex >= 0 && params.startIndex <= params.messages.length
+      ? params.startIndex
+      : resolveCompactedPromptBoundaryIndex(params.messages, params.historyBeforePrompt);


Rewind reconciliation cursor when boundary shifts in-range

The cursor is only rewound when startIndex is out of bounds, but compaction can shift the pre-prompt boundary left while keeping messages.length >= startIndex (for example, dropping old history while adding new turn messages). In that case reconciliation starts too far right and permanently skips finalized assistant messages between the shifted boundary and the stale cursor, which can drop terminal reply segments from assistantOutputs.

Useful? React with 👍 / 👎.

…t week (openclaw#43642) * Models: add temporary Hunter and Healer alpha to OpenRouter catalog * Add temporary OpenRouter stealth catalog entries --------- Co-authored-by: Tak Hoffman <781889+Takhoffman@users.noreply.github.com>

openclaw-barnacle · 2026-03-12T05:48:52Z