Skip to content

WhatsApp: send commentary live#57279

Closed
tristanmanchester wants to merge 1 commit into
openclaw:mainfrom
tristanmanchester:codex/whatsapp-commentary-v2026-3-28
Closed

WhatsApp: send commentary live#57279
tristanmanchester wants to merge 1 commit into
openclaw:mainfrom
tristanmanchester:codex/whatsapp-commentary-v2026-3-28

Conversation

@tristanmanchester

Copy link
Copy Markdown
Contributor

Summary

  • Problem: WhatsApp currently accumulates assistant commentary and returns it together with the final answer, which makes live progress updates unusable in chat.
  • Why it matters: users who opt into live commentary want incremental WhatsApp updates during the run, without changing behavior for every other channel.
  • What changed: added opt-in channels.whatsapp.commentaryDelivery: "off" | "live", threaded a channel-agnostic onCommentaryReply seam through the embedded runner, and delivered finalized commentary segments live through the existing WhatsApp send path.
  • What did NOT change (scope boundary): no cross-channel rollout, no generic commentary dispatcher kind, and no change to default WhatsApp behavior when the new config is unset or off.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

Root Cause / Regression History (if applicable)

  • Root cause: N/A
  • Missing detection / guardrail: N/A
  • Prior context (git blame, prior PR, issue, or refactor if known): clean replacement for WhatsApp: send commentary updates live #43731 after the earlier branch became too stale relative to current review expectations.
  • Why this regressed now: N/A
  • If unknown, what was ruled out: N/A

Regression Test Plan (if applicable)

  • Coverage level that should have caught this:
    • Unit test
    • Seam / integration test
    • End-to-end test
    • Existing coverage already sufficient
  • Target test or file: src/agents/pi-embedded-subscribe.commentary.test.ts, src/agents/pi-embedded-runner/run/payloads.test.ts, extensions/whatsapp/src/auto-reply/deliver-reply.test.ts, extensions/whatsapp/src/auto-reply/monitor/process-message.inbound-context.test.ts, extensions/telegram/src/bot-message-dispatch.test.ts
  • Scenario the test should lock in: live commentary emits once for finalized commentary segments, delivered commentary is filtered out of the final payload, WhatsApp send normalization/suppression behaves correctly, and non-WhatsApp channels remain unwired.
  • Why this is the smallest reliable guardrail: the feature spans subscribe lifecycle, final payload assembly, and WhatsApp delivery, so narrow seam coverage at those boundaries catches the real failure modes without requiring full end-to-end infrastructure.
  • Existing test that already covers this (if any): the Telegram boundary check covers the non-rollout scope.
  • If no new test is added, why not: N/A

User-visible / Behavior Changes

  • WhatsApp keeps its existing default behavior when channels.whatsapp.commentaryDelivery is unset or off.
  • With channels.whatsapp.commentaryDelivery: "live", assistant commentary is delivered to WhatsApp in real time as separate messages.
  • The final WhatsApp reply still arrives separately.
  • Commentary already delivered live is removed from the terminal payload so it does not get repeated with the final answer.

Diagram (if applicable)

Before:
[user message] -> [assistant commentary accumulates internally] -> [final WhatsApp reply includes commentary + final answer]

After:
[user message] -> [commentary segment finalizes] -> [live WhatsApp commentary message]
[user message] -> [assistant final answer] -> [final WhatsApp reply only]

Security Impact (required)

  • New permissions/capabilities? (No)
  • Secrets/tokens handling changed? (No)
  • New/changed network calls? (No)
  • Command/tool execution surface changed? (No)
  • Data access scope changed? (No)
  • If any Yes, explain risk + mitigation: N/A

Repro + Verification

Environment

  • OS: macOS
  • Runtime/container: repo-run OpenClaw gateway
  • Model/provider: Pi embedded runner with WhatsApp channel enabled
  • Integration/channel (if any): WhatsApp
  • Relevant config (redacted): channels.whatsapp.commentaryDelivery = "live"

Steps

  1. Configure WhatsApp with channels.whatsapp.commentaryDelivery = "live".
  2. Send a prompt that causes the assistant to emit commentary before the final answer.
  3. Observe the WhatsApp thread while the run is still in progress.

Expected

  • Commentary messages appear live during the run.
  • The final reply is delivered separately.
  • Previously delivered commentary is not repeated in the final reply.

Actual

  • Matches expected behavior in manual WhatsApp verification.

Evidence

Attach at least one:

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios: user manually verified live commentary on WhatsApp; I verified the stable repo-run gateway starts cleanly with the feature branch, and I ran targeted commentary/WhatsApp/Telegram regressions locally.
  • Edge cases checked: default-off behavior remains unchanged, silent commentary with no visible content is suppressed, media-only commentary remains deliverable, and delivered commentary is filtered out of the final payload.
  • What you did not verify: other channels beyond the Telegram regression boundary test, and current main startup behavior unrelated to this feature.

Review Conversations

  • I replied to or resolved every bot review conversation I addressed in this PR.
  • I left unresolved only the conversations that still need reviewer or maintainer judgment.

If a bot review conversation is addressed by this PR, resolve that conversation yourself. Do not leave bot review conversation cleanup for maintainers.

Compatibility / Migration

  • Backward compatible? (Yes)
  • Config/env changes? (Yes)
  • Migration needed? (No)
  • If yes, exact upgrade steps: set channels.whatsapp.commentaryDelivery = "live" to opt in.

Risks and Mitigations

  • Risk: duplicate or missing commentary delivery if commentary segment bookkeeping drifts.
    • Mitigation: commentary ownership stays inside subscribeEmbeddedPiSession() with stable segment IDs and targeted seam coverage.
  • Risk: directive or silent-message handling diverges from normal WhatsApp replies.
    • Mitigation: commentary send reuses the existing WhatsApp normalization path and dedicated delivery tests.

@openclaw-barnacle openclaw-barnacle Bot added channel: telegram Channel integration: telegram channel: whatsapp-web Channel integration: whatsapp-web agents Agent runtime and tooling size: XL labels Mar 29, 2026

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 82d9235a28

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

};
const deliveredCommentarySegmentIds = new Set(params.deliveredCommentarySegmentIds ?? []);
const resolvedAssistantTexts = (() => {
if (params.assistantOutputs) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Fall back to assistantTexts when outputs are missing

buildEmbeddedRunPayloads() now treats any defined assistantOutputs as authoritative, but runEmbeddedPiAgent() always passes this field (src/agents/pi-embedded-runner/run.ts:1253-1256), including the empty-array case. Because an empty array is truthy, the fallback to assistantTexts is skipped, so turns that stream text but never reach message_end (for example timeout/abort races where assistantOutputs is never populated) can lose all user-facing reply text and surface as empty/incomplete payloads instead of returning the captured stream content.

Useful? React with 👍 / 👎.

@greptile-apps

greptile-apps Bot commented Mar 29, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR adds opt-in live commentary delivery for WhatsApp (channels.whatsapp.commentaryDelivery: "live"), threading a new onCommentaryReply seam through the embedded runner and subscribe layer all the way to the existing deliverWebReply path. The default behaviour (commentary batched into the final reply) is unchanged.

Key changes:

  • pi-embedded-commentary.ts — new file that extracts structured AssistantOutputEntry segments (with stable IDs and commentary/final_answer phases) from assistant messages
  • pi-embedded-subscribe.ts — new commentary delivery queue: segment-level AbortControllers, generation tracking for compaction-retry safety, and waitForCommentaryDelivery draining logic
  • pi-embedded-subscribe.handlers.messages.ts — non-terminal commentary segments are queued live on message_update; remaining undelivered segments are flushed on message_end
  • payloads.ts — commentary segments already delivered live are filtered out of the terminal payload to prevent duplication
  • deliver-reply.ts — refactored to inline media send logic and adds abortSignal/timeoutMs support so hung sends can be interrupted
  • process-message.ts — wires onCommentaryReply when commentaryDelivery === "live", normalises directives, and fixes the didSendReply return value

Issues found:

  • createCommentaryAbortError mutates the reason error's name property in-place, causing isRunnerAbortError to return true for any Error, silently swallowing unexpected errors from the commentary wait path.
  • fallbackAnswerText in buildEmbeddedRunPayloads is now dead code since params.assistantOutputs is always defined.
  • Per-segment upstreamAbort listeners accumulate on params.abortSignal for the run lifetime.

Confidence Score: 4/5

Safe to merge for the opt-in path; one P1 error-swallowing bug in the commentary wait loop should be addressed first.

The feature is well-scoped and well-tested. The mutation bug in createCommentaryAbortError is a real P1: unexpected errors from waitForCommentaryDeliveryRound will have their name silently overwritten to AbortError, causing them to match isRunnerAbortError and be swallowed rather than re-thrown. The blast radius is narrow but it will make commentary-related incidents invisible in production. The other two findings are P2 and do not block merge.

src/agents/pi-embedded-subscribe.ts (createCommentaryAbortError mutation); src/agents/pi-embedded-runner/run/payloads.ts (dead fallbackAnswerText branch)

Comments Outside Diff (1)

  1. src/agents/pi-embedded-runner/run/payloads.ts, line 1409-1425 (link)

    P2 fallbackAnswerText is now unreachable dead code

    params.assistantOutputs is always defined when called from the new code path — subscribeEmbeddedPiSession initialises state.assistantOutputs as [] and the attempt result always passes it through — so the if (params.assistantOutputs) branch is always taken. The fallbackAnswerText / params.assistantTexts fallback can never be reached.

    If the current behaviour is intentional, the dead branch should be removed with an explanatory comment. If unintentional, fallbackAnswerText should be consulted when resolvedAssistantTexts resolves to an empty array.

    Prompt To Fix With AI
    This is a comment left during a code review.
    Path: src/agents/pi-embedded-runner/run/payloads.ts
    Line: 1409-1425
    
    Comment:
    **`fallbackAnswerText` is now unreachable dead code**
    
    `params.assistantOutputs` is always defined when called from the new code path — `subscribeEmbeddedPiSession` initialises `state.assistantOutputs` as `[]` and the attempt result always passes it through — so the `if (params.assistantOutputs)` branch is always taken. The `fallbackAnswerText` / `params.assistantTexts` fallback can never be reached.
    
    If the current behaviour is intentional, the dead branch should be removed with an explanatory comment. If unintentional, `fallbackAnswerText` should be consulted when `resolvedAssistantTexts` resolves to an empty array.
    
    How can I resolve this? If you propose a fix, please make it concise.
Prompt To Fix All With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-subscribe.ts
Line: 627-635

Comment:
**`createCommentaryAbortError` mutates the caller's error object**

When `reason instanceof Error`, this function sets `reason.name = "AbortError"` in-place and returns the same object. This mutates the original error before the caller checks it. In `waitForCommentaryDeliveryBounded` inside `attempt.ts`, the pattern is:

```ts
} catch (err) {
  abortCommentaryDelivery(err);   // <-- mutates err.name = "AbortError" if err instanceof Error
  if (err === timeoutError) { … }
  if (isRunnerAbortError(err)) { … return; }  // <-- always true now for any Error
  throw err;                      // <-- never reached for Error instances
}
```

Because `isRunnerAbortError` checks `err.name === "AbortError"`, any unexpected Error thrown from `waitForCommentaryDeliveryRound()` will have its name quietly rewritten and be silently swallowed with `log.debug` instead of being re-thrown. The fix is to wrap the reason in a new error rather than mutating the original:

```ts
const createCommentaryAbortError = (message: string, reason?: unknown): Error => {
  const error = new Error(message);
  error.name = "AbortError";
  if (reason instanceof Error) {
    error.cause = reason;
  }
  return error;
};
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/agents/pi-embedded-runner/run/payloads.ts
Line: 1409-1425

Comment:
**`fallbackAnswerText` is now unreachable dead code**

`params.assistantOutputs` is always defined when called from the new code path — `subscribeEmbeddedPiSession` initialises `state.assistantOutputs` as `[]` and the attempt result always passes it through — so the `if (params.assistantOutputs)` branch is always taken. The `fallbackAnswerText` / `params.assistantTexts` fallback can never be reached.

If the current behaviour is intentional, the dead branch should be removed with an explanatory comment. If unintentional, `fallbackAnswerText` should be consulted when `resolvedAssistantTexts` resolves to an empty array.

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/agents/pi-embedded-subscribe.ts
Line: 660-668

Comment:
**`upstreamAbort` listener is never explicitly removed**

The listener is added with `{ once: true }` so it self-removes when the signal fires, but in normal operation commentary delivery completes before the run-level abort signal fires. In a long-running session with many commentary segments, O(N) listeners accumulate on `params.abortSignal` until run end. Consider removing it in the `.finally` block alongside the other cleanup, mirroring the pattern used in `deliverWebReply`.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "WhatsApp: send commentary live" | Re-trigger Greptile

Comment on lines +627 to +635
const createCommentaryAbortError = (message: string, reason?: unknown): Error => {
if (reason instanceof Error) {
reason.name = "AbortError";
return reason;
}
const error = new Error(message);
error.name = "AbortError";
return error;
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 createCommentaryAbortError mutates the caller's error object

When reason instanceof Error, this function sets reason.name = "AbortError" in-place and returns the same object. This mutates the original error before the caller checks it. In waitForCommentaryDeliveryBounded inside attempt.ts, the pattern is:

} catch (err) {
  abortCommentaryDelivery(err);   // <-- mutates err.name = "AbortError" if err instanceof Error
  if (err === timeoutError) {  }
  if (isRunnerAbortError(err)) {  return; }  // <-- always true now for any Error
  throw err;                      // <-- never reached for Error instances
}

Because isRunnerAbortError checks err.name === "AbortError", any unexpected Error thrown from waitForCommentaryDeliveryRound() will have its name quietly rewritten and be silently swallowed with log.debug instead of being re-thrown. The fix is to wrap the reason in a new error rather than mutating the original:

const createCommentaryAbortError = (message: string, reason?: unknown): Error => {
  const error = new Error(message);
  error.name = "AbortError";
  if (reason instanceof Error) {
    error.cause = reason;
  }
  return error;
};
Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-subscribe.ts
Line: 627-635

Comment:
**`createCommentaryAbortError` mutates the caller's error object**

When `reason instanceof Error`, this function sets `reason.name = "AbortError"` in-place and returns the same object. This mutates the original error before the caller checks it. In `waitForCommentaryDeliveryBounded` inside `attempt.ts`, the pattern is:

```ts
} catch (err) {
  abortCommentaryDelivery(err);   // <-- mutates err.name = "AbortError" if err instanceof Error
  if (err === timeoutError) { … }
  if (isRunnerAbortError(err)) { … return; }  // <-- always true now for any Error
  throw err;                      // <-- never reached for Error instances
}
```

Because `isRunnerAbortError` checks `err.name === "AbortError"`, any unexpected Error thrown from `waitForCommentaryDeliveryRound()` will have its name quietly rewritten and be silently swallowed with `log.debug` instead of being re-thrown. The fix is to wrap the reason in a new error rather than mutating the original:

```ts
const createCommentaryAbortError = (message: string, reason?: unknown): Error => {
  const error = new Error(message);
  error.name = "AbortError";
  if (reason instanceof Error) {
    error.cause = reason;
  }
  return error;
};
```

How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +660 to +668
if (params.abortSignal) {
const upstreamAbort = () => {
abortController.abort(params.abortSignal?.reason);
};
if (params.abortSignal.aborted) {
upstreamAbort();
} else {
params.abortSignal.addEventListener("abort", upstreamAbort, { once: true });
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 upstreamAbort listener is never explicitly removed

The listener is added with { once: true } so it self-removes when the signal fires, but in normal operation commentary delivery completes before the run-level abort signal fires. In a long-running session with many commentary segments, O(N) listeners accumulate on params.abortSignal until run end. Consider removing it in the .finally block alongside the other cleanup, mirroring the pattern used in deliverWebReply.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/agents/pi-embedded-subscribe.ts
Line: 660-668

Comment:
**`upstreamAbort` listener is never explicitly removed**

The listener is added with `{ once: true }` so it self-removes when the signal fires, but in normal operation commentary delivery completes before the run-level abort signal fires. In a long-running session with many commentary segments, O(N) listeners accumulate on `params.abortSignal` until run end. Consider removing it in the `.finally` block alongside the other cleanup, mirroring the pattern used in `deliverWebReply`.

How can I resolve this? If you propose a fix, please make it concise.

@tristanmanchester

Copy link
Copy Markdown
Contributor Author

Closing this for now because the implementation is intentionally based on the latest stable release line, not current main.

This feature worked as expected in manual WhatsApp verification, but we are not going to fold unrelated main regressions into this PR just to make the branch target merge cleanly. The clean source of truth remains the release-based branch/commit for this feature.

@tristanmanchester tristanmanchester deleted the codex/whatsapp-commentary-v2026-3-28 branch March 29, 2026 20:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling channel: telegram Channel integration: telegram channel: whatsapp-web Channel integration: whatsapp-web size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant