Fix heartbeat pending final delivery cleanup#87583
Conversation
|
Thanks for the context here. I swept through the related work, and this is now duplicate or superseded. Keep open, but human-only before merge. The patch targets a real heartbeat pending-final cleanup gap on current main, yet it lacks contributor real-behavior proof and overlaps the proof-positive canonical cleanup path at #83187, with an extra duplicate-suppression cleanup decision maintainers should reconcile. Canonical path: Close this PR as superseded by #83187. So I’m closing this here and keeping the remaining discussion on #83187. Review detailsBest possible solution: Close this PR as superseded by #83187. Do we have a high-confidence way to reproduce the issue? Yes at source level, but not with a live run in this read-only review. Current main shows heartbeat send success records Is this the best way to solve the issue? Unclear as the final merge path. The send-success cleanup is well aligned with the existing dispatch cleanup, but this PR also adds duplicate-suppression cleanup and overlaps a proof-positive sibling PR, so maintainers should pick or combine one canonical path. Security review: Security review cleared: The diff touches heartbeat runtime and tests only; no concrete security or supply-chain regression was found. AGENTS.md: found and applied where relevant. What I checked:
Likely related people:
Codex review notes: model gpt-5.5, reasoning high; reviewed against bd02977e294d. |
|
ClawSweeper PR egg: 🎁 locked until real behavior proof passes. Details
|
|
ClawSweeper applied the proposed close for this PR.
|
Problem
Heartbeat runs can persist non-ack final output in
pendingFinalDelivery*so the final user-visible message can be retried. Normal reply dispatch clears that pending state after successful delivery, butrunHeartbeatOncesends heartbeat output throughsendDurableMessageBatchdirectly and bypasses the normal cleanup path.That leaves a stale
pendingFinalDeliveryTextin the session store after the heartbeat message was already delivered. On later heartbeat wakes, the pending text can be replayed or re-classified before fresh work runs, which is why an old heartbeat report can keep resurfacing instead of disappearing after a successful delivery.Relationship to #83187
#83187 covers the heartbeat send-success cleanup path and already has strong proof for that narrower case.
This PR additionally covers the duplicate-suppression path: if the same heartbeat payload was already delivered recently (
lastHeartbeatText/lastHeartbeatSentAt) but matching stalependingFinalDelivery*fields remain, the duplicate branch currently returns before the send-success store write and therefore needs its own cleanup. If maintainers prefer #83187 as the canonical landing branch, the duplicate-suppression cleanup and regression coverage from this PR can be ported there.Fix
pendingFinalDelivery*only when the stored pending text exactly matches the heartbeat text that was delivered.lastHeartbeatText/lastHeartbeatSentAtshow that exact heartbeat payload was already sent recently.Safety
The cleanup is intentionally narrow: it requires
pendingFinalDelivery === trueand a trimmed exact text match. It does not clear unrelated pending delivery state, and it does not run on failed sends, no-target skips, or alerts-disabled skips.Verification
node scripts/run-vitest.mjs run --config test/vitest/vitest.infra.config.ts src/infra/heartbeat-runner.returns-default-unset.test.ts— 41 tests passed.pnpm exec oxfmt --check --threads=1 src/infra/heartbeat-runner.ts src/infra/heartbeat-runner.returns-default-unset.test.ts— clean.Real behavior proof
Environment: disposable local OpenClaw source-runtime run on branch
codex/fix-heartbeat-pending-replay, head9497a7a032cf. The run used a temporary config/workspace/session store and the actualrunHeartbeatOnce+sendDurableMessageBatchheartbeat path with a Telegram outbound adapter registered in the plugin registry. The model reply was stubbed to deterministic text so the proof isolates heartbeat delivery/session-state behavior. No production OpenClaw data or real chat credentials were used.Command shape:
Observed runtime logs:
Send-success scenario:
{ "scenario": "send-success", "branch": "codex/fix-heartbeat-pending-replay", "head": "9497a7a032cf", "result": { "status": "ran", "durationMs": 387 }, "sendCallCount": 1, "sends": [ { "to": "proof-chat", "text": "PR87583 send-success heartbeat payload", "accountId": null } ], "before": { "pendingFinalDelivery": true, "pendingFinalDeliveryText": "PR87583 send-success heartbeat payload", "pendingFinalDeliveryCreatedAt": 1779957645636, "pendingFinalDeliveryLastAttemptAt": 1779957705636, "pendingFinalDeliveryAttemptCount": 2, "pendingFinalDeliveryLastError": "redacted prior delivery failure", "pendingFinalDeliveryContext": { "channel": "telegram", "to": "proof-chat" }, "pendingFinalDeliveryIntentId": "intent-send-success", "lastHeartbeatText": "<absent>", "lastHeartbeatSentAt": "<absent>" }, "after": { "pendingFinalDelivery": "<absent>", "pendingFinalDeliveryText": "<absent>", "pendingFinalDeliveryCreatedAt": "<absent>", "pendingFinalDeliveryLastAttemptAt": "<absent>", "pendingFinalDeliveryAttemptCount": "<absent>", "pendingFinalDeliveryLastError": "<absent>", "pendingFinalDeliveryContext": "<absent>", "pendingFinalDeliveryIntentId": "<absent>", "lastHeartbeatText": "PR87583 send-success heartbeat payload", "lastHeartbeatSentAt": 1779957765636 }, "clearedAllPendingFields": true, "deliveryEvidence": "sendDurableMessageBatch reached Telegram outbound adapter" }Duplicate-suppression scenario:
{ "scenario": "duplicate-skip", "branch": "codex/fix-heartbeat-pending-replay", "head": "9497a7a032cf", "result": { "status": "ran", "durationMs": 51 }, "sendCallCount": 0, "before": { "pendingFinalDelivery": true, "pendingFinalDeliveryText": "PR87583 duplicate-skip heartbeat payload", "pendingFinalDeliveryCreatedAt": 1779957646044, "pendingFinalDeliveryLastAttemptAt": 1779957706044, "pendingFinalDeliveryAttemptCount": 2, "pendingFinalDeliveryLastError": "redacted prior delivery failure", "pendingFinalDeliveryContext": { "channel": "telegram", "to": "proof-chat" }, "pendingFinalDeliveryIntentId": "intent-duplicate-skip", "lastHeartbeatText": "PR87583 duplicate-skip heartbeat payload", "lastHeartbeatSentAt": 1779957706044 }, "after": { "pendingFinalDelivery": "<absent>", "pendingFinalDeliveryText": "<absent>", "pendingFinalDeliveryCreatedAt": "<absent>", "pendingFinalDeliveryLastAttemptAt": "<absent>", "pendingFinalDeliveryAttemptCount": "<absent>", "pendingFinalDeliveryLastError": "<absent>", "pendingFinalDeliveryContext": "<absent>", "pendingFinalDeliveryIntentId": "<absent>", "lastHeartbeatText": "PR87583 duplicate-skip heartbeat payload", "lastHeartbeatSentAt": 1779957706044 }, "clearedAllPendingFields": true, "deliveryEvidence": "duplicate suppression skipped outbound send because lastHeartbeatText matched within 24h" }