feat(lifecycle): persistent inbound dedup across gateway restarts#29957

Closed

nohat wants to merge 17 commits intoopenclaw:mainfrom

nohat:lifecycle/persistent-dedup-temp

Contributor

nohat commented Feb 28, 2026

Summary

Stack: merge after #29956 (the diff will be correct once #29956 is merged; until then this shows the combined diff)

Single commit: removes disablePersistentDedupe flag so acceptTurn() uses INSERT OR IGNORE with unique index on dedupe_key.

Dedup survives gateway restarts; pruned turns free their dedupe keys
Minimal change on top of the turn-tracking PR

Closes #14431
Related: #28941

Test plan

pnpm build passes
pnpm test passes (dedup persistence tests)
Manual: send message, restart gateway, resend same message — verify dedup rejects it

🤖 Generated with Claude Code

nohat and others added 17 commits

February 28, 2026 09:34


          feat(outbox): migrate delivery queue from file-based to SQLite outbox

16632ba

Replace unbounded file-based delivery queue with queryable SQLite
message_outbox table. Adds TTL/expiry for stale entries, delivery
outcome retention, and one-time legacy file queue import on startup.

Closes openclaw#23777, openclaw#16555, openclaw#29128


          feat(outbox): add write-ahead outbox, recovery worker, and plugin com…

ffe8337

…pat layer

Write-ahead delivery pattern: enqueue outbox entry before sending, ack on
success, retry on failure. Continuous outbox worker replaces one-shot
recovery. Plugin channels get durable delivery guarantees via v1/v2
adapter compat layer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>


          fix(outbound): track best-effort failures when onError is absent (ope…

73e208b

…nclaw#29148)


          fix(gateway): await lifecycle worker startup so shutdown can stop it …

37744de

…(P2 openclaw#29148)


          fix(lifecycle): capture startup cutoff before legacy queue import; al…

6264c17

…low v2 sendFinal-only plugin adapters (Codex P1+P2 openclaw#29148)


          fix(outbound): format deliver.ts and align recovery backoff with fail…

e23117e

…Delivery


          fix(outbound): restrict sendPayload fast path to native v2 adapters

62fa6e0


          fix(outbound): preserve sendPayload fast path for v1 adapters with na…

6b71122

…tive sendPayload


          fix(outbox): store dispatchKind and skip non-final entries during rec…

f27f9bd

…overy


          fix(outbox): remove v2 contract from adapters without native sendFina…

2505ec1

…l and separate ackDelivery errors in recovery


          fix(outbound): make sendFinal required on all adapters, remove v1/v2 …

7f4421e

…compat layer


          feat(lifecycle): add inbound turn tracking and orphan recovery

54b0b23

Every inbound message creates a durable turn record in message_turns.
Turn worker detects orphaned turns (accepted but never completed after
crash) and recovers them. Abort commands mark turns as aborted,
preventing re-delivery. Outbox entries are linked to turns for
coordinated finalization.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>


          fix(lifecycle): add active-turn registry to prevent recovery races an…

14120b4

…d premature finalization (Codex P1+P2 openclaw#29149)


          fix(lifecycle): remove dead minAgeMs parameter from listRecoverableTurns

536640f


          fix(lifecycle): cancel outbox entries for aborted turns

895508e


          fix(lifecycle): bypass inbound dedupe for resumed turns and gate fail…

e5e4c0d

…-open on delivery stats


          feat(lifecycle): enable persistent inbound dedup across restarts

32c8710

openclaw-barnacle bot added channel: bluebubbles channel: discord channel: googlechat channel: imessage channel: line channel: matrix channel: mattermost channel: msteams channel: nextcloud-talk channel: nostr channel: signal channel: slack channel: telegram labels

openclaw-barnacle bot added the channel: irc label

This was referenced Feb 28, 2026

feat(lifecycle): add inbound turn tracking and orphan recovery #29149

Closed

feat(lifecycle): enable persistent inbound dedup across restarts #29262

Closed

openclaw-barnacle bot added the size: XL label

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

6 similar comments

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot closed this

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

15 similar comments

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

openclaw-barnacle bot commented Feb 28, 2026

Closing this PR because it looks dirty (too many unrelated or unexpected changes). This usually happens when a branch picks up unrelated commits or a merge went sideways. Please recreate the PR from a clean branch.

chatgpt-codex-connector bot reviewed

View reviewed changes

chatgpt-codex-connector bot left a comment

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 32c87109ac

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

src/infra/outbound/deliver.ts

Comment on lines +204 to +205

		sendPayload: async (payload, overrides) =>
		outbound.sendFinal({

chatgpt-codex-connector bot Feb 28, 2026

Keep legacy outbound adapter fallback paths

createPluginHandler now always routes channel-data sends through outbound.sendFinal(...), but dynamically loaded third-party plugins built against the previous SDK may still expose only sendPayload/sendText/sendMedia. In that case this call becomes a runtime TypeError as soon as a payload with channelData is emitted, which breaks outbound delivery for existing plugins after upgrade; keep a runtime fallback for adapters that don't implement sendFinal yet.

Useful? React with 👍 / 👎.

Contributor

greptile-apps bot commented Feb 28, 2026

Greptile Summary

This PR enables persistent inbound message deduplication across gateway restarts by removing the disablePersistentDedupe flag. The change is minimal (6 lines removed) and builds on the turn-tracking infrastructure from PR #29956.

Key changes:

Removes the disablePersistentDedupe = true flag from acceptTurn() function
Enables INSERT OR IGNORE with unique index on dedupe_key column
Adds comprehensive tests covering deduplication persistence, restart simulation, and pruning behavior

How it works:

acceptTurn() now always computes dedupeKey from buildInboundDedupeKey(ctx) (provider + accountId + sessionKey + peerId + threadId + messageId)
Database has partial unique index on dedupe_key column (only when NOT NULL)
Duplicate messages return accepted: false and are logged but not processed
Pruned turns free their dedupe keys, allowing re-acceptance after cleanup
Fallback to in-memory dedup cache if database write fails (fail-open with warning)

Testing:
All scenarios covered including first accept, duplicate rejection, cross-restart persistence, messages without dedupe keys, and pruning behavior.

Confidence Score: 5/5

This PR is safe to merge with minimal risk
The change is minimal (6 lines), well-tested with comprehensive coverage, and has defensive fallback behavior. The implementation correctly uses SQLite's INSERT OR IGNORE with a partial unique index, and the integration with existing code (dispatch.ts, recovery workers) is correct. No logical errors, security issues, or performance concerns identified.
No files require special attention - the actual PR changes only 2 files (turns.ts and turns.test.ts). The 66 other files shown in the diff are from the stacked PR feat(lifecycle): inbound turn tracking, orphan recovery, and abort coordination #29956 which should be reviewed separately.

_{Last reviewed commit: 32c8710}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: bluebubbles channel: discord channel: feishu channel: googlechat channel: imessage channel: irc channel: line channel: matrix channel: mattermost channel: msteams channel: nextcloud-talk channel: nostr channel: signal channel: slack channel: telegram channel: tlon channel: twitch channel: whatsapp-web channel: zalo channel: zalouser commands gateway size: XL