Skip to content

Matrix: dedupe replayed inbound events on restart#50922

Merged
gumadeiras merged 7 commits intoopenclaw:mainfrom
gumadeiras:codex/matrix-durable-inbound-dedupe
Mar 20, 2026
Merged

Matrix: dedupe replayed inbound events on restart#50922
gumadeiras merged 7 commits intoopenclaw:mainfrom
gumadeiras:codex/matrix-durable-inbound-dedupe

Conversation

@gumadeiras
Copy link
Copy Markdown
Member

Summary

  • add durable per-account inbound Matrix event dedupe keyed by room id and event id
  • preserve clean-restart backlog delivery for unseen events instead of dropping all pre-startup history
  • commit dedupe only after successful handling or intentional ignore paths so transient failures can retry

Validation

  • pnpm test -- extensions/matrix/src/matrix/monitor/inbound-dedupe.test.ts
  • pnpm test -- extensions/matrix/src/matrix/monitor/index.test.ts
  • pnpm test -- extensions/matrix/src/matrix/monitor/handler.test.ts -t "durable inbound dedupe|pre-startup"
  • pnpm build

Copilot AI review requested due to automatic review settings March 20, 2026 07:57
@aisle-research-bot
Copy link
Copy Markdown

aisle-research-bot Bot commented Mar 20, 2026

🔒 Aisle Security Analysis

We found 1 potential security issue(s) in this PR:

# Severity Title
1 🟡 Medium Potential CPU/IO DoS from per-event O(n) pruning and full-state JSON persistence in Matrix inbound dedupe cache

1. 🟡 Potential CPU/IO DoS from per-event O(n) pruning and full-state JSON persistence in Matrix inbound dedupe cache

Property Value
Severity Medium
CWE CWE-400
Location extensions/matrix/src/matrix/monitor/inbound-dedupe.ts:82-108

Description

The Matrix inbound dedupe implementation performs linear-time pruning on the hot path and periodically persists the entire cache to disk as formatted JSON.

  • claimEvent() and commitEvent() both call pruneSeenEvents(), which iterates over all entries in the seen Map to remove TTL-expired entries.
  • With the default maxEntries = 20_000, an attacker (or any high-volume room) can keep seen near the cap by sending many unique events, making each inbound message pay an O(maxEntries) cost.
  • commitEvent() schedules persistence; persistence serializes the full state (Array.from(seen.entries())) and writes it atomically. Under sustained traffic this can become significant CPU (JSON stringify) and disk IO amplification.
  • Keys are built from roomId|eventId with only .trim() normalization, so if upstream IDs are unexpectedly large, both in-memory and on-disk costs increase further.

Vulnerable code (hot-path prune + full-state persistence):

pruneSeenEvents({ seen, ttlMs, maxEntries, nowMs: nowMs() });
...
entries: Array.from(params.seen.entries()).map(([key, ts]) => ({ key, ts })),

Recommendation

Reduce worst-case per-event work and persistence amplification:

  1. Amortize pruning instead of scanning the entire map on every event.

    • Prune on a timer (e.g., once every N seconds), or
    • Prune only a bounded number of entries per call, or
    • Maintain an auxiliary queue/min-heap for expirations.
  2. Avoid rewriting the entire state too frequently:

    • Persist less often under load (adaptive backoff), or
    • Persist incremental updates (append-only log) and compact occasionally.
  3. Bound key size defensively (even if upstream typically limits IDs):

    • Reject or hash overly long roomId/eventId when building the key.

Example (amortized prune + key length cap):

const MAX_KEY_PART = 512;
let lastPruneMs = 0;
const PRUNE_INTERVAL_MS = 5_000;

function buildEventKey({ roomId, eventId }: { roomId: string; eventId: string }): string {
  const r = roomId.trim().slice(0, MAX_KEY_PART);
  const e = eventId.trim().slice(0, MAX_KEY_PART);
  return r && e ? `${r}|${e}` : "";
}

function maybePrune(nowMs: number) {
  if (nowMs - lastPruneMs < PRUNE_INTERVAL_MS) return;
  lastPruneMs = nowMs;
  pruneSeenEvents({ seen, ttlMs, maxEntries, nowMs });
}// in claimEvent/commitEvent: call maybePrune(nowMs()) instead of pruneSeenEvents(...)

Analyzed PR: #50922 at commit 10d9770

Last updated on: 2026-03-20T19:46:59Z

@openclaw-barnacle openclaw-barnacle Bot added channel: matrix Channel integration: matrix size: L maintainer Maintainer-authored PR labels Mar 20, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds durable inbound event deduplication for the Matrix monitor so that already-handled Matrix room events aren’t replayed as new after gateway restarts, while still allowing clean-restart backlog delivery for unseen events.

Changes:

  • Introduces a per-account, file-backed inbound event dedupe store keyed by roomId|eventId with TTL + max-size pruning.
  • Wires the deduper into the Matrix monitor lifecycle (create on startup, stop/flush on shutdown) and into the inbound handler (claim/commit/release).
  • Adds/updates tests covering persistence, skip-on-replay behavior, and claim/commit/release semantics.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
extensions/matrix/src/matrix/monitor/index.ts Creates and passes a durable inbound deduper into the handler; stops it during cleanup.
extensions/matrix/src/matrix/monitor/index.test.ts Mocks the inbound deduper for monitor boot/cleanup tests.
extensions/matrix/src/matrix/monitor/inbound-dedupe.ts New durable dedupe implementation with debounced atomic persistence + pruning.
extensions/matrix/src/matrix/monitor/inbound-dedupe.test.ts New unit tests for persistence, pending-release behavior, and pruning on load.
extensions/matrix/src/matrix/monitor/handler.ts Claims inbound events after basic filtering; commits on successful handling or explicit ignore paths; releases on errors.
extensions/matrix/src/matrix/monitor/handler.test.ts Adds tests asserting skip/commit/release behavior and ordering.
extensions/matrix/src/matrix/monitor/handler.test-helpers.ts Extends harness to inject an inbound deduper.
CHANGELOG.md Documents the new durable inbound dedupe behavior for Matrix.

Comment thread extensions/matrix/src/matrix/monitor/index.ts
Comment thread extensions/matrix/src/matrix/monitor/handler.ts Outdated
Comment thread extensions/matrix/src/matrix/monitor/inbound-dedupe.ts Outdated
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 77d62b2c59

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread extensions/matrix/src/matrix/monitor/handler.ts
Comment thread extensions/matrix/src/matrix/monitor/inbound-dedupe.ts Outdated
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Mar 20, 2026

Greptile Summary

This PR introduces durable deduplication for inbound Matrix room events, keyed by roomId + eventId, so that previously handled messages are not replayed as new events after a gateway restart. The implementation adds inbound-dedupe.ts, a file-backed dedupe store with a claim/commit/release lifecycle, and integrates it into the existing monitor handler and startup flow.

Key changes:

  • New createMatrixInboundEventDeduper factory: maintains an in-memory seen Map and an in-flight pending Set, backed by a debounced atomic JSON write, with configurable TTL (default 30 days) and max-entries (default 20,000) pruning.
  • claimEvent is called synchronously after fast-path early filters (pre-startup age, self-events, redacted events, etc.) but before any expensive async work that could produce side effects.
  • commitEvent is called on all deliberate-ignore paths (policy-blocked, no-mention, no body, etc.) and after successful reply dispatch, ensuring idempotent events are durably marked.
  • releaseEvent is called in a finally block so that transient failures (disk errors, upstream failures) release the claim and allow retry on the next restart.
  • stop() flushes any pending debounced write before the client is torn down.

The claim/commit/release lifecycle is consistently applied across all ~15 return paths in the handler. Test coverage directly exercises ordering guarantees, failure-release paths, and persistence across restarts.

Confidence Score: 4/5

  • This PR is safe to merge — the dedupe logic is well-structured and extensively tested, with no identified correctness issues.
  • The claim/commit/release lifecycle is correctly applied across all ~15 return paths in the handler. The finally block provides a reliable safety net for transient failures. Test coverage is thorough for ordering guarantees, failure-release paths, and persistence across restarts. A score of 4 (rather than 5) reflects the inherent complexity of a multi-path lifecycle integration that is hard to exhaustively verify by reading alone, and the fact that persist-failure recovery relies on the next commitEvent call to reschedule rather than an explicit retry.
  • No files require special attention; handler.ts is the most complex integration point but has been carefully audited.

Last reviewed commit: "Matrix: dedupe repla..."

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ba40b2c5a0

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread extensions/matrix/src/matrix/monitor/index.ts Outdated
Comment thread extensions/matrix/src/matrix/monitor/handler.ts
@gumadeiras gumadeiras self-assigned this Mar 20, 2026
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e77f8231ef

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread extensions/matrix/src/matrix/monitor/handler.ts
@gumadeiras gumadeiras force-pushed the codex/matrix-durable-inbound-dedupe branch 2 times, most recently from af5cc26 to cc00728 Compare March 20, 2026 17:27
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cc00728bcc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread extensions/matrix/src/matrix/monitor/index.ts Outdated
@gumadeiras gumadeiras force-pushed the codex/matrix-durable-inbound-dedupe branch from 9e4fff2 to 12840a3 Compare March 20, 2026 18:16
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 12840a38a9

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread extensions/matrix/src/matrix/monitor/index.ts
@gumadeiras gumadeiras force-pushed the codex/matrix-durable-inbound-dedupe branch from 8470fbd to 1a0de51 Compare March 20, 2026 18:29
Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a0de51c7b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread extensions/matrix/src/matrix/monitor/index.ts Outdated
Comment thread extensions/matrix/src/matrix/sdk/decrypt-bridge.ts
@openclaw-barnacle openclaw-barnacle Bot added the docs Improvements or additions to documentation label Mar 20, 2026
@gumadeiras gumadeiras force-pushed the codex/matrix-durable-inbound-dedupe branch from 2fbaaf1 to 10d9770 Compare March 20, 2026 19:10
@openclaw-barnacle openclaw-barnacle Bot removed the docs Improvements or additions to documentation label Mar 20, 2026
@gumadeiras gumadeiras merged commit a05da76 into openclaw:main Mar 20, 2026
34 of 37 checks passed
@gumadeiras
Copy link
Copy Markdown
Member Author

Merged via squash.

Thanks @gumadeiras!

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 10d9770aa6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +157 to 161
await client.drainPendingDecryptions("matrix monitor shutdown");
await waitForInFlightRoomMessages();
threadBindingManager?.stop();
} finally {
await inboundDeduper.stop();
await releaseSharedClientInstance(client, "persist");
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep shared-client release in a finally block

If drainPendingDecryptions() or threadBindingManager.stop() throws during shutdown, this path exits before releaseSharedClientInstance(...) runs. client.stopSyncWithoutPersist() has already stopped the underlying Matrix client, but extensions/matrix/src/matrix/client/shared.ts:79-104 still keeps the shared state marked as started until release deletes it. In a long-lived gateway process, the next Matrix monitor start for the same account can therefore reuse a cached-but-stopped client and never resume syncing.

Useful? React with 👍 / 👎.

Comment on lines +145 to +149
for (let attempts = 0; attempts < MATRIX_DECRYPT_RETRY_MAX_ATTEMPTS; attempts += 1) {
if (this.decryptRetries.size === 0) {
return;
}
this.retryPendingNow(reason);
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Report unfinished decrypt retries before persisting shutdown

This loop resolves after at most eight immediate passes even when decryptRetries still contains timers or in-flight work. extensions/matrix/src/matrix/monitor/index.ts:157-161 treats a resolved drain as success and then persists a clean shutdown; MatrixClient.stop() clears the remaining retry timers, so an encrypted event that still needs a late-arriving key at shutdown is dropped instead of being replayed on the next start. drainPendingDecryptions() needs to surface that it did not actually finish draining.

Useful? React with 👍 / 👎.

frankekn pushed a commit to artwalker/openclaw that referenced this pull request Mar 23, 2026
Merged via squash.

Prepared head SHA: 10d9770
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
furaul pushed a commit to furaul/openclaw that referenced this pull request Mar 24, 2026
Merged via squash.

Prepared head SHA: 10d9770
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
lovewanwan pushed a commit to lovewanwan/openclaw that referenced this pull request Apr 28, 2026
Merged via squash.

Prepared head SHA: 10d9770
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
ogt-redknie pushed a commit to ogt-redknie/OPENX that referenced this pull request May 2, 2026
Merged via squash.

Prepared head SHA: 10d9770
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com>
Reviewed-by: @gumadeiras
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

channel: matrix Channel integration: matrix maintainer Maintainer-authored PR size: XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants