Matrix: dedupe replayed inbound events on restart#50922
Matrix: dedupe replayed inbound events on restart#50922gumadeiras merged 7 commits intoopenclaw:mainfrom
Conversation
🔒 Aisle Security AnalysisWe found 1 potential security issue(s) in this PR:
1. 🟡 Potential CPU/IO DoS from per-event O(n) pruning and full-state JSON persistence in Matrix inbound dedupe cache
DescriptionThe Matrix inbound dedupe implementation performs linear-time pruning on the hot path and periodically persists the entire cache to disk as formatted JSON.
Vulnerable code (hot-path prune + full-state persistence): pruneSeenEvents({ seen, ttlMs, maxEntries, nowMs: nowMs() });
...
entries: Array.from(params.seen.entries()).map(([key, ts]) => ({ key, ts })),RecommendationReduce worst-case per-event work and persistence amplification:
Example (amortized prune + key length cap): const MAX_KEY_PART = 512;
let lastPruneMs = 0;
const PRUNE_INTERVAL_MS = 5_000;
function buildEventKey({ roomId, eventId }: { roomId: string; eventId: string }): string {
const r = roomId.trim().slice(0, MAX_KEY_PART);
const e = eventId.trim().slice(0, MAX_KEY_PART);
return r && e ? `${r}|${e}` : "";
}
function maybePrune(nowMs: number) {
if (nowMs - lastPruneMs < PRUNE_INTERVAL_MS) return;
lastPruneMs = nowMs;
pruneSeenEvents({ seen, ttlMs, maxEntries, nowMs });
}
// in claimEvent/commitEvent: call maybePrune(nowMs()) instead of pruneSeenEvents(...)Analyzed PR: #50922 at commit Last updated on: 2026-03-20T19:46:59Z |
There was a problem hiding this comment.
Pull request overview
Adds durable inbound event deduplication for the Matrix monitor so that already-handled Matrix room events aren’t replayed as new after gateway restarts, while still allowing clean-restart backlog delivery for unseen events.
Changes:
- Introduces a per-account, file-backed inbound event dedupe store keyed by
roomId|eventIdwith TTL + max-size pruning. - Wires the deduper into the Matrix monitor lifecycle (create on startup, stop/flush on shutdown) and into the inbound handler (claim/commit/release).
- Adds/updates tests covering persistence, skip-on-replay behavior, and claim/commit/release semantics.
Reviewed changes
Copilot reviewed 8 out of 8 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| extensions/matrix/src/matrix/monitor/index.ts | Creates and passes a durable inbound deduper into the handler; stops it during cleanup. |
| extensions/matrix/src/matrix/monitor/index.test.ts | Mocks the inbound deduper for monitor boot/cleanup tests. |
| extensions/matrix/src/matrix/monitor/inbound-dedupe.ts | New durable dedupe implementation with debounced atomic persistence + pruning. |
| extensions/matrix/src/matrix/monitor/inbound-dedupe.test.ts | New unit tests for persistence, pending-release behavior, and pruning on load. |
| extensions/matrix/src/matrix/monitor/handler.ts | Claims inbound events after basic filtering; commits on successful handling or explicit ignore paths; releases on errors. |
| extensions/matrix/src/matrix/monitor/handler.test.ts | Adds tests asserting skip/commit/release behavior and ordering. |
| extensions/matrix/src/matrix/monitor/handler.test-helpers.ts | Extends harness to inject an inbound deduper. |
| CHANGELOG.md | Documents the new durable inbound dedupe behavior for Matrix. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 77d62b2c59
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Greptile SummaryThis PR introduces durable deduplication for inbound Matrix room events, keyed by Key changes:
The claim/commit/release lifecycle is consistently applied across all ~15 return paths in the handler. Test coverage directly exercises ordering guarantees, failure-release paths, and persistence across restarts. Confidence Score: 4/5
Last reviewed commit: "Matrix: dedupe repla..." |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ba40b2c5a0
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e77f8231ef
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
af5cc26 to
cc00728
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cc00728bcc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
9e4fff2 to
12840a3
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 12840a38a9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
8470fbd to
1a0de51
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1a0de51c7b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
2fbaaf1 to
10d9770
Compare
|
Merged via squash.
Thanks @gumadeiras! |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 10d9770aa6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| await client.drainPendingDecryptions("matrix monitor shutdown"); | ||
| await waitForInFlightRoomMessages(); | ||
| threadBindingManager?.stop(); | ||
| } finally { | ||
| await inboundDeduper.stop(); | ||
| await releaseSharedClientInstance(client, "persist"); |
There was a problem hiding this comment.
Keep shared-client release in a finally block
If drainPendingDecryptions() or threadBindingManager.stop() throws during shutdown, this path exits before releaseSharedClientInstance(...) runs. client.stopSyncWithoutPersist() has already stopped the underlying Matrix client, but extensions/matrix/src/matrix/client/shared.ts:79-104 still keeps the shared state marked as started until release deletes it. In a long-lived gateway process, the next Matrix monitor start for the same account can therefore reuse a cached-but-stopped client and never resume syncing.
Useful? React with 👍 / 👎.
| for (let attempts = 0; attempts < MATRIX_DECRYPT_RETRY_MAX_ATTEMPTS; attempts += 1) { | ||
| if (this.decryptRetries.size === 0) { | ||
| return; | ||
| } | ||
| this.retryPendingNow(reason); |
There was a problem hiding this comment.
Report unfinished decrypt retries before persisting shutdown
This loop resolves after at most eight immediate passes even when decryptRetries still contains timers or in-flight work. extensions/matrix/src/matrix/monitor/index.ts:157-161 treats a resolved drain as success and then persists a clean shutdown; MatrixClient.stop() clears the remaining retry timers, so an encrypted event that still needs a late-arriving key at shutdown is dropped instead of being replayed on the next start. drainPendingDecryptions() needs to surface that it did not actually finish draining.
Useful? React with 👍 / 👎.
Merged via squash. Prepared head SHA: 10d9770 Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com> Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com> Reviewed-by: @gumadeiras
Merged via squash. Prepared head SHA: 10d9770 Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com> Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com> Reviewed-by: @gumadeiras
Merged via squash. Prepared head SHA: 10d9770 Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com> Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com> Reviewed-by: @gumadeiras
Merged via squash. Prepared head SHA: 10d9770 Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com> Co-authored-by: gumadeiras <5599352+gumadeiras@users.noreply.github.com> Reviewed-by: @gumadeiras
Summary
Validation