fix(bluebubbles): dedupe inbound webhooks across restarts (#19176, #12053)#66810
fix(bluebubbles): dedupe inbound webhooks across restarts (#19176, #12053)#66810omarshahine wants to merge 1 commit intomainfrom
Conversation
…2053) BlueBubbles MessagePoller replays its ~1-week lookback window as new-message webhooks after BB Server restart or reconnect. Without persistent dedup, the gateway re-replies to messages it already handled before the restart. Add a persistent file-backed GUID dedupe (TTL=7d, matching BB's lookback window) at the top of processMessage, using the same createClaimableDedupe SDK primitive as Feishu. The on-disk store at ~/.openclaw/bluebubbles/inbound-dedupe/<account>.json survives gateway restarts. Claim/finalize/release semantics ensure transient delivery failures release the GUID so a later replay can retry, while successful deliveries are committed and block future replays. Fixes #19176, #12053.
|
Branch name is stuck — GitHub won't fire CI events for it. Moving to a fresh branch. |
Greptile SummaryAdds a persistent file-backed GUID dedupe to the BlueBubbles Confidence Score: 5/5Safe to merge — the core dedupe logic is correct, well-tested for the claim/release lifecycle, and consistent with the existing Feishu pattern. All remaining findings are P2: a slightly misleading code comment and missing unit tests for the balloon-event key-resolution function. Neither blocks the fix or affects correctness of the production path. No files require special attention. Prompt To Fix All With AIThis is a comment left during a code review.
Path: extensions/bluebubbles/src/inbound-dedupe.ts
Line: 26-27
Comment:
**Misleading comment about persistence test**
The comment "Stable-per-pid so the scoped dedupe test can observe persistence" implies a test exercises the file-backed path, but `inbound-dedupe.test.ts` calls `_resetBlueBubblesInboundDedupForTest()` in `beforeEach`, which installs a memory-only impl for every test. No current test actually reaches disk. The stable-per-pid path is a useful safety net for future persistence tests, but the comment overstates what the current test suite covers.
```suggestion
// Stable-per-pid so concurrent test workers do not share the same temp dir
// if a future test exercises file-backed persistence.
```
How can I resolve this? If you propose a fix, please make it concise.
---
This is a comment left during a code review.
Path: extensions/bluebubbles/src/inbound-dedupe.test.ts
Line: 1-5
Comment:
**`resolveBlueBubblesInboundDedupeKey` has no test coverage**
The new `resolveBlueBubblesInboundDedupeKey` function contains non-trivial logic (the balloon-event key-aliasing path: when both `balloonBundleId` and `associatedMessageGuid` are present, use the `associatedMessageGuid` rather than `messageId`) and a documented tradeoff around post-merge balloon+text ordering. Since this key-resolution function determines what the dedupe actually gates on, a few targeted tests would help lock the contract and make the tradeoff concrete:
- Regular message → keyed by `messageId`
- Balloon with both fields → keyed by `associatedMessageGuid`
- Balloon without `balloonBundleId` (reply context only) → keyed by `messageId`
How can I resolve this? If you propose a fix, please make it concise.Reviews (1): Last reviewed commit: "fix(bluebubbles): dedupe inbound webhook..." | Re-trigger Greptile |
…66810 The inbound-dedupe PR was reopened as #66810 (#66230 closed without merging at 21:10:28Z, #66810 created 7s later on the same lobster/bb-inbound-dedupe branch). Updating code comments and the catchup CHANGELOG entry to point at the live PR. The branch dependency chain (this PR stacked on lobster/bb-inbound-dedupe) is unchanged. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
| // Stable-per-pid so the scoped dedupe test can observe persistence. | ||
| const name = "openclaw-vitest-" + process.pid; |
There was a problem hiding this comment.
Misleading comment about persistence test
The comment "Stable-per-pid so the scoped dedupe test can observe persistence" implies a test exercises the file-backed path, but inbound-dedupe.test.ts calls _resetBlueBubblesInboundDedupForTest() in beforeEach, which installs a memory-only impl for every test. No current test actually reaches disk. The stable-per-pid path is a useful safety net for future persistence tests, but the comment overstates what the current test suite covers.
| // Stable-per-pid so the scoped dedupe test can observe persistence. | |
| const name = "openclaw-vitest-" + process.pid; | |
| // Stable-per-pid so concurrent test workers do not share the same temp dir | |
| // if a future test exercises file-backed persistence. |
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/bluebubbles/src/inbound-dedupe.ts
Line: 26-27
Comment:
**Misleading comment about persistence test**
The comment "Stable-per-pid so the scoped dedupe test can observe persistence" implies a test exercises the file-backed path, but `inbound-dedupe.test.ts` calls `_resetBlueBubblesInboundDedupForTest()` in `beforeEach`, which installs a memory-only impl for every test. No current test actually reaches disk. The stable-per-pid path is a useful safety net for future persistence tests, but the comment overstates what the current test suite covers.
```suggestion
// Stable-per-pid so concurrent test workers do not share the same temp dir
// if a future test exercises file-backed persistence.
```
How can I resolve this? If you propose a fix, please make it concise.| import { beforeEach, describe, expect, it } from "vitest"; | ||
| import { | ||
| _resetBlueBubblesInboundDedupForTest, | ||
| claimBlueBubblesInboundMessage, | ||
| } from "./inbound-dedupe.js"; |
There was a problem hiding this comment.
resolveBlueBubblesInboundDedupeKey has no test coverage
The new resolveBlueBubblesInboundDedupeKey function contains non-trivial logic (the balloon-event key-aliasing path: when both balloonBundleId and associatedMessageGuid are present, use the associatedMessageGuid rather than messageId) and a documented tradeoff around post-merge balloon+text ordering. Since this key-resolution function determines what the dedupe actually gates on, a few targeted tests would help lock the contract and make the tradeoff concrete:
- Regular message → keyed by
messageId - Balloon with both fields → keyed by
associatedMessageGuid - Balloon without
balloonBundleId(reply context only) → keyed bymessageId
Prompt To Fix With AI
This is a comment left during a code review.
Path: extensions/bluebubbles/src/inbound-dedupe.test.ts
Line: 1-5
Comment:
**`resolveBlueBubblesInboundDedupeKey` has no test coverage**
The new `resolveBlueBubblesInboundDedupeKey` function contains non-trivial logic (the balloon-event key-aliasing path: when both `balloonBundleId` and `associatedMessageGuid` are present, use the `associatedMessageGuid` rather than `messageId`) and a documented tradeoff around post-merge balloon+text ordering. Since this key-resolution function determines what the dedupe actually gates on, a few targeted tests would help lock the contract and make the tradeoff concrete:
- Regular message → keyed by `messageId`
- Balloon with both fields → keyed by `associatedMessageGuid`
- Balloon without `balloonBundleId` (reply context only) → keyed by `messageId`
How can I resolve this? If you propose a fix, please make it concise.There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 82bce2647e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const claim = await claimBlueBubblesInboundMessage({ | ||
| guid: dedupeKey, | ||
| accountId: account.accountId, | ||
| onDiskError: (error) => |
There was a problem hiding this comment.
Skip dedupe claim for fromMe messages
The new wrapper claims and commits dedupe keys before the existing message.fromMe early-return path runs, so outbound/self messages now consume slots in the persistent inbound replay file. Because inbound-dedupe.ts caps storage at 50,000 entries, high-volume accounts can evict real inbound GUIDs earlier than the 7-day TTL, allowing restart replays to slip through and be re-processed. Gate this dedupe to inbound user messages (or at least after the fromMe branch) so outbound traffic does not age out the keys this fix depends on.
Useful? React with 👍 / 👎.
🔒 Aisle Security AnalysisWe found 4 potential security issue(s) in this PR:
1. 🟡 Sensitive identifiers (message GUID and senderId) logged on inbound dedupe drops
Description
These values are likely stable identifiers that can be correlated with users/conversations (e.g., sender phone/email/handle and message GUIDs). Writing them to logs can expose PII/sensitive metadata to:
Vulnerable code: if (claim.kind === "duplicate" || claim.kind === "inflight") {
logVerbose(
core,
runtime,
`drop: ${claim.kind} inbound key=${sanitizeForLog(dedupeKey ?? "")} sender=${sanitizeForLog(message.senderId)}`,
);
return;
}RecommendationAvoid logging raw sender identifiers and message GUIDs. Prefer one of:
Example (hash + truncation): import { createHash } from "node:crypto";
function hashForLog(value: string): string {
return createHash("sha256").update(value, "utf8").digest("hex").slice(0, 12);
}
const keyTag = dedupeKey ? hashForLog(dedupeKey) : "";
const senderTag = message.senderId ? hashForLog(String(message.senderId)) : "";
logVerbose(core, runtime, `drop: ${claim.kind} inbound key_hash=${keyTag} sender_hash=${senderTag}`);Also consider gating even the hashed output behind an explicit debug setting, and ensure logs are access-controlled and retained minimally. 2. 🟡 Log injection / sensitive data exposure via unsanitized error logging in BlueBubbles reply error handler
DescriptionIn
Vulnerable code: onError: (err, info) => {
if (info.kind === "final") {
dedupeSignal.deliveryFailed = true;
}
runtime.error?.(`BlueBubbles ${info.kind} reply failed: ${String(err)}`);
},RecommendationSanitize or structure error logging to prevent log forging and reduce sensitive data exposure. Option A (reuse existing helper): runtime.error?.(
`BlueBubbles ${info.kind} reply failed: ${sanitizeForLog(err)}`,
);Option B (structured logging with minimal fields): runtime.error?.(
`BlueBubbles reply failed kind=${sanitizeForLog(info.kind)} err=${sanitizeForLog((err as Error)?.message ?? err)}`
);Additionally consider avoiding logging raw HTTP response bodies from upstream services; prefer status codes + correlation ids. 3. 🟡 Symlink-following file write in atomic JSON writer enables local arbitrary file overwrite via BlueBubbles inbound dedupe persistence
Description
Impact (local attacker / same-host):
Vulnerable code (Windows fallback overwrite): // src/infra/json-files.ts
await fs.copyFile(tempPath, filePath);This becomes reachable via BlueBubbles inbound dedupe persistence, which derives a per-account file path and ultimately calls RecommendationHarden atomic writes against symlink/hardlink attacks (CWE-59):
Example (sketch): import fs from "node:fs/promises";
async function assertNotSymlink(p: string) {
try {
const st = await fs.lstat(p);
if (st.isSymbolicLink()) throw new Error(`Refusing to write through symlink: ${p}`);
} catch (e: any) {
if (e?.code !== "ENOENT") throw e;
}
}
await assertNotSymlink(filePath);
// then proceed with rename/copyAdditionally, consider resolving 4. 🔵 Unbounded attacker-controlled fields logged in inbound dedupe path (potential log/CPU DoS)
DescriptionThe inbound-dedupe wrapper logs If an attacker can send authenticated BlueBubbles webhook events (or if the webhook token is leaked), they can supply extremely large strings in fields that flow into these logs (e.g.,
Vulnerable code (new logging): function sanitizeForLog(value: unknown): string {
return String(value).replace(/[\r\n\t\p{C}]/gu, " ");
}
...
`drop: ${claim.kind} inbound key=${sanitizeForLog(dedupeKey ?? "")} sender=${sanitizeForLog(message.senderId)}`
...
`inbound-dedupe: finalize failed for key=${sanitizeForLog(dedupeKey ?? "")}: ${sanitizeForLog(finalizeError)}`RecommendationAdd size limits when logging values derived from webhook payloads and error objects. For example, cap log fields to a small, safe maximum (e.g., 200 chars) and consider logging hashes instead of raw identifiers: function sanitizeForLog(value: unknown, maxLen = 200): string {
const s = String(value).replace(/[\r\n\t\p{C}]/gu, " ");
return s.length > maxLen ? s.slice(0, maxLen) + "…" : s;
}
// Or log hashes for high-entropy IDs
// const keyHash = createHash("sha256").update(dedupeKey ?? "").digest("hex").slice(0, 12);Also consider avoiding Analyzed PR: #66810 at commit Last updated on: 2026-04-14T21:20:02Z |
…66816 The inbound-dedupe PR was reopened again as #66816 (closed-without-merge trail: #66230 → #66810 → #66816). The branch was force-pushed and the new PR uses the parallel `fix/bb-inbound-dedupe` branch. Updating code comments and the catchup CHANGELOG entry to point at the live PR. Stacking on top of the dedupe branch will be addressed in a follow-up rebase. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Summary
BlueBubbles `MessagePoller` keeps a ~1-week lookback and re-fires `new-message` webhooks after BB Server restart or reconnection. With no sequence number or ack in the BB webhook protocol, the gateway previously had no way to recognize replays and would happily re-reply to messages it had already handled before the restart — producing duplicated outbound messages, and confusing replies to stale inbound that the user had already moved on from (see #19176, #12053).
This PR adds a persistent, file-backed inbound dedupe keyed by message GUID, modeled after the same `createPersistentDedupe` pattern used by the Feishu plugin.
Why this approach
Other channels with monotonic sequence IDs (Telegram's `update_id`, Matrix's sync token, Discord's gateway sequence) can dedupe natively via protocol. BlueBubbles does not expose anything like that, so an identity-based persistent dedupe at the message layer is the closest equivalent that fits how BB actually delivers webhooks.
Interaction with edit events (`updated-message`)
PR #52277 raised a related concern: if dedupe keys are GUID-only, a legitimate `updated-message` event would share a GUID with its original `new-message` and get dropped as a duplicate.
In the current codebase this cannot happen: `monitor.ts` routes `updated-message` payloads differently — without a reaction they are dropped at the webhook layer ("ignored without reaction"), and with a reaction they flow through `processReaction`, not `processMessage`. Our dedupe sits inside `processMessage`, so only `new-message` events are gated. Edits can't collide today.
If the separate work in #52277 lands and begins routing text-edit bodies into `processMessage`, the dedupe key will need to expand to include event type and edit metadata (e.g. `guid + eventType + (dateEdited||"")`) so an edit is treated as a distinct key. That's a straightforward forward-compatible change when it's needed.
Validation
Credits
Re-creates and improves on the focused fix from #31159 by @dashhuang — same behavioral goal (drop stale BB webhook replays), implemented as a persistent on-disk dedupe so it actually survives the gateway-restart case that drives the bug, and without the module-global mutable state that made the original patch need test-reset plumbing.
Fixes #19176, #12053.
Test plan