Skip to content

Commit 9caff5f

Browse files
omarshahineomarshahine
authored
fix(imessage): gate split-send coalescing on imsg balloon metadata with back-compat (#90858)
Gate iMessage same-sender DM split-send coalescing on imsg's structural `balloon_bundle_id` URL-balloon marker (openclaw/imsg#137) instead of timing/ text-shape inference, with a session capability latch and a back-compat path: - URL-balloon marker present -> merge (precise split-send). - Build known to emit balloon metadata (session latch) -> keep non-marker buckets separate (the precision win). - Build that never emits balloon metadata (older imsg) -> preserve the legacy unconditional merge, so split-send users do not regress to two turns. Never merges more than shipped main already did. Verified live end-to-end: the patched gateway, watching a real chat.db via an imsg #137 build, merged a real iPhone-sent `Dump <url>` split-send into one turn. Client-side removal once imsg coalesces upstream is tracked in #91243 (openclaw/imsg#141). Closes #90795
1 parent f2530de commit 9caff5f

8 files changed

Lines changed: 327 additions & 25 deletions

File tree

‎docs/channels/imessage.md‎

Lines changed: 15 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -654,14 +654,14 @@ When a user types a command and a URL together — e.g. `Dump https://example.co
654654

655655
The two rows arrive at OpenClaw ~0.8-2.0 s apart on most setups. Without coalescing, the agent receives the command alone on turn 1, replies (often "send me the URL"), and only sees the URL on turn 2 — at which point the command context is already lost. This is Apple's send pipeline, not anything OpenClaw or `imsg` introduces.
656656

657-
`channels.imessage.coalesceSameSenderDms` opts a DM into merging consecutive same-sender rows into a single agent turn. Group chats continue to dispatch per-message so multi-user turn structure is preserved.
657+
`channels.imessage.coalesceSameSenderDms` opts a DM into buffering consecutive same-sender rows. When `imsg` exposes the structural URL-preview marker `balloon_bundle_id: "com.apple.messages.URLBalloonProvider"` on one of the source rows, OpenClaw merges only that real split-send and keeps any other buffered rows as separate turns. On older `imsg` builds that emit no balloon metadata at all, OpenClaw cannot tell a split-send from separate sends, so it falls back to merging the bucket. That preserves the pre-metadata behavior rather than regressing `Dump <url>` split-sends into two turns. Group chats continue to dispatch per-message so multi-user turn structure is preserved.
658658

659659
<Tabs>
660660
<Tab title="When to enable">
661661
Enable when:
662662

663663
- You ship skills that expect `command + payload` in one message (dump, paste, save, queue, etc.).
664-
- Your users paste URLs, images, or long content alongside commands.
664+
- Your users paste URLs alongside commands.
665665
- You can accept the added DM turn latency (see below).
666666

667667
Leave disabled when:
@@ -702,7 +702,8 @@ The two rows arrive at OpenClaw ~0.8-2.0 s apart on most setups. Without coalesc
702702

703703
</Tab>
704704
<Tab title="Trade-offs">
705-
- **Added latency for DM messages.** With the flag on, every DM (including standalone control commands and single-text follow-ups) waits up to the debounce window before dispatching, in case a payload row is coming. Group-chat messages keep instant dispatch.
705+
- **Precise merging needs current `imsg` payload metadata.** When the URL row includes `balloon_bundle_id`, only that real split-send merges and other buffered rows stay separate. On older `imsg` builds that expose no balloon metadata, OpenClaw falls back to merging the buffered bucket so `Dump <url>` split-sends are not regressed into two turns (interim back-compat, removed once `imsg` coalesces split-sends upstream).
706+
- **Added latency for DM messages.** With the flag on, every DM (including standalone control commands and single-text follow-ups) waits up to the debounce window before dispatching, in case a URL-preview row is coming. Group-chat messages keep instant dispatch.
706707
- **Merged output is bounded.** Merged text caps at 4000 chars with an explicit `…[truncated]` marker; attachments cap at 20; source entries cap at 10 (first-plus-latest retained beyond that). Every source GUID is tracked in `coalescedMessageGuids` for downstream telemetry.
707708
- **DM-only.** Group chats fall through to per-message dispatch so the bot stays responsive when multiple people are typing.
708709
- **Opt-in, per-channel.** Other channels (Telegram, WhatsApp, Slack, …) are unaffected. Legacy BlueBubbles configs that set `channels.bluebubbles.coalesceSameSenderDms` should migrate that value to `channels.imessage.coalesceSameSenderDms`.
@@ -712,15 +713,17 @@ The two rows arrive at OpenClaw ~0.8-2.0 s apart on most setups. Without coalesc
712713

713714
### Scenarios and what the agent sees
714715

715-
| User composes | `chat.db` produces | Flag off (default) | Flag on + 2500 ms window |
716-
| ------------------------------------------------------------------ | --------------------- | --------------------------------------- | ----------------------------------------------------------------------- |
717-
| `Dump https://example.com` (one send) | 2 rows ~1 s apart | Two agent turns: "Dump" alone, then URL | One turn: merged text `Dump https://example.com` |
718-
| `Save this 📎image.jpg caption` (attachment + text) | 2 rows | Two turns (attachment dropped on merge) | One turn: text + image preserved |
719-
| `/status` (standalone command) | 1 row | Instant dispatch | **Wait up to window, then dispatch** |
720-
| URL pasted alone | 1 row | Instant dispatch | Instant dispatch (only one entry in bucket) |
721-
| Text + URL sent as two deliberate separate messages, minutes apart | 2 rows outside window | Two turns | Two turns (window expires between them) |
722-
| Rapid flood (>10 small DMs inside window) | N rows | N turns | One turn, bounded output (first + latest, text/attachment caps applied) |
723-
| Two people typing in a group chat | N rows from M senders | M+ turns (one per sender bucket) | M+ turns — group chats are not coalesced |
716+
The "Flag on" column shows behavior on an `imsg` build that emits `balloon_bundle_id`. On older `imsg` builds that emit no balloon metadata at all, the rows below marked "Two turns" / "N turns" instead fall back to a legacy merge (one turn): OpenClaw cannot structurally tell a split-send from separate sends, so it preserves the pre-metadata merge. Precise separation activates once the build emits balloon metadata.
717+
718+
| User composes | `chat.db` produces | Flag off (default) | Flag on + window (imsg emits balloon metadata) |
719+
| ------------------------------------------------------------------ | ----------------------------------- | --------------------------------------- | ------------------------------------------------ |
720+
| `Dump https://example.com` (one send) | 2 rows ~1 s apart | Two agent turns: "Dump" alone, then URL | One turn: merged text `Dump https://example.com` |
721+
| `Save this 📎image.jpg caption` (attachment + text) | 2 rows without URL balloon metadata | Two turns | Two turns (legacy merge on metadata-less builds) |
722+
| `/status` (standalone command) | 1 row | Instant dispatch | **Wait up to window, then dispatch** |
723+
| URL pasted alone | 1 row | Instant dispatch | Wait up to window, then dispatch |
724+
| Text + URL sent as two deliberate separate messages, minutes apart | 2 rows outside window | Two turns | Two turns (window expires between them) |
725+
| Rapid flood (>10 small DMs inside window) | N rows without URL balloon metadata | N turns | N turns (legacy merge on metadata-less builds) |
726+
| Two people typing in a group chat | N rows from M senders | M+ turns (one per sender bucket) | M+ turns — group chats are not coalesced |
724727

725728
## Catching up after gateway downtime
726729

‎extensions/imessage/src/monitor.last-route.test.ts‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1053,6 +1053,7 @@ describe("iMessage monitor last-route updates", () => {
10531053
id: 78,
10541054
guid: "LIVE-GUID-78",
10551055
text: "https://example.com",
1056+
balloon_bundle_id: "com.apple.messages.URLBalloonProvider",
10561057
created_at: "2026-05-22T15:30:01.000Z",
10571058
},
10581059
]) {
@@ -1105,4 +1106,154 @@ describe("iMessage monitor last-route updates", () => {
11051106
expect((await loadIMessageCatchupCursor("default"))?.lastSeenRowid).toBe(78);
11061107
});
11071108
});
1109+
1110+
it("legacy-merges coalesce buckets when imsg emits no balloon metadata (older builds)", async () => {
1111+
// Back-compat: older imsg builds emit no balloon_bundle_id, so a Dump + URL
1112+
// split-send arrives as two fieldless rows. We cannot structurally tell that
1113+
// apart from separate sends, so we preserve the pre-metadata merge rather
1114+
// than regress split-send users to two turns. Removed once imsg coalesces
1115+
// upstream (openclaw/imsg#141, tracked by #91243).
1116+
debouncerControl.holdEntries = true;
1117+
1118+
let onNotification: ((message: { method: string; params: unknown }) => void) | undefined;
1119+
const client = {
1120+
request: vi.fn(async (method: string) => {
1121+
if (method === "watch.subscribe") {
1122+
return { subscription: 1 };
1123+
}
1124+
throw new Error(`unexpected imsg method ${method}`);
1125+
}),
1126+
waitForClose: vi.fn(async () => {
1127+
for (const row of [
1128+
{ id: 91, guid: "LIVE-GUID-91", text: "Dump", created_at: "2026-05-22T15:30:00.000Z" },
1129+
{
1130+
id: 92,
1131+
guid: "LIVE-GUID-92",
1132+
text: "https://example.com",
1133+
created_at: "2026-05-22T15:30:01.000Z",
1134+
},
1135+
]) {
1136+
onNotification?.({
1137+
method: "message",
1138+
params: {
1139+
message: {
1140+
...row,
1141+
chat_id: 123,
1142+
sender: "+15550001111",
1143+
is_from_me: false,
1144+
is_group: false,
1145+
},
1146+
},
1147+
});
1148+
}
1149+
await vi.waitFor(() => {
1150+
expect(debouncerControl.flush).toBeDefined();
1151+
});
1152+
await debouncerControl.flush?.();
1153+
await Promise.resolve();
1154+
}),
1155+
stop: vi.fn(async () => {}),
1156+
};
1157+
createIMessageRpcClientMock.mockImplementation(async (params) => {
1158+
if (!params?.onNotification) {
1159+
throw new Error("expected iMessage notification handler");
1160+
}
1161+
onNotification = params.onNotification;
1162+
return client as never;
1163+
});
1164+
1165+
await monitorIMessageProvider({
1166+
config: {
1167+
channels: {
1168+
imessage: {
1169+
coalesceSameSenderDms: true,
1170+
dmPolicy: "allowlist",
1171+
allowFrom: ["+15550001111"],
1172+
sendReadReceipts: false,
1173+
},
1174+
},
1175+
messages: { inbound: { debounceMs: 2500 } },
1176+
session: { mainKey: "main" },
1177+
} as never,
1178+
runtime: { error: vi.fn(), exit: vi.fn(), log: vi.fn() },
1179+
});
1180+
1181+
expect(dispatchInboundMessageMock).toHaveBeenCalledTimes(1);
1182+
const mergedBody = dispatchInboundMessageMock.mock.calls[0]?.[0].ctx.Body ?? "";
1183+
expect(mergedBody).toContain("Dump");
1184+
expect(mergedBody).toContain("https://example.com");
1185+
});
1186+
1187+
it("merges coalesce buckets when imsg marks the URL balloon row structurally", async () => {
1188+
debouncerControl.holdEntries = true;
1189+
1190+
let onNotification: ((message: { method: string; params: unknown }) => void) | undefined;
1191+
const client = {
1192+
request: vi.fn(async (method: string) => {
1193+
if (method === "watch.subscribe") {
1194+
return { subscription: 1 };
1195+
}
1196+
throw new Error(`unexpected imsg method ${method}`);
1197+
}),
1198+
waitForClose: vi.fn(async () => {
1199+
for (const row of [
1200+
{ id: 93, guid: "LIVE-GUID-93", text: "Dump", created_at: "2026-05-22T15:30:00.000Z" },
1201+
{
1202+
id: 94,
1203+
guid: "LIVE-GUID-94",
1204+
text: "https://example.com",
1205+
balloon_bundle_id: "com.apple.messages.URLBalloonProvider",
1206+
created_at: "2026-05-22T15:30:01.000Z",
1207+
},
1208+
]) {
1209+
onNotification?.({
1210+
method: "message",
1211+
params: {
1212+
message: {
1213+
...row,
1214+
chat_id: 123,
1215+
sender: "+15550001111",
1216+
is_from_me: false,
1217+
is_group: false,
1218+
},
1219+
},
1220+
});
1221+
}
1222+
await vi.waitFor(() => {
1223+
expect(debouncerControl.flush).toBeDefined();
1224+
});
1225+
await debouncerControl.flush?.();
1226+
await Promise.resolve();
1227+
}),
1228+
stop: vi.fn(async () => {}),
1229+
};
1230+
createIMessageRpcClientMock.mockImplementation(async (params) => {
1231+
if (!params?.onNotification) {
1232+
throw new Error("expected iMessage notification handler");
1233+
}
1234+
onNotification = params.onNotification;
1235+
return client as never;
1236+
});
1237+
1238+
await monitorIMessageProvider({
1239+
config: {
1240+
channels: {
1241+
imessage: {
1242+
coalesceSameSenderDms: true,
1243+
dmPolicy: "allowlist",
1244+
allowFrom: ["+15550001111"],
1245+
sendReadReceipts: false,
1246+
},
1247+
},
1248+
messages: { inbound: { debounceMs: 2500 } },
1249+
session: { mainKey: "main" },
1250+
} as never,
1251+
runtime: { error: vi.fn(), exit: vi.fn(), log: vi.fn() },
1252+
});
1253+
1254+
expect(dispatchInboundMessageMock).toHaveBeenCalledTimes(1);
1255+
expect(dispatchInboundMessageMock.mock.calls[0]?.[0].ctx.Body).toContain(
1256+
"Dump https://example.com",
1257+
);
1258+
});
11081259
});

‎extensions/imessage/src/monitor/coalesce.test.ts‎

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,12 @@
22
import { describe, expect, it } from "vitest";
33
import {
44
combineIMessagePayloads,
5+
hasIMessageUrlBalloonBundleID,
6+
IMESSAGE_URL_BALLOON_BUNDLE_ID,
57
MAX_COALESCED_ATTACHMENTS,
68
MAX_COALESCED_ENTRIES,
79
MAX_COALESCED_TEXT_CHARS,
10+
shouldCombineIMessagePayloadBucket,
811
} from "./coalesce.js";
912
import type { IMessagePayload } from "./types.js";
1013

@@ -21,6 +24,52 @@ const makePayload = (overrides: Partial<IMessagePayload> = {}): IMessagePayload
2124
});
2225

2326
describe("combineIMessagePayloads", () => {
27+
it("recognizes URL balloon rows from imsg structural metadata", () => {
28+
const text = makePayload({ text: "Dump" });
29+
const balloon = makePayload({
30+
text: "https://example.com/article",
31+
balloon_bundle_id: IMESSAGE_URL_BALLOON_BUNDLE_ID,
32+
});
33+
34+
expect(hasIMessageUrlBalloonBundleID(text)).toBe(false);
35+
expect(hasIMessageUrlBalloonBundleID(balloon)).toBe(true);
36+
// A real URL split-send merges regardless of the session capability latch.
37+
expect(shouldCombineIMessagePayloadBucket([text, balloon], false)).toBe(true);
38+
expect(shouldCombineIMessagePayloadBucket([text, balloon], true)).toBe(true);
39+
});
40+
41+
it("falls back to a legacy merge when the build has never emitted balloon metadata (older imsg)", () => {
42+
// Older imsg builds emit no balloon_bundle_id at all. We cannot tell a URL
43+
// split-send from separate sends, so we preserve the pre-metadata merge
44+
// rather than regress split-send users to two turns. Back-compat path,
45+
// removed once imsg coalesces upstream (openclaw/imsg#141, tracked by #91243).
46+
const text = makePayload({ text: "Dump" });
47+
const url = makePayload({ text: "https://example.com/article" });
48+
expect(shouldCombineIMessagePayloadBucket([text, url], false)).toBe(true);
49+
});
50+
51+
it("keeps a plain bucket separate once the build is known to emit balloon metadata", () => {
52+
// Capability latch is true (a prior row this session carried metadata), so a
53+
// plain bucket with no URL marker is genuinely not a split-send. imsg omits
54+
// the field for plain rows, so this case is indistinguishable per-bucket and
55+
// depends on the session-level signal.
56+
const a = makePayload({ text: "first" });
57+
const b = makePayload({ text: "second" });
58+
expect(shouldCombineIMessagePayloadBucket([a, b], true)).toBe(false);
59+
});
60+
61+
it("keeps a bucket separate when imsg exposes balloon metadata in the bucket but no URL marker", () => {
62+
// New imsg surfaced balloon metadata in this very bucket, proving this build
63+
// emits the field, but the bucket is not a URL split-send. Keep separate even
64+
// if the latch had not flipped yet.
65+
const text = makePayload({ text: "hi" });
66+
const nonUrlBalloon = makePayload({
67+
text: "tap to vote",
68+
balloon_bundle_id: "com.apple.messages.MSMessageExtensionBalloonPlugin",
69+
});
70+
expect(shouldCombineIMessagePayloadBucket([text, nonUrlBalloon], false)).toBe(false);
71+
});
72+
2473
it("throws on empty input", () => {
2574
expect(() => combineIMessagePayloads([])).toThrow(
2675
"combineIMessagePayloads: cannot combine empty payloads",
@@ -44,6 +93,7 @@ describe("combineIMessagePayloads", () => {
4493
const balloon = makePayload({
4594
id: 42,
4695
text: "https://example.com/article",
96+
balloon_bundle_id: IMESSAGE_URL_BALLOON_BUNDLE_ID,
4797
guid: "row-2",
4898
created_at: "2025-01-01T00:00:01.500Z",
4999
});

‎extensions/imessage/src/monitor/coalesce.ts‎

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,60 @@ import type { IMessagePayload } from "./types.js";
1616
export const MAX_COALESCED_TEXT_CHARS = 4000;
1717
export const MAX_COALESCED_ATTACHMENTS = 20;
1818
export const MAX_COALESCED_ENTRIES = 10;
19+
export const IMESSAGE_URL_BALLOON_BUNDLE_ID = "com.apple.messages.URLBalloonProvider";
20+
21+
export function hasIMessageUrlBalloonBundleID(payload: IMessagePayload): boolean {
22+
return payload.balloon_bundle_id === IMESSAGE_URL_BALLOON_BUNDLE_ID;
23+
}
24+
25+
// imsg only emits `balloon_bundle_id` for rows that actually carry a balloon
26+
// (the nil case is omitted on the wire), so a present, non-empty value is the
27+
// signal that this build exposes balloon metadata at all.
28+
export function hasIMessageBalloonMetadata(payload: IMessagePayload): boolean {
29+
return typeof payload.balloon_bundle_id === "string" && payload.balloon_bundle_id.length > 0;
30+
}
31+
32+
/**
33+
* Decide whether a debounced same-sender bucket should merge into one turn.
34+
*
35+
* `buildEmitsBalloonMetadata` is a session-level capability latch: once any
36+
* inbound row from this imsg build has carried balloon metadata, absence of a
37+
* URL marker is meaningful (the row genuinely is not a URL split-send), so we
38+
* can keep ordinary buffered DMs separate. It must be session-scoped, not
39+
* per-bucket: imsg omits `balloon_bundle_id` on the wire for non-balloon rows,
40+
* so a bucket of plain text rows looks identical on old and new builds.
41+
*/
42+
export function shouldCombineIMessagePayloadBucket(
43+
payloads: readonly IMessagePayload[],
44+
buildEmitsBalloonMetadata: boolean,
45+
): boolean {
46+
// Precise path: a real Apple URL-preview split-send carries the URL-balloon
47+
// marker on the preview row — merge it into one turn.
48+
if (payloads.some(hasIMessageUrlBalloonBundleID)) {
49+
return true;
50+
}
51+
// Metadata-capable build (observed earlier this session or in this bucket):
52+
// the missing URL marker is trustworthy, so keep ordinary buffered DMs as
53+
// separate turns. This is the precision the structural gate exists for.
54+
if (buildEmitsBalloonMetadata || payloads.some(hasIMessageBalloonMetadata)) {
55+
return false;
56+
}
57+
// Back-compat (remove once imsg coalesces split-sends upstream — see
58+
// openclaw/imsg#141, tracked by #91243): a build that has never emitted any
59+
// balloon metadata cannot structurally tell a `Dump <url>` split-send from
60+
// separate sends. Preserve the pre-metadata merge so split-send users do not
61+
// regress to two turns on a released imsg that lacks the field.
62+
//
63+
// This never merges more than the shipped behavior already did: with
64+
// `coalesceSameSenderDms` enabled, `main` debounces every same-sender DM and
65+
// merges each multi-entry bucket unconditionally. So an unlatched session
66+
// (old build, or a metadata-capable build before its first balloon row) is
67+
// identical to today, not a new regression. Flushing these buckets instead
68+
// would re-break old-imsg split-sends — the very case this guards. Fully
69+
// closing the pre-latch window needs an imsg-advertised capability flag, which
70+
// is part of the upstream #141 work.
71+
return true;
72+
}
1973

2074
export type CoalescedIMessagePayload = IMessagePayload & {
2175
/**

0 commit comments

Comments
 (0)