fix(imessage): make inbound image attachments readable by agents#78580
Conversation
d612cdf to
f715416
Compare
byungskers
left a comment
There was a problem hiding this comment.
Nice work on the HEIC-to-JPEG conversion path! A few observations:
-
sips dependency: The conversion relies on macOS
sipsbeing available. Should there be a runtime check or fallback whensipsis missing (e.g., non-macOS hosts or sandboxed environments)? -
Error handling for conversion failures: If
sipsfails or returns a non-zero exit code, the current path returns an empty attachments array. Would it be better to preserve the original HEIC path as a fallback so the agent at least sees something, even if downstream tools can't decode it? -
Cleanup on failure: If
sipspartially writes a corrupted JPEG before failing, should the temp output file be cleaned up to avoid polluting the inbound media directory?
Tests look solid and the real behavior proof is thorough. Just these edge cases to consider.
|
Codex review: needs maintainer review before merge. Summary Reproducibility: yes. Current main is source-reproducible because it forwards validated iMessage original_path values into agent media fields, and the discussion includes real v2026.5.12-beta.3 HEIC evidence showing that exact raw Messages path reached the prompt. Real behavior proof Next step before merge Security Review detailsBest possible solution: Land the plugin-owned staging/conversion fix after normal maintainer merge gates, keeping core media root policy unchanged. Do we have a high-confidence way to reproduce the issue? Yes. Current main is source-reproducible because it forwards validated iMessage original_path values into agent media fields, and the discussion includes real v2026.5.12-beta.3 HEIC evidence showing that exact raw Messages path reached the prompt. Is this the best way to solve the issue? Yes. The PR keeps the fix at the iMessage plugin boundary, stages only after dispatch policy and loop guards, preserves remote-host raw paths, and avoids widening generic image-tool safe roots. What I checked:
Likely related people:
Codex review notes: model gpt-5.5, reasoning high; reviewed against 2a67a7f65e26. |
f715416 to
8c8b623
Compare
|
Updated the PR for the security review. Changes pushed in
On the edge-case questions from @byungskers:
Validated after the update:
Re-review progress:
|
8c8b623 to
8b5385d
Compare
8b5385d to
460dd6d
Compare
460dd6d to
a6b2502
Compare
|
@clawsweeper re-review |
Confirming the bug shape on a real inbound HEIC (v2026.5.12-beta.3)Pulled from a separate machine running OpenClaw chat.db row for a real inbound iPhone HEIC ( {
"transfer_name": "IMG_9372.heic",
"original_path": "~/Library/Messages/Attachments/d7/07/F92CCA7E-3434-4B5E-BCAA-7EF25D83FB0F/IMG_9372.heic",
"mime_type": "image/heic",
"uti": "public.heic"
}What the unpatched monitor handed to the agent (from session JSONL — same text content the gpt-5.5 prompt received): So the agent was handed a raw The PR's Fix LGTM — addresses the observed delivery shape on a real HEIC. (I'll send a fresh HEIC end-to-end through this gateway and post the after-the-merge confirmation once #78580 lands here.) Re-review progress:
|
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
|
Follow-up pushed in Changes:
Validated locally:
@clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
adf93ea to
97ddb8f
Compare
97ddb8f to
0fac83f
Compare
|
Rebased again on current New head: Validated locally after the rebase:
GitHub now reports the PR as mergeable again; checks are rerunning on the new head. @clawsweeper re-review |
|
🦞🧹 I asked ClawSweeper to review this item again. Re-review progress:
|
omarshahine
left a comment
There was a problem hiding this comment.
LGTM. The fix stays inside the iMessage plugin boundary, stages only after inbound policy/loop guards, keeps remote-host raw path behavior, and has focused coverage for staging, HEIC conversion, and drop-policy behavior.
Merge evidence checked: ClawSweeper reports no blocking findings with sufficient real behavior proof, live maintainer proof confirms the raw HEIC bug shape on v2026.5.12-beta.3, and the current head 0fac83f has green CI/status rollup for lint, tests, docs, boundaries, build smoke, security, and real behavior proof.
|
Landed via squash merge.
Thanks @homer-byte. |
) Stage native iMessage inbound attachments into managed media and convert HEIC/HEIF images to JPEG before dispatch. Thanks @homer-byte. (cherry picked from commit 1d6e5f7)
…nclaw#78580) Stage native iMessage inbound attachments into managed media and convert HEIC/HEIF images to JPEG before dispatch. Thanks @homer-byte.
…nclaw#78580) Stage native iMessage inbound attachments into managed media and convert HEIC/HEIF images to JPEG before dispatch. Thanks @homer-byte.
…nclaw#78580) Stage native iMessage inbound attachments into managed media and convert HEIC/HEIF images to JPEG before dispatch. Thanks @homer-byte.
…nclaw#78580) Stage native iMessage inbound attachments into managed media and convert HEIC/HEIF images to JPEG before dispatch. Thanks @homer-byte.
Summary
sipsso image-understanding tools can read iPhone photosProblem
Native iMessage image attachments were not reliably getting through to agents/tools. In practice, iPhone screenshots/photos could arrive as raw
~/Library/Messages/Attachments/...HEIC paths; those paths are outside the image tool's safe media roots, and HEIC decoding can also fail depending on the local image backend. The result was that users could send an iMessage image, but the agent could not actually inspect it without manual copying/conversion.Real behavior proof
~/Library/Messages/Attachments/...paths, which downstream image tooling could not safely read or decode. This patch stages local native iMessage attachments into OpenClaw-managed inbound media and converts HEIC/HEIF to JPEG before dispatch.MACMINI-2026, using a real inbound iMessage.heicfile previously captured in OpenClaw media storage.stageIMessageAttachmentscode path viapnpm exec tsx tmp-openclaw-imessage-stage-proof.ts /Users/openclaw/.openclaw/media/inbound/6f7b474f-4798-4619-994f-3566c5b472e1.heic, then verified the staged output withfile, and sent the staged JPEG through the image-understanding tool.contentType: "image/jpeg";fileconfirmed a decodable 4096x2304 JPEG, and the image-understanding tool successfully decoded and described the staged JPEG.Tests
pnpm exec oxfmt --check --threads=1 extensions/imessage/src/monitor/monitor-provider.ts extensions/imessage/src/monitor/media-staging.ts extensions/imessage/src/monitor/media-staging.test.ts extensions/imessage/src/monitor/types.ts CHANGELOG.mdpnpm exec oxlint extensions/imessage/src/monitor/monitor-provider.ts extensions/imessage/src/monitor/media-staging.ts extensions/imessage/src/monitor/media-staging.test.ts extensions/imessage/src/monitor/types.tspnpm test extensions/imessagepnpm check:changed