Skip to content

[externalization] native image blocks bypass interceptor for assistant/tool roles #565

@100yenadmin

Description

@100yenadmin

Why this is one issue, not N

This is a single-symptom, single-fix bug — the role gating in interceptNativeUserImageBlocks is too narrow. Filing it as one issue with the extension-map gap as part of the same diff is the right scope; both are touched in the same function and the extension map is a trivial follow-on the reviewer would request anyway.

Sub-item 1 (B4): Native image blocks on assistant/tool roles bypass interceptNativeUserImageBlocks

  • Repro hypothesis: PR fix: externalize native user image blocks #521's interceptNativeUserImageBlocks is gated on role === "user". Assistant or tool messages with {type: 'image', data: ...} blocks (some MCP tools return these — e.g. screenshot/vision tools) fall through to the generic raw-payload externalizer and get stored as raw-assistant-payload.json blobs with embedded base64. The result is the exact failure mode fix: externalize native user image blocks #521 was meant to close (oversized rows containing base64 image data), just on the assistant/tool side.
  • Evidence: fix: externalize native user image blocks #521 introduced interceptNativeUserImageBlocks with an explicit role === "user" check at the entry; the generic raw-payload path (raw-assistant-payload.json) remains in place for non-user roles. Per Large raw payloads are stored/reassembled as normal messages and can dominate LCM context #492 and fix: externalize native user image blocks #521, the original symptom was multi-MB rows; this gap reproduces that for any MCP tool that returns native image blocks.
  • Suggested fix: generalize the interceptor to all roles — rename to interceptNativeImageBlocks (or drop the role gate) and route assistant/tool image blocks through the same externalization path. Additionally, extend the extension map used by the interceptor to include image/heic, image/avif, and image/bmp so non-PNG/JPEG blocks (increasingly common from vision tools) get proper extensions instead of falling back to .bin or the generic raw-payload path.

Suggested PR scope

A single PR that (a) drops the role === "user" gate in interceptNativeUserImageBlocks (renaming for clarity), (b) extends the mime-to-extension map with image/heic, image/avif, image/bmp, and (c) adds two regression tests — one assistant-role native-image fixture, one tool-role native-image fixture — both asserting the resulting row references an externalized file rather than a raw-assistant-payload.json blob with embedded base64. Should be a small diff; touches one helper plus its call sites and the extension map constant.

Cross-links

Related: #492, #521

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions