Conversation
β¦ding to vision models Vision models like Claude and Gemini don't support SVG images (image/svg+xml). Previously, SVG images were passed through unchanged, causing runtime errors. Changes: - Add supported image types check in Anthropic context builder - Add supported image types check in Google context builder - Filter out unsupported formats (like SVG) by returning undefined - Add 4 test cases for SVG filtering (base64 and URL scenarios) Supported formats: image/jpeg, image/jpg, image/png, image/gif, image/webp Closes: LOBE-4125 π€ Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Reviewer's GuideAdds MIME-type-based filtering of unsupported image formats (notably SVG) in Anthropic and Google vision context builders, returning undefined for unsupported images so they are dropped by existing filtering, and adds tests to cover both base64 and URL SVG cases. Sequence diagram for Anthropic image_url handling with MIME-type filteringsequenceDiagram
actor Caller
participant buildAnthropicBlock
participant parseDataUri
participant imageUrlToBase64
participant isImageTypeSupported
Caller->>buildAnthropicBlock: buildAnthropicBlock(content_image_url)
buildAnthropicBlock->>parseDataUri: parseDataUri(image_url.url)
parseDataUri-->>buildAnthropicBlock: type,mimeType,base64
alt type === base64
buildAnthropicBlock->>isImageTypeSupported: isImageTypeSupported(mimeType)
alt supported
buildAnthropicBlock-->>Caller: Anthropic_ImageBlockParam
else unsupported (e.g. image/svg+xml)
buildAnthropicBlock-->>Caller: undefined
end
else type === url
buildAnthropicBlock->>imageUrlToBase64: imageUrlToBase64(image_url.url)
imageUrlToBase64-->>buildAnthropicBlock: base64,mimeType
buildAnthropicBlock->>isImageTypeSupported: isImageTypeSupported(mimeType)
alt supported
buildAnthropicBlock-->>Caller: Anthropic_ImageBlockParam
else unsupported (e.g. image/svg+xml)
buildAnthropicBlock-->>Caller: undefined
end
end
Sequence diagram for Google image_url handling with MIME-type filteringsequenceDiagram
actor Caller
participant buildGooglePart
participant parseDataUri
participant imageUrlToBase64
participant isImageTypeSupported
Caller->>buildGooglePart: buildGooglePart(content_image_url)
buildGooglePart->>parseDataUri: parseDataUri(image_url.url)
parseDataUri-->>buildGooglePart: type,mimeType,base64
alt type === base64
buildGooglePart->>isImageTypeSupported: isImageTypeSupported(mimeType)
alt supported
buildGooglePart-->>Caller: Google_Part_with_inlineData
else unsupported (e.g. image/svg+xml)
buildGooglePart-->>Caller: undefined
end
else type === url
buildGooglePart->>imageUrlToBase64: imageUrlToBase64(image_url.url)
imageUrlToBase64-->>buildGooglePart: base64,mimeType
buildGooglePart->>isImageTypeSupported: isImageTypeSupported(mimeType)
alt supported
buildGooglePart-->>Caller: Google_Part_with_inlineData
else unsupported (e.g. image/svg+xml)
buildGooglePart-->>Caller: undefined
end
end
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
TestGru AssignmentSummary
Files
Tip You can |
There was a problem hiding this comment.
Hey - I've left some high level feedback:
- The
isImageTypeSupportedhelper and supported image type sets are duplicated between the Anthropic and Google context builders; consider extracting a shared utility (with per-provider allowlists if needed) so future changes to supported formats stay consistent. - The implementation currently treats a
nullmimeType as supported inisImageTypeSupported, which seems to contradict the intent to filter out non-standard/unsupported formats; you might want to treatnullas unsupported (or only allow it in the specific base64 fallback case) to avoid silently passing unknown types through.
Prompt for AI Agents
Please address the comments from this code review:
## Overall Comments
- The `isImageTypeSupported` helper and supported image type sets are duplicated between the Anthropic and Google context builders; consider extracting a shared utility (with per-provider allowlists if needed) so future changes to supported formats stay consistent.
- The implementation currently treats a `null` mimeType as supported in `isImageTypeSupported`, which seems to contradict the intent to filter out non-standard/unsupported formats; you might want to treat `null` as unsupported (or only allow it in the specific base64 fallback case) to avoid silently passing unknown types through.Help me be more useful! Please click π or π on each comment and I'll use the feedback to improve your reviews.
There was a problem hiding this comment.
π‘ Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 52a70410f6
βΉοΈ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with π.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const isImageTypeSupported = (mimeType: string | null): boolean => { | ||
| if (!mimeType) return true; | ||
| return ANTHROPIC_SUPPORTED_IMAGE_TYPES.has(mimeType.toLowerCase()); |
There was a problem hiding this comment.
Normalize MIME parameters before whitelist check
When the image comes from a URL, imageUrlToBase64 returns blob.type, which can include parameters (e.g., image/png; charset=binary). The new isImageTypeSupported does an exact match against the whitelist, so valid images with parameters are treated as unsupported and dropped (undefined). This can silently strip otherwise supported images (or entire messages) from Anthropic/Gemini requests when servers add MIME parameters. Consider normalizing by splitting on ; before the set lookup.
Useful? React with πΒ / π.
Codecov Reportβ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## next #11698 +/- ##
=========================================
Coverage 74.07% 74.08%
=========================================
Files 1188 1188
Lines 94668 94695 +27
Branches 10429 12470 +2041
=========================================
+ Hits 70126 70151 +25
- Misses 24452 24454 +2
Partials 90 90
Flags with carried forward coverage won't be shown. Click here to find out more.
π New features to boost your workflow:
|
|
β€οΈ Great PR @arvinxx β€οΈ The growth of project is inseparable from user feedback and contribution, thanks for your contribution! If you are interesting with the lobehub developer community, please join our discord and then dm @arvinxx or @canisminor1990. They will invite you to our private developer channel. We are talking about the lobe-chat development or sharing ai newsletter around the world. |
## [Version 2.0.0-next.339](v2.0.0-next.338...v2.0.0-next.339) <sup>Released on **2026-01-22**</sup> #### β» Code Refactoring - **misc**: Move vercel-react-best-practices skills to .agents directory. #### β¨ Features - **misc**: Skill setting page and skill store. #### π Bug Fixes - **model-runtime**: Filter unsupported image types (SVG) before sending to vision models. - **misc**: Fix group broadcast trigger tool use, fix local system tools. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### Code refactoring * **misc**: Move vercel-react-best-practices skills to .agents directory, closes [#11703](#11703) ([6df7731](6df7731)) #### What's improved * **misc**: Skill setting page and skill store, closes [#11665](#11665) ([d8c0c26](d8c0c26)) #### What's fixed * **model-runtime**: Filter unsupported image types (SVG) before sending to vision models, closes [#11698](#11698) ([c0c99a7](c0c99a7)) * **misc**: Fix group broadcast trigger tool use, closes [#11646](#11646) ([831a9b3](831a9b3)) * **misc**: Fix local system tools, closes [#11702](#11702) ([6548fc7](6548fc7)) </details> <div align="right"> [](#readme-top) </div>
|
π This PR is included in version 2.0.0-next.339 π The release is available on: Your semantic-release bot π¦π |
## [Version 1.153.0](v1.152.0...v1.153.0) <sup>Released on **2026-01-23**</sup> #### β» Code Refactoring - **auth**: Remove NEXT_PUBLIC_AUTH_URL env variable. - **model-select**: Migrate FunctionCallingModelSelect to LobeSelect. - **ModelSwitchPanel**: Migrate from Popover to DropdownMenu with virtual scrolling. - **userMemories**: Removed un-used code. - **misc**: Improve memory data with experience and identity, move vercel-react-best-practices skills to .agents directory. #### β¨ Features - **database**: Added user memory activity. - **desktop**: Add legacy local database detection and migration guidance. - **misc**: Add platform-aware download client menu option, add server version check for desktop app, remove Clerk authentication code, skill setting page and skill store, support agent group unpublish agents, support client tasks mode, update the sandbox preinstall libs in sys role. #### π Bug Fixes - **copilot**: Pass correct scope when creating new session in PageEditor. - **desktop**: Gracefully handle missing update manifest 404 errors. - **model-runtime**: Filter unsupported image types (SVG) before sending to vision models. - **pdf**: Upgrade pdfjs-dist and react-pdf to v5.x. - **sidebar-drawer**: Fix drawer positioning and title style. - **misc**: Fix group broadcast trigger tool use, fix local system tools, fix memory schema, fix multi agent tasks issue, fix multi tasks no summary issue, fix scope issue, fix tool argument scape and improve multi task run, fixed the sandbox tools call when error should use right callback, improve e2e server and complete i18n resources, slove the agent group editor not focus in editdata area, slove the agents header switch agents the lobeAI not show problem, sloved the old removeSessionTopics not work, TypewriterEffect not refreshing on language change, updata cron job ui & fixed commnuity pagenation goto error, update the agentbuilder tools not always use humanIntervention. #### π Styles - **misc**: Improve auto scroll and group profile, update og, update share style. <br/> <details> <summary><kbd>Improvements and Fixes</kbd></summary> #### Code refactoring * **auth**: Remove NEXT_PUBLIC_AUTH_URL env variable, closes [lobehub#11658](https://github.com/jaworldwideorg/OneJA-Bot/issues/11658) ([c0f9875](c0f9875)) * **model-select**: Migrate FunctionCallingModelSelect to LobeSelect, closes [lobehub#11664](https://github.com/jaworldwideorg/OneJA-Bot/issues/11664) ([ad51305](ad51305)) * **ModelSwitchPanel**: Migrate from Popover to DropdownMenu with virtual scrolling, closes [lobehub#11663](https://github.com/jaworldwideorg/OneJA-Bot/issues/11663) ([c9d9dff](c9d9dff)) * **userMemories**: Removed un-used code, closes [lobehub#11713](https://github.com/jaworldwideorg/OneJA-Bot/issues/11713) ([89750fc](89750fc)) * **misc**: Improve memory data with experience and identity, closes [lobehub#11717](https://github.com/jaworldwideorg/OneJA-Bot/issues/11717) ([bdb3eb4](bdb3eb4)) * **misc**: Move vercel-react-best-practices skills to .agents directory, closes [lobehub#11703](https://github.com/jaworldwideorg/OneJA-Bot/issues/11703) ([6df7731](6df7731)) #### What's improved * **database**: Added user memory activity, closes [lobehub#11680](https://github.com/jaworldwideorg/OneJA-Bot/issues/11680) ([0160fbd](0160fbd)) * **desktop**: Add legacy local database detection and migration guidance, closes [lobehub#11682](https://github.com/jaworldwideorg/OneJA-Bot/issues/11682) ([5664b84](5664b84)) * **misc**: Add platform-aware download client menu option, closes [lobehub#11676](https://github.com/jaworldwideorg/OneJA-Bot/issues/11676) ([55abddc](55abddc)) * **misc**: Add server version check for desktop app, closes [lobehub#11710](https://github.com/jaworldwideorg/OneJA-Bot/issues/11710) ([0cf2723](0cf2723)) * **misc**: Remove Clerk authentication code, closes [lobehub#11711](https://github.com/jaworldwideorg/OneJA-Bot/issues/11711) ([395595a](395595a)) * **misc**: Skill setting page and skill store, closes [lobehub#11665](https://github.com/jaworldwideorg/OneJA-Bot/issues/11665) ([d8c0c26](d8c0c26)) * **misc**: Support agent group unpublish agents, closes [lobehub#11687](https://github.com/jaworldwideorg/OneJA-Bot/issues/11687) ([4e060be](4e060be)) * **misc**: Support client tasks mode, closes [lobehub#11666](https://github.com/jaworldwideorg/OneJA-Bot/issues/11666) ([98cf57b](98cf57b)) * **misc**: Update the sandbox preinstall libs in sys role, closes [lobehub#11688](https://github.com/jaworldwideorg/OneJA-Bot/issues/11688) ([404c577](404c577)) #### What's fixed * **copilot**: Pass correct scope when creating new session in PageEditor, closes [lobehub#11714](https://github.com/jaworldwideorg/OneJA-Bot/issues/11714) ([0259270](0259270)) * **desktop**: Gracefully handle missing update manifest 404 errors, closes [lobehub#11625](https://github.com/jaworldwideorg/OneJA-Bot/issues/11625) ([13e95b9](13e95b9)) * **model-runtime**: Filter unsupported image types (SVG) before sending to vision models, closes [lobehub#11698](https://github.com/jaworldwideorg/OneJA-Bot/issues/11698) ([c0c99a7](c0c99a7)) * **pdf**: Upgrade pdfjs-dist and react-pdf to v5.x, closes [lobehub#11686](https://github.com/jaworldwideorg/OneJA-Bot/issues/11686) ([2b620df](2b620df)) * **sidebar-drawer**: Fix drawer positioning and title style, closes [lobehub#11655](https://github.com/jaworldwideorg/OneJA-Bot/issues/11655) ([cf5320e](cf5320e)) * **misc**: Fix group broadcast trigger tool use, closes [lobehub#11646](https://github.com/jaworldwideorg/OneJA-Bot/issues/11646) ([831a9b3](831a9b3)) * **misc**: Fix local system tools, closes [lobehub#11702](https://github.com/jaworldwideorg/OneJA-Bot/issues/11702) ([6548fc7](6548fc7)) * **misc**: Fix memory schema, closes [lobehub#11645](https://github.com/jaworldwideorg/OneJA-Bot/issues/11645) ([3baf780](3baf780)) * **misc**: Fix multi agent tasks issue, closes [lobehub#11672](https://github.com/jaworldwideorg/OneJA-Bot/issues/11672) ([9de773b](9de773b)) * **misc**: Fix multi tasks no summary issue, closes [lobehub#11685](https://github.com/jaworldwideorg/OneJA-Bot/issues/11685) ([26ce317](26ce317)) * **misc**: Fix scope issue, closes [lobehub#11719](https://github.com/jaworldwideorg/OneJA-Bot/issues/11719) ([17adde8](17adde8)) * **misc**: Fix tool argument scape and improve multi task run, closes [lobehub#11691](https://github.com/jaworldwideorg/OneJA-Bot/issues/11691) ([b13bb8a](b13bb8a)) * **misc**: Fixed the sandbox tools call when error should use right callback, closes [lobehub#11721](https://github.com/jaworldwideorg/OneJA-Bot/issues/11721) ([e8fce68](e8fce68)) * **misc**: Improve e2e server and complete i18n resources, closes [lobehub#11678](https://github.com/jaworldwideorg/OneJA-Bot/issues/11678) ([d450dd9](d450dd9)) * **misc**: Slove the agent group editor not focus in editdata area, closes [lobehub#11677](https://github.com/jaworldwideorg/OneJA-Bot/issues/11677) ([9ac84e6](9ac84e6)) * **misc**: Slove the agents header switch agents the lobeAI not show problem, closes [lobehub#11726](https://github.com/jaworldwideorg/OneJA-Bot/issues/11726) ([f45f508](f45f508)) * **misc**: Sloved the old removeSessionTopics not work, closes [lobehub#11671](https://github.com/jaworldwideorg/OneJA-Bot/issues/11671) ([06d41e5](06d41e5)) * **misc**: TypewriterEffect not refreshing on language change, closes [lobehub#11657](https://github.com/jaworldwideorg/OneJA-Bot/issues/11657) ([ba30f46](ba30f46)) * **misc**: Updata cron job ui & fixed commnuity pagenation goto error, closes [lobehub#11700](https://github.com/jaworldwideorg/OneJA-Bot/issues/11700) ([42ad2a0](42ad2a0)) * **misc**: Update the agentbuilder tools not always use humanIntervention, closes [lobehub#11696](https://github.com/jaworldwideorg/OneJA-Bot/issues/11696) ([0d3017b](0d3017b)) #### Styles * **misc**: Improve auto scroll and group profile, closes [lobehub#11725](https://github.com/jaworldwideorg/OneJA-Bot/issues/11725) ([550acc2](550acc2)) * **misc**: Update og, closes [lobehub#11709](https://github.com/jaworldwideorg/OneJA-Bot/issues/11709) ([01cf4e4](01cf4e4)) * **misc**: Update share style, closes [lobehub#11716](https://github.com/jaworldwideorg/OneJA-Bot/issues/11716) ([3c70dfa](3c70dfa)) </details> <div align="right"> [](#readme-top) </div>
Summary
undefinedfor unsupported formats, which gets filtered out by existingfilter(Boolean)logicProblem
Vision models like Claude and Gemini don't support SVG images (
image/svg+xml). Previously, SVG images were converted to base64 and passed through unchanged, causing runtime errors when the model API rejected the unsupported format.Solution
Added MIME type validation at the context builder level:
Supported formats:
image/jpegimage/jpgimage/pngimage/gifimage/webpUnsupported formats (filtered out):
image/svg+xmlChanges
packages/model-runtime/src/core/contextBuilders/anthropic.ts: Add image type validationpackages/model-runtime/src/core/contextBuilders/google.ts: Add image type validationpackages/model-runtime/src/core/contextBuilders/anthropic.test.ts: Add 2 test casespackages/model-runtime/src/core/contextBuilders/google.test.ts: Add 2 test casesTest plan
Related Issues
Closes: LOBE-4125
π€ Generated with Claude Code
Summary by Sourcery
Filter unsupported image formats from vision model context builders to avoid runtime errors when sending images to Anthropic and Google models.
Bug Fixes:
Tests: