Skip to content

🔨 chore(google): Support External URL file input with SSRF validation to optimize transmission#12657

Merged
tjx666 merged 13 commits into
lobehub:canaryfrom
sxjeru:34444
Jun 9, 2026
Merged

🔨 chore(google): Support External URL file input with SSRF validation to optimize transmission#12657
tjx666 merged 13 commits into
lobehub:canaryfrom
sxjeru:34444

Conversation

@sxjeru

@sxjeru sxjeru commented Mar 4, 2026

Copy link
Copy Markdown
Contributor

… to optimize transmission

💻 Change Type

  • ✨ feat
  • 🐛 fix
  • ♻️ refactor
  • 💄 style
  • 👷 build
  • ⚡️ perf
  • ✅ test
  • 📝 docs
  • 🔨 chore

🔗 Related Issue

🔀 Description of Change

https://ai.google.dev/gemini-api/docs/file-input-methods#external-urls

添加对 External URL 文件输入方式的支持。非常适配目前使用 s3 的 LobeChat .
向 Gemini 上传图片和 PDF 文件不再需要转成 base64 传输,可减少服务端出口流量。

目前测试仅 Gemini 3 可用,但文档称 Gemini 2.5 也可用,目前添加了模型名限制,后续可再行观察。


同时将视频限制提升到 100 MB(Gemini 将内嵌文件大小由 20MB 提升到了 100MB)。

🧪 How to Test

  • Tested locally
  • Added/updated tests
  • No tests needed

📸 Screenshots / Videos

Before After
... ...

📝 Additional Information

@vercel

vercel Bot commented Mar 4, 2026

Copy link
Copy Markdown

@sxjeru is attempting to deploy a commit to the LobeHub OSS Team on Vercel.

A member of the Team first needs to authorize it.

@sourcery-ai sourcery-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @sxjeru, you have reached your weekly rate limit of 500000 diff characters.

Please try again later or upgrade to continue using Sourcery

@codecov

codecov Bot commented Mar 4, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 75.95628% with 44 lines in your changes missing coverage. Please review.
✅ Project coverage is 70.50%. Comparing base (0a6b02c) to head (cb28e79).
⚠️ Report is 3 commits behind head on canary.

Additional details and impacted files
@@            Coverage Diff             @@
##           canary   #12657      +/-   ##
==========================================
- Coverage   70.50%   70.50%   -0.01%     
==========================================
  Files        3312     3312              
  Lines      327060   327258     +198     
  Branches    34721    35719     +998     
==========================================
+ Hits       230582   230721     +139     
- Misses      96296    96354      +58     
- Partials      182      183       +1     
Flag Coverage Δ
app 61.23% <14.28%> (-0.01%) ⬇️
database 92.24% <ø> (ø)
packages/agent-manager-runtime 49.69% <ø> (ø)
packages/agent-runtime 81.06% <ø> (ø)
packages/builtin-tool-lobe-agent 18.52% <ø> (ø)
packages/context-engine 84.12% <ø> (ø)
packages/conversation-flow 91.29% <ø> (ø)
packages/device-gateway-client 90.18% <ø> (ø)
packages/eval-dataset-parser 95.15% <ø> (ø)
packages/eval-rubric 76.11% <ø> (ø)
packages/fetch-sse 87.28% <ø> (+1.71%) ⬆️
packages/file-loaders 87.89% <ø> (ø)
packages/memory-user-memory 74.99% <ø> (ø)
packages/model-bank 99.99% <ø> (ø)
packages/model-runtime 84.19% <78.16%> (-0.04%) ⬇️
packages/prompts 72.51% <ø> (ø)
packages/python-interpreter 92.90% <ø> (ø)
packages/ssrf-safe-fetch 0.00% <ø> (ø)
packages/types 35.25% <ø> (ø)
packages/utils 85.06% <100.00%> (+<0.01%) ⬆️
packages/web-crawler 88.08% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Components Coverage Δ
Store 68.23% <100.00%> (-0.01%) ⬇️
Services 54.21% <ø> (ø)
Server 71.31% <ø> (ø)
Libs 55.82% <ø> (-0.14%) ⬇️
Utils 81.93% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 17f267d76b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread packages/model-runtime/src/core/contextBuilders/google.ts
Comment thread packages/model-runtime/src/core/contextBuilders/google.ts
@sxjeru

sxjeru commented Mar 4, 2026

Copy link
Copy Markdown
Contributor Author

Confirmed working.

image image

This comment was translated by Claude.

Original Content

确认可用。

image image

Copilot AI review requested due to automatic review settings March 18, 2026 13:15

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for Google Gemini “External URL” file inputs (with SSRF-safe URL validation) to reduce base64 re-uploading, and increases the client-side video size limit to 100MB to match updated Gemini limits.

Changes:

  • Add external URL validation utilities (HEAD + SSRF filtering) and integrate them into the Google context builder to prefer fileData.fileUri on Gemini 3+ (fallback to inline base64 when invalid).
  • Increase video file size validation limit from 20MB to 100MB and update related tests and i18n strings.
  • Add @lobechat/ssrf-safe-fetch as a dependency for model-runtime.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 6 comments.

Show a summary per file
File Description
src/locales/default/chat.ts Updates default locale string for the new 100MB video limit.
packages/utils/src/client/videoValidation.ts Raises client-side video size validation limit to 100MB.
packages/utils/src/client/videoValidation.test.ts Updates/extends tests to reflect the 100MB limit.
packages/model-runtime/src/utils/uriParser.ts Adds SSRF-safe external URL validation helpers and Gemini file size/type constants.
packages/model-runtime/src/providers/google/index.ts Passes model info into message building to enable model-gated external URL behavior.
packages/model-runtime/src/core/contextBuilders/google.ts Uses validated external URLs (fileData.fileUri) for Gemini 3+ when possible; fallback to inline data.
packages/model-runtime/src/core/contextBuilders/google.test.ts Adds tests for external URL behavior and fallback paths.
packages/model-runtime/package.json Adds @lobechat/ssrf-safe-fetch dependency.
locales/zh-CN/chat.json Updates zh-CN string for the 100MB video limit.
locales/en-US/chat.json Updates en-US string for the 100MB video limit.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread packages/model-runtime/src/utils/uriParser.ts
Comment thread packages/model-runtime/src/utils/uriParser.ts Outdated
Comment thread packages/model-runtime/src/utils/uriParser.ts
Comment thread packages/utils/src/client/videoValidation.test.ts Outdated
Comment thread locales/en-US/chat.json Outdated
Comment thread packages/model-runtime/src/core/contextBuilders/google.ts Outdated
@dosubot dosubot Bot added the size:XL This PR changes 500-999 lines, ignoring generated files. label May 18, 2026
@dosubot dosubot Bot added size:L This PR changes 100-499 lines, ignoring generated files. and removed size:XL This PR changes 500-999 lines, ignoring generated files. labels May 21, 2026
@sxjeru sxjeru mentioned this pull request Jun 2, 2026
12 tasks
@tjx666

tjx666 commented Jun 3, 2026

Copy link
Copy Markdown
Member

The tests failed.


This comment was translated by Claude.

Original Content

测试挂了

@tjx666 tjx666 merged commit 77dbe4b into lobehub:canary Jun 9, 2026
18 of 21 checks passed
@lobehubbot

Copy link
Copy Markdown
Member

❤️ Great PR @sxjeru ❤️

The growth of project is inseparable from user feedback and contribution, thanks for your contribution! If you are interesting with the lobehub developer community, please join our discord and then dm @arvinxx or @canisminor1990. They will invite you to our private developer channel. We are talking about the lobe-chat development or sharing ai newsletter around the world.

@sxjeru sxjeru deleted the 34444 branch June 9, 2026 08:34
arvinxx added a commit that referenced this pull request Jun 10, 2026
# 🚀 LobeHub Release (20260610)

**Release Date:** June 10, 2026  
**Since v2.2.2:** 131 merged PRs · 13 contributors

> This weekly release strengthens agent collaboration across cloud,
desktop, CLI, and workspace flows, with steadier runtime behavior and a
broader foundation for workspace-scoped data.

---

## ✨ Highlights

- **Agent execution across devices** — Unifies per-device working
directories, project skill discovery, and sub-agent suspend/resume
behavior across server, QStash, and device RPC flows. (#15543, #15566,
#15481, #15620, #15591)
- **Connector and sandbox platform** — Expands connector permissions,
custom OAuth MCP connector onboarding, sandbox provider support, and
user-uploaded file sync into cloud sandbox runs. (#15463, #15546,
#15184, #15550)
- **Desktop and CLI reliability** — Fixes desktop cold-start,
auto-update, Windows build, CLI skill discovery, and `lh connect` agent
dispatch paths. (#15547, #15525, #15527, #15562, #15632, #15634)
- **Pages and sharing** — Refreshes topic sharing, improves Page Editor
layout behavior, and routes Page Agent tool execution through the
server-side editor path. (#15581, #15556, #15588, #15023, #15610)
- **Model availability and provider updates** — Adds user-scoped LobeHub
model availability, Claude Fable 5, Qwen thinking preservation, and
MiniMax M3 updates. (#15590, #15639, #13494, #15376)

---

## 🏗️ Core Product & Architecture

### Agent Runtime & Heterogeneous Agents

- Improves sub-agent lifecycle handling, including async suspend/resume,
queue-mode QStash resume delivery, and blocking nested sub-agent calls.
(#15481, #15620, #15575)
- Stabilizes heterogeneous agent ingestion and streaming with raw stream
dumps, per-turn usage, image forwarding on regenerate, and
duplicate-text fixes. (#15602, #15577, #15592, #15585)
- Adds execution-device and working-directory controls across device
RPC, legacy defaults, and remote-spawned Claude Code sessions. (#15543,
#15566, #15591, #15572)
- Improves runtime diagnostics and compatibility, including Gemini
multimodal output capture, abort stream semantics, and trace quality
analysis. (#15535, #13677, #15508)

---

## 📱 Platforms, Integrations & UX

### Connectors, Sandbox & Tools

- Ships API-level connector tool permissions, custom OAuth MCP connector
onboarding, and connector-first runtime execution. (#15463, #15546)
- Adds sandbox provider support, cloud sandbox file sync, and safer
external URL file input handling with SSRF validation. (#15184, #15550,
#12657)
- Improves tool visibility and execution with pinned app-fixed tools,
ANSI output rendering, gateway-tunneled MCP calls, and automatic
headless tool runs. (#15509, #15516, #15469, #15492)

### Desktop, CLI & Web UX

- Restores desktop startup and reload behavior, preserves IPC error
causes, and keeps the tab bar new-tab action visible across routes.
(#15547, #15597, #15638)
- Fixes desktop update and build stability for browser quit guards,
macOS update signing, and Windows Visual Studio detection. (#15525,
#15527, #15562)
- Shows the plan-limit upgrade UI on desktop builds. (#15628)
- Adds the Agent Run delivery checker and fixes CLI device dispatch plus
skill list/search output. (#15489, #15634, #15632)
- Refreshes onboarding, auth source preservation, topic UI states,
referral/Fable campaign copy, and chat-input control bar behavior.
(#15629, #15544, #15573, #15614, #15616, #15617, #15622, #15643)

---

## 🔒 Security, Reliability & Rollout Notes

- External URL file input now includes SSRF validation for safer Google
file handling. (#12657)
- Database workspace-scope migrations are part of this release;
self-hosted operators should run the normal migration path before
serving the updated app. (#15446, #15465, #15468, #15472)
- The release branch was re-cut from `canary` and includes the latest
`main` release-version commit so `v2.2.2` is the verified compare base.

---

## 👥 Contributors

@ONLY-yours, @sxjeru, @hardy-one, @xujingli, @hezhijie0327, @Coooolfan,
@arvinxx, @tjx666, @Innei, @rivertwilight, @rdmclin2, @cy948,
@AmAzing129

**Full Changelog**:
v2.2.2...release/weekly-20260610-recut-3
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:L This PR changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants