feat(google): add support for image search, replace obsolete google_search_retrieval implementation#12926
Merged
felixarntz merged 9 commits intomainfrom Mar 4, 2026
Merged
Conversation
shaper
approved these changes
Mar 4, 2026
Contributor
|
|
Contributor
felixarntz
added a commit
that referenced
this pull request
Mar 5, 2026
…e google_search_retrieval implementation (#13070) This is an automated backport of #12926 to the release-v5.0 branch. FYI @felixarntz ~~This backport has conflicts that need to be resolved manually.~~ Conflicts resolved. ### `git cherry-pick` output ``` Auto-merging content/providers/01-ai-sdk-providers/15-google-generative-ai.mdx CONFLICT (content): Merge conflict in content/providers/01-ai-sdk-providers/15-google-generative-ai.mdx CONFLICT (file location): examples/ai-functions/src/generate-text/google/image-search.ts added in 2565e70 (feat(google): add support for image search, replace obsolete google_search_retrieval implementation (#12926)) inside a directory that was renamed in HEAD, suggesting it should perhaps be moved to examples/ai-core/src/generate-text/image-search.ts. CONFLICT (file location): examples/ai-functions/src/generate-text/google/search-types.ts added in 2565e70 (feat(google): add support for image search, replace obsolete google_search_retrieval implementation (#12926)) inside a directory that was renamed in HEAD, suggesting it should perhaps be moved to examples/ai-core/src/generate-text/search-types.ts. CONFLICT (file location): examples/ai-functions/src/stream-text/google/image-search.ts added in 2565e70 (feat(google): add support for image search, replace obsolete google_search_retrieval implementation (#12926)) inside a directory that was renamed in HEAD, suggesting it should perhaps be moved to examples/ai-core/src/stream-text/image-search.ts. CONFLICT (file location): examples/ai-functions/src/stream-text/google/search-types.ts added in 2565e70 (feat(google): add support for image search, replace obsolete google_search_retrieval implementation (#12926)) inside a directory that was renamed in HEAD, suggesting it should perhaps be moved to examples/ai-core/src/stream-text/search-types.ts. Auto-merging packages/google/src/google-generative-ai-language-model.test.ts CONFLICT (content): Merge conflict in packages/google/src/google-generative-ai-language-model.test.ts Auto-merging packages/google/src/google-generative-ai-language-model.ts Auto-merging packages/google/src/google-prepare-tools.test.ts Auto-merging packages/google/src/google-prepare-tools.ts CONFLICT (content): Merge conflict in packages/google/src/google-prepare-tools.ts Auto-merging packages/google/src/tool/google-search.ts CONFLICT (content): Merge conflict in packages/google/src/tool/google-search.ts error: could not apply 2565e70... feat(google): add support for image search, replace obsolete google_search_retrieval implementation (#12926) hint: After resolving the conflicts, mark them with hint: "git add/rm <pathspec>", then run hint: "git cherry-pick --continue". hint: You can instead skip this commit with "git cherry-pick --skip". hint: To abort and get back to the state before "git cherry-pick", hint: run "git cherry-pick --abort". hint: Disable this message with "git config set advice.mergeConflict false" ``` --------- Co-authored-by: Felix Arntz <felix.arntz@vercel.com>
4 tasks
felixarntz
added a commit
that referenced
this pull request
Mar 6, 2026
…arch with multimodal output (#13171) ## Summary This helps test and verify that thought signatures now work correctly end-to-end, for both images and web search and image search sources, including in multi-turn content. ## Manual Verification Did some extensive testing with these two new examples - everything works as expected. ## Checklist - [ ] Tests have been added / updated (for bug fixes / features) - [ ] Documentation has been added / updated (for bug fixes / features) - [ ] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] I have reviewed this pull request (self-review) ## Related Issues Follow up to #12926
vercel-ai-sdk bot
pushed a commit
that referenced
this pull request
Mar 6, 2026
…arch with multimodal output (#13171) ## Summary This helps test and verify that thought signatures now work correctly end-to-end, for both images and web search and image search sources, including in multi-turn content. ## Manual Verification Did some extensive testing with these two new examples - everything works as expected. ## Checklist - [ ] Tests have been added / updated (for bug fixes / features) - [ ] Documentation has been added / updated (for bug fixes / features) - [ ] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] I have reviewed this pull request (self-review) ## Related Issues Follow up to #12926
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Background
Google introduced image search to their API, which can be used for grounding when generating images (e.g. with
gemini-3.1-flash-image-preview): https://ai.google.dev/gemini-api/docs/image-generation#image-searchIn the current AI SDK implementation, the
googleSearchtool hadmodeanddynamicThresholdparameters, which were actually from thegoogle_search_retrievaltool that Google defines, which only worked with Gemini 1.5 models. Since those models are retired and can no longer be accessed, these parameters were effectively dead code — silently ignored for all Gemini 2.0+ models. Meanwhile, the current Google Search API supports new capabilities (searchTypesfor image search,timeRangeFilter) that the SDK didn't support.Summary
Replaces the obsolete
google_search_retrievalcode paths with support for the currentgoogleSearchAPI parameters:searchTypes: Enables web and/or image search grounding (e.g.{ imageSearch: {} })timeRangeFilter: Restricts search results to a time range viastartTime/endTimeThe
googleSearchRetrievallogic andsupportsDynamicRetrievalvariable are removed entirely. Since they were impossible to use in practice, this is effectively not a breaking change (because such code would already be broken).The grounding metadata schema and source extraction are extended to handle
imagegrounding chunks (withsourceUri,imageUri,title,domain), enabling proper source attribution for image search results.Manual Verification
Ran the new examples against the Google API:
examples/ai-functions/src/generate-text/google/search-types.ts— verifiedsearchTypesandtimeRangeFilterpass through correctlyexamples/ai-functions/src/generate-text/google/image-search.ts— verified image search grounding returns image results with proper source attributionexamples/ai-functions/src/stream-text/google/search-types.ts— verified streaming with search typesexamples/ai-functions/src/stream-text/google/image-search.ts— verified streaming image searchChecklist
pnpm changesetin the project root)Future Work
Consider adding a dedicated source type in v7 to include both
imageUriandsourceUriseparately for image grounding chunks. In the PR implementation, we cannot surface both because our spec does not support it.