Skip to content

Commit 2565e70

Browse files
felixarntzfelixarntz
authored
feat(google): add support for image search, replace obsolete google_search_retrieval implementation (#12926)
## Background Google introduced image search to their API, which can be used for grounding when generating images (e.g. with `gemini-3.1-flash-image-preview`): https://ai.google.dev/gemini-api/docs/image-generation#image-search In the current AI SDK implementation, the `googleSearch` tool had `mode` and `dynamicThreshold` parameters, which were actually from the `google_search_retrieval` tool that Google defines, which only worked with Gemini 1.5 models. Since those models are retired and can no longer be accessed, these parameters were effectively dead code — silently ignored for all Gemini 2.0+ models. Meanwhile, the current Google Search API supports new capabilities (`searchTypes` for image search, `timeRangeFilter`) that the SDK didn't support. ## Summary Replaces the obsolete `google_search_retrieval` code paths with support for the current `googleSearch` API parameters: - **`searchTypes`**: Enables web and/or image search grounding (e.g. `{ imageSearch: {} }`) - **`timeRangeFilter`**: Restricts search results to a time range via `startTime`/`endTime` The `googleSearchRetrieval` logic and `supportsDynamicRetrieval` variable are removed entirely. Since they were impossible to use in practice, this is effectively not a breaking change (because such code would already be broken). The grounding metadata schema and source extraction are extended to handle `image` grounding chunks (with `sourceUri`, `imageUri`, `title`, `domain`), enabling proper source attribution for image search results. ## Manual Verification Ran the new examples against the Google API: - `examples/ai-functions/src/generate-text/google/search-types.ts` — verified `searchTypes` and `timeRangeFilter` pass through correctly - `examples/ai-functions/src/generate-text/google/image-search.ts` — verified image search grounding returns image results with proper source attribution - `examples/ai-functions/src/stream-text/google/search-types.ts` — verified streaming with search types - `examples/ai-functions/src/stream-text/google/image-search.ts` — verified streaming image search ## Checklist - [x] Tests have been added / updated (for bug fixes / features) - [x] Documentation has been added / updated (for bug fixes / features) - [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root) - [x] I have reviewed this pull request (self-review) ## Future Work Consider adding a dedicated source type in v7 to include both `imageUri` and `sourceUri` separately for image grounding chunks. In the PR implementation, we cannot surface both because our spec does not support it.
1 parent 3884335 commit 2565e70

File tree

11 files changed

+502
-138
lines changed

11 files changed

+502
-138
lines changed

‎.changeset/happy-pans-fix.md‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
'@ai-sdk/google': patch
3+
---
4+
5+
feat(google): add support for image search, replace obsolete google_search_retrieval implementation

‎content/providers/01-ai-sdk-providers/15-google-generative-ai.mdx‎

Lines changed: 22 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -505,9 +505,8 @@ The response will contain the tool calls and results from the code execution.
505505

506506
### Google Search
507507

508-
With [search grounding](https://ai.google.dev/gemini-api/docs/google-search),
509-
the model has access to the latest information using Google search.
510-
Google search can be used to provide answers around current events:
508+
With [Google Search grounding](https://ai.google.dev/gemini-api/docs/google-search),
509+
the model has access to the latest information using Google Search.
511510

512511
```ts highlight="8,17-20"
513512
import { google } from '@ai-sdk/google';
@@ -535,18 +534,31 @@ const safetyRatings = metadata?.safetyRatings;
535534

536535
The `googleSearch` tool accepts the following optional configuration options:
537536

538-
- **mode** _'MODE_DYNAMIC' | 'MODE_UNSPECIFIED'_
537+
- **searchTypes** _object_
539538

540-
The mode of the predictor to be used in dynamic retrieval. Default is `'MODE_UNSPECIFIED'`.
539+
Enables specific search types. Both can be combined.
541540

542-
- `MODE_DYNAMIC`: Run retrieval only when the system decides it is necessary
543-
- `MODE_UNSPECIFIED`: Always trigger retrieval
541+
- `webSearch`: Enable web search grounding (pass `{}` to enable). This is the default.
542+
- `imageSearch`: Enable [image search grounding](https://ai.google.dev/gemini-api/docs/image-generation#image-search) (pass `{}` to enable).
544543

545-
- **dynamicThreshold** _number_
544+
- **timeRangeFilter** _object_
546545

547-
The threshold to be used in dynamic retrieval. Default is `1`. If not set, a system default value is used.
546+
Restricts search results to a specific time range. Both `startTime` and `endTime` are required.
548547

549-
When Search Grounding is enabled, the model will include sources in the response.
548+
- `startTime`: Start time in ISO 8601 format (e.g. `'2025-01-01T00:00:00Z'`).
549+
- `endTime`: End time in ISO 8601 format (e.g. `'2025-12-31T23:59:59Z'`).
550+
551+
```ts
552+
google.tools.googleSearch({
553+
searchTypes: { webSearch: {} },
554+
timeRangeFilter: {
555+
startTime: '2025-01-01T00:00:00Z',
556+
endTime: '2025-12-31T23:59:59Z',
557+
},
558+
});
559+
```
560+
561+
When Google Search grounding is enabled, the model will include sources in the response.
550562

551563
Additionally, the grounding metadata includes detailed information about how search results were used to ground the model's response. Here are the available fields:
552564

‎examples/ai-functions/src/generate-text/google/image-search.ts‎

Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
import {
2+
google,
3+
GoogleLanguageModelOptions,
4+
GoogleGenerativeAIProviderMetadata,
5+
} from '@ai-sdk/google';
6+
import { generateText } from 'ai';
7+
import { presentImages } from '../../lib/present-image';
8+
import { run } from '../../lib/run';
9+
10+
run(async () => {
11+
const result = await generateText({
12+
model: google('gemini-3.1-flash-image-preview'),
13+
tools: {
14+
google_search: google.tools.googleSearch({
15+
searchTypes: { imageSearch: {} },
16+
}),
17+
},
18+
providerOptions: {
19+
google: {
20+
responseModalities: ['TEXT', 'IMAGE'],
21+
} satisfies GoogleLanguageModelOptions,
22+
},
23+
prompt:
24+
'Search for live footage photos of the 2026 Super Bowl halftime show artist. I want an image with a close-up of them during the show, but in space.',
25+
});
26+
27+
for (const file of result.files) {
28+
if (file.mediaType.startsWith('image/')) {
29+
await presentImages([file]);
30+
}
31+
}
32+
33+
console.log('SOURCES');
34+
console.log(result.sources);
35+
36+
const metadata = (await result.providerMetadata)?.google as
37+
| GoogleGenerativeAIProviderMetadata
38+
| undefined;
39+
const groundingMetadata = metadata?.groundingMetadata;
40+
41+
console.log();
42+
console.log('GROUNDING METADATA');
43+
console.log(groundingMetadata);
44+
});

‎examples/ai-functions/src/generate-text/google/search-types.ts‎

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
import { google, GoogleGenerativeAIProviderMetadata } from '@ai-sdk/google';
2+
import { generateText } from 'ai';
3+
import { run } from '../../lib/run';
4+
5+
run(async () => {
6+
const { text, sources, providerMetadata } = await generateText({
7+
model: google('gemini-2.5-flash'),
8+
tools: {
9+
google_search: google.tools.googleSearch({
10+
searchTypes: { webSearch: {} },
11+
timeRangeFilter: {
12+
startTime: '2026-02-18T00:00:00Z',
13+
endTime: '2026-02-25T00:00:00Z',
14+
},
15+
}),
16+
},
17+
prompt:
18+
'List the top 5 San Francisco news from the past week.' +
19+
'You must include the date of each article.',
20+
});
21+
22+
const metadata = providerMetadata?.google as
23+
| GoogleGenerativeAIProviderMetadata
24+
| undefined;
25+
const groundingMetadata = metadata?.groundingMetadata;
26+
27+
console.log(text);
28+
console.log();
29+
console.log('SOURCES');
30+
console.log(sources);
31+
console.log();
32+
console.log('PROVIDER METADATA');
33+
console.log(groundingMetadata);
34+
});

‎examples/ai-functions/src/stream-text/google/image-search.ts‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
import {
2+
google,
3+
GoogleLanguageModelOptions,
4+
GoogleGenerativeAIProviderMetadata,
5+
} from '@ai-sdk/google';
6+
import { streamText } from 'ai';
7+
import { presentImages } from '../../lib/present-image';
8+
import { run } from '../../lib/run';
9+
10+
run(async () => {
11+
const result = streamText({
12+
model: google('gemini-3.1-flash-image-preview'),
13+
tools: {
14+
google_search: google.tools.googleSearch({
15+
searchTypes: { imageSearch: {} },
16+
}),
17+
},
18+
providerOptions: {
19+
google: {
20+
responseModalities: ['TEXT', 'IMAGE'],
21+
} satisfies GoogleLanguageModelOptions,
22+
},
23+
prompt:
24+
'Search for live footage photos of the 2026 Super Bowl halftime show artist. I want an image with a close-up of them during the show, but in space.',
25+
});
26+
27+
for await (const part of result.fullStream) {
28+
switch (part.type) {
29+
case 'text-delta': {
30+
process.stdout.write(part.text);
31+
break;
32+
}
33+
34+
case 'file': {
35+
if (part.file.mediaType.startsWith('image/')) {
36+
await presentImages([part.file]);
37+
}
38+
break;
39+
}
40+
41+
case 'source': {
42+
if (part.sourceType === 'url') {
43+
console.log('\x1b[36m%s\x1b[0m', 'Source');
44+
console.log('ID:', part.id);
45+
console.log('Title:', part.title);
46+
console.log('URL:', part.url);
47+
console.log();
48+
}
49+
break;
50+
}
51+
}
52+
}
53+
54+
const metadata = (await result.providerMetadata)?.google as
55+
| GoogleGenerativeAIProviderMetadata
56+
| undefined;
57+
const groundingMetadata = metadata?.groundingMetadata;
58+
59+
console.log();
60+
console.log('GROUNDING METADATA');
61+
console.log(groundingMetadata);
62+
});

‎examples/ai-functions/src/stream-text/google/search-types.ts‎

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
import { google, GoogleGenerativeAIProviderMetadata } from '@ai-sdk/google';
2+
import { streamText } from 'ai';
3+
import { run } from '../../lib/run';
4+
5+
run(async () => {
6+
const result = streamText({
7+
model: google('gemini-2.5-flash'),
8+
tools: {
9+
google_search: google.tools.googleSearch({
10+
searchTypes: { webSearch: {} },
11+
timeRangeFilter: {
12+
startTime: '2026-02-18T00:00:00Z',
13+
endTime: '2026-02-25T00:00:00Z',
14+
},
15+
}),
16+
},
17+
prompt:
18+
'List the top 5 San Francisco news from the past week.' +
19+
'You must include the date of each article.',
20+
});
21+
22+
for await (const part of result.fullStream) {
23+
if (part.type === 'text-delta') {
24+
process.stdout.write(part.text);
25+
}
26+
27+
if (part.type === 'source' && part.sourceType === 'url') {
28+
console.log('\x1b[36m%s\x1b[0m', 'Source');
29+
console.log('ID:', part.id);
30+
console.log('Title:', part.title);
31+
console.log('URL:', part.url);
32+
console.log();
33+
}
34+
}
35+
36+
const metadata = (await result.providerMetadata)?.google as
37+
| GoogleGenerativeAIProviderMetadata
38+
| undefined;
39+
const groundingMetadata = metadata?.groundingMetadata;
40+
41+
console.log();
42+
console.log('GROUNDING METADATA');
43+
console.log(groundingMetadata);
44+
console.log();
45+
console.log('Token usage:', await result.usage);
46+
console.log('Finish reason:', await result.finishReason);
47+
});

0 commit comments

Comments
 (0)