feat(provider/google): support combining built-in tools with function calling on Gemini 3 by edwardwc · Pull Request #13920 · vercel/ai

edwardwc · 2026-03-30T17:24:43Z

Summary

Gemini 3 models now support combining built-in tools (Google Search, Google Maps, URL Context, etc.) with custom function calling in a single API request via the tool combination API (announced March 17, 2026).

Previously, prepareTools treated provider-defined tools and function tools as mutually exclusive — when both were present, it emitted an "unsupported" warning and dropped all function declarations, sending only the provider tools. This caused models to emit function calls as inline JSON text since no function declarations were registered.

This PR enables tool combination for Gemini 3+ models by:

Merging provider tools and functionDeclarations into the same tools array when both are present on Gemini 3+ models
Setting includeServerSideToolInvocations: true in toolConfig to enable server-side tool context circulation
Using VALIDATED function calling mode (required by the tool combination API)
Parsing toolCall/toolResponse parts from the API response as provider-executed tool calls/results
Serializing them back correctly in subsequent turns to maintain cross-tool context circulation

For older models (Gemini 2.x and below), the existing behavior is preserved — a warning is emitted and only provider tools are sent.

Test plan

Existing prepareTools tests updated for new merge behavior on Gemini 3+
New tests for combined tool output: [{ googleSearch: {} }, { functionDeclarations: [...] }]
Verify older models still get the warning and provider-only behavior
Manual test: Google Search + custom function calling on gemini-3-flash-preview
Verify toolCall/toolResponse parts are preserved across multi-turn conversations

Made with Cursor

… calling on Gemini 3 Gemini 3 models now support combining built-in tools (Google Search, Google Maps, URL Context, etc.) with custom function calling in a single API request via the tool combination API. Previously, prepareTools treated provider-defined tools and function tools as mutually exclusive — when both were present, it emitted an "unsupported" warning and dropped all function declarations, sending only the provider tools. This caused models to emit function calls as inline JSON text since no function declarations were registered. Changes: - Merge provider tools and functionDeclarations into the same tools array when both are present on Gemini 3+ models - Set includeServerSideToolInvocations: true in toolConfig to enable server-side tool context circulation - Use VALIDATED function calling mode (required by the tool combination API) - Parse toolCall/toolResponse parts from the API response as provider-executed tool calls/results - Serialize them back correctly in subsequent turns to maintain cross-tool context circulation - For older models (Gemini 2.x and below), preserve existing behavior Ref: https://ai.google.dev/gemini-api/docs/tool-combination Made-with: Cursor

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e42c6dcb4f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-30T17:30:18Z

packages/google/src/google-generative-ai-language-model.ts

+      } else if ('toolCall' in part && part.toolCall) {
+        const toolCallId = part.toolCall.id ?? this.config.generateId();


Parse toolCall/toolResponse in response schema

The new handling for part.toolCall/part.toolResponse in doGenerate and doStream is effectively unreachable because responseSchema/chunkSchema still parse content.parts with getContentSchema(), which only models functionCall, inlineData, and executable/text fields. With Zod object parsing, unknown keys are stripped, so Gemini tool-combination parts are reduced before this branch and server-side tool invocations/results are silently dropped, breaking multi-turn context circulation for combined tools.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-03-30T17:30:18Z

packages/google/src/convert-to-google-generative-ai-messages.ts

+          const partProviderOpts =
+            part.providerMetadata?.[providerOptionsName] ??
+            (providerOptionsName !== 'google'


Read provider options from tool parts, not providerMetadata

In the role: 'tool' conversion path, this reads part.providerMetadata, but prompt tool parts carry provider data under providerOptions (including when produced by toResponseMessages/convertToLanguageModelPrompt). That means serverToolCallId/serverToolType are never recovered, so provider-executed server tool results are serialized as generic function responses instead of toolResponse, which loses the tool-combination state on the next Gemini turn.

Useful? React with 👍 / 👎.

VALIDATED mode rejects anyOf schemas in function parameters, which breaks tools using discriminated unions. The Gemini API docs say "default to VALIDATED" but omitting functionCallingConfig works and lets the API handle mode selection internally. Made-with: Cursor

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e554ab5934

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-03-30T17:58:10Z

packages/google/src/google-prepare-tools.ts

      }
    });

+    if (hasFunctionTools && isGemini3orNewer && googleTools.length > 0) {


Keep function declarations when provider tools are filtered out

This branch only combines function declarations when googleTools.length > 0, so on Gemini 3 a request that includes valid function tools plus only unsupported/typoed provider tools falls through and returns without any functionDeclarations. In that case the warning reports unsupported provider tools, but function calling is silently disabled even though the function tools are valid.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-03-30T17:58:10Z

packages/google/src/convert-to-google-generative-ai-messages.ts

+                  if (serverToolCallId && serverToolType) {
+                    return {
+                      toolCall: {
+                        toolType: serverToolType,


Avoid hard-failing on non-JSON server tool-call input

The server-tool replay path unconditionally calls JSON.parse on part.input when it is a string. If persisted or user-supplied prompt history contains a non-JSON string, prompt conversion throws before the API call and the entire request fails. This path should parse defensively (or fall back) instead of crashing on malformed input.

Useful? React with 👍 / 👎.

chatgpt-codex-connector · 2026-03-30T17:58:10Z

packages/google/src/google-generative-ai-language-model.ts

+                  serverToolCallId: part.toolCall.id,
+                  serverToolType: part.toolCall.toolType,


Persist fallback server tool IDs into provider metadata

When a server toolCall arrives without an id, this code generates a fallback toolCallId for emitted content but still stores serverToolCallId: part.toolCall.id (undefined) in provider metadata. On the next turn, server-tool replay requires both serverToolType and serverToolCallId, so these calls/responses are downgraded and lose server-side tool-context linkage.

Useful? React with 👍 / 👎.

vercel

Additional Suggestion:

The Zod response schema's getContentSchema() is missing toolCall and toolResponse union variants, causing Zod to strip these properties during parsing and making the entire tool combination feature non-functional.

…ages

…ol call annotations

felixarntz

@edwardwc Thank you for the PR! This was looking good for starters, however we needed some examples to verify the behavior, so I added them. From there, I ran into a few follow up issues, which should be addressed now.

vercel-ai-sdk · 2026-04-01T00:04:47Z

⚠️ Backport to release-v6.0 created but has conflicts: #13993

vercel-ai-sdk · 2026-04-01T00:16:14Z

🚀 Published in:

Package	Version
`@ai-sdk/google`	`4.0.0-beta.19`
`@ai-sdk/google-vertex`	`5.0.0-beta.26`

@edwardwc

…h function calling on Gemini 3 (#13993) This is an automated backport of #13920 to the release-v6.0 branch. FYI @edwardwc ~~This backport has conflicts that need to be resolved manually.~~ Conflicts resolved. ### `git cherry-pick` output ``` Auto-merging packages/google/src/convert-to-google-generative-ai-messages.test.ts Auto-merging packages/google/src/convert-to-google-generative-ai-messages.ts Auto-merging packages/google/src/google-generative-ai-language-model.test.ts Auto-merging packages/google/src/google-generative-ai-language-model.ts CONFLICT (content): Merge conflict in packages/google/src/google-generative-ai-language-model.ts Auto-merging packages/google/src/google-prepare-tools.test.ts Auto-merging packages/google/src/google-prepare-tools.ts error: could not apply 01fa606... feat(provider/google): support combining built-in tools with function calling on Gemini 3 (#13920) hint: After resolving the conflicts, mark them with hint: "git add/rm <pathspec>", then run hint: "git cherry-pick --continue". hint: You can instead skip this commit with "git cherry-pick --skip". hint: To abort and get back to the state before "git cherry-pick", hint: run "git cherry-pick --abort". hint: Disable this message with "git config set advice.mergeConflict false" ``` --------- Co-authored-by: edwardwc <edwardwc@protonmail.com> Co-authored-by: Felix Arntz <felix.arntz@vercel.com>

tigent bot added ai/provider related to a provider package. Must be assigned together with at least one `provider/*` label feature New feature or request provider/google Issues related to the @ai-sdk/google provider labels Mar 30, 2026

chatgpt-codex-connector bot reviewed Mar 30, 2026

View reviewed changes

vercel bot reviewed Mar 30, 2026

View reviewed changes

felixarntz added 11 commits March 31, 2026 08:21

Merge branch 'main' into feat/google-tool-combination

57c56f3

delete unrelated design doc

2265461

add missing definition for new types in getContentSchema()

f00e5ca

fix unnecessary JSON roundtrips, and incorrect providerOptions lookup

bb35040

respect toolChoice parameter with combined tool calling

4f1c72f

do not add warning for toolChoice auto because it is the default

5a2c51e

add missing tool-result processing in convertToGoogleGenerativeAIMess…

00b7f6f

…ages

Merge branch 'main' into feat/google-tool-combination

f224ebb

fix subsequent steps not being executed because of missing dynamic to…

6a05dfb

…ol call annotations

add examples for combined tool calling

e3e40ec

changeset

3b60f75

felixarntz approved these changes Mar 31, 2026

View reviewed changes

felixarntz added the backport Admins only: add this label to a pull request in order to backport it to the prior version label Mar 31, 2026

felixarntz added 2 commits March 31, 2026 15:36

Merge branch 'main' into feat/google-tool-combination

1f20e47

correctly propagate fallback tool call ID

d2400ea

felixarntz merged commit 01fa606 into vercel:main Apr 1, 2026
17 of 18 checks passed

vercel-ai-sdk bot mentioned this pull request Apr 1, 2026

Backport: feat(provider/google): support combining built-in tools with function calling on Gemini 3 #13993

Merged

vercel-ai-sdk bot removed the backport Admins only: add this label to a pull request in order to backport it to the prior version label Apr 1, 2026

vercel-ai-sdk bot pushed a commit that referenced this pull request Apr 1, 2026

Backport conflicts for PR #13920 to release-v6.0

c56f375

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(provider/google): support combining built-in tools with function calling on Gemini 3#13920

feat(provider/google): support combining built-in tools with function calling on Gemini 3#13920
felixarntz merged 15 commits intovercel:mainfrom
edwardwc:feat/google-tool-combination

edwardwc commented Mar 30, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Mar 30, 2026

Uh oh!

chatgpt-codex-connector bot Mar 30, 2026

Uh oh!

chatgpt-codex-connector bot left a comment

Uh oh!

chatgpt-codex-connector bot Mar 30, 2026

Uh oh!

chatgpt-codex-connector bot Mar 30, 2026

Uh oh!

chatgpt-codex-connector bot Mar 30, 2026

Uh oh!

vercel bot left a comment

Uh oh!

felixarntz left a comment

Uh oh!

Uh oh!

vercel-ai-sdk bot commented Apr 1, 2026

Uh oh!

vercel-ai-sdk bot commented Apr 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		} else if ('toolCall' in part && part.toolCall) {
		const toolCallId = part.toolCall.id ?? this.config.generateId();

		serverToolCallId: part.toolCall.id,
		serverToolType: part.toolCall.toolType,

Conversation

edwardwc commented Mar 30, 2026

Summary

Test plan

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot Mar 30, 2026

Choose a reason for hiding this comment

Uh oh!

vercel bot left a comment

Choose a reason for hiding this comment

Uh oh!

felixarntz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

vercel-ai-sdk bot commented Apr 1, 2026

Uh oh!

vercel-ai-sdk bot commented Apr 1, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants