Backport: feat(@ai-sdk/google): preserve per-modality token details in usage data#14110
Merged
vercel-ai-sdk[bot] merged 1 commit intorelease-v6.0from Apr 3, 2026
Merged
Conversation
…ta (#14016) ## Summary - Add `promptTokensDetails` and `candidatesTokensDetails` to the Gemini response `usageSchema` so per-modality token counts (TEXT, IMAGE, AUDIO, VIDEO) are no longer stripped by Zod parsing - These fields now flow through to `usage.raw`, enabling downstream consumers to distinguish token usage by modality ## Why Gemini charges different rates for different input modalities (e.g. audio input is $0.50/1M tokens vs $0.25/1M for text/image/video). The ai-gateway needs per-modality token counts to bill correctly. Previously, `promptTokensDetails` was present in the Gemini API response but stripped during Zod schema validation, making it impossible to differentiate modalities downstream. ## Validation - Ran the `generate-text/google/image.ts` example and confirmed `promptTokensDetails` now appears in `usage.raw` with both `TEXT` and `IMAGE` modality entries - All 328 existing tests pass; 10 snapshots updated to include the new fields --------- Co-authored-by: Felix Arntz <felix.arntz@vercel.com>
felixarntz
approved these changes
Apr 3, 2026
Contributor
Author
|
|
Contributor
Author
felixarntz
added a commit
that referenced
this pull request
Apr 3, 2026
… details in usage data (#14113) This is an automated backport of #14110 to the release-v5.0 branch. FYI @vercel-ai-sdk[bot] ~~This backport has conflicts that need to be resolved manually.~~ Conflicts resolved. ### `git cherry-pick` output ``` CONFLICT (file location): examples/ai-functions/src/generate-text/google/modality-token-details.ts added in f20ba77 (Backport: feat(@ai-sdk/google): preserve per-modality token details in usage data (#14110)) inside a directory that was renamed in HEAD, suggesting it should perhaps be moved to examples/ai-core/src/generate-text/modality-token-details.ts. CONFLICT (file location): examples/ai-functions/src/stream-text/google/modality-token-details.ts added in f20ba77 (Backport: feat(@ai-sdk/google): preserve per-modality token details in usage data (#14110)) inside a directory that was renamed in HEAD, suggesting it should perhaps be moved to examples/ai-core/src/stream-text/modality-token-details.ts. CONFLICT (modify/delete): packages/google/src/__snapshots__/google-generative-ai-language-model.test.ts.snap deleted in HEAD and modified in f20ba77 (Backport: feat(@ai-sdk/google): preserve per-modality token details in usage data (#14110)). Version f20ba77 (Backport: feat(@ai-sdk/google): preserve per-modality token details in usage data (#14110)) of packages/google/src/__snapshots__/google-generative-ai-language-model.test.ts.snap left in tree. CONFLICT (modify/delete): packages/google/src/convert-google-generative-ai-usage.ts deleted in HEAD and modified in f20ba77 (Backport: feat(@ai-sdk/google): preserve per-modality token details in usage data (#14110)). Version f20ba77 (Backport: feat(@ai-sdk/google): preserve per-modality token details in usage data (#14110)) of packages/google/src/convert-google-generative-ai-usage.ts left in tree. Auto-merging packages/google/src/google-generative-ai-language-model.ts error: could not apply f20ba77... Backport: feat(@ai-sdk/google): preserve per-modality token details in usage data (#14110) hint: After resolving the conflicts, mark them with hint: "git add/rm <pathspec>", then run hint: "git cherry-pick --continue". hint: You can instead skip this commit with "git cherry-pick --skip". hint: To abort and get back to the state before "git cherry-pick", hint: run "git cherry-pick --abort". hint: Disable this message with "git config set advice.mergeConflict false" ``` --------- Co-authored-by: vercel-ai-sdk[bot] <225926702+vercel-ai-sdk[bot]@users.noreply.github.com> Co-authored-by: Felix Arntz <felix.arntz@vercel.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is an automated backport of #14016 to the release-v6.0 branch. FYI @R-Taneja