Skip to content

feat (provider/gateway): add get-generation support#13842

Merged
shaper merged 3 commits intomainfrom
03-26-feat_provider_gateway_add_get-generation_support
Mar 27, 2026
Merged

feat (provider/gateway): add get-generation support#13842
shaper merged 3 commits intomainfrom
03-26-feat_provider_gateway_add_get-generation_support

Conversation

@shaper
Copy link
Copy Markdown
Collaborator

@shaper shaper commented Mar 26, 2026 •
edited
Loading

Background

The AI Gateway provider was missing the ability to look up detailed information about specific generations after they were created. Users needed a way to retrieve cost, token usage, latency, and provider details for individual generations using their IDs.

Changes

This PR adds generation lookup functionality to the AI Gateway provider:

  • New getGenerationInfo() method: Allows fetching detailed information about a specific generation by its ID
  • Generation ID access: Generation IDs are now available in providerMetadata.gateway.generationId on both generateText and streamText responses
  • Comprehensive generation data: Returns cost breakdown, token usage, latency metrics, provider details, and more
  • Error handling: Proper error handling for authentication, server errors, and malformed responses
  • Documentation: Added complete documentation with examples showing how to use the new functionality
  • Example implementation: Added a working example demonstrating the feature

The getGenerationInfo() method returns detailed information including:

  • Cost breakdown (total cost, upstream inference cost for BYOK)
  • Token usage (prompt, completion, reasoning, cached tokens)
  • Performance metrics (latency, generation time)
  • Provider and model information
  • Generation metadata (streaming, finish reason, etc.)

Manual Verification

Tested the feature by:

  1. Making a generation request with the gateway provider
  2. Extracting the generation ID from the provider metadata
  3. Using getGenerationInfo() to fetch detailed generation information
  4. Verifying all returned fields contain expected data types and values
  5. Testing error scenarios (invalid IDs, authentication failures)

Checklist

  • Tests have been added / updated (for bug fixes / features)
  • Documentation has been added / updated (for bug fixes / features)
  • A patch changeset for relevant packages has been added (for bug fixes / features - run pnpm changeset in the project root)
  • I have reviewed this pull request (self-review)

@shaper Graphite App
Copy link
Copy Markdown
Collaborator Author

shaper commented Mar 26, 2026 •
edited
Loading

This stack of pull requests is managed by Graphite. Learn more about stacking.

@shaper shaper mentioned this pull request Mar 26, 2026
Merged
4 tasks
@tigent tigent bot added ai/provider related to a provider package. Must be assigned together with at least one `provider/*` label feature New feature or request provider/gateway Issues related to the @ai-sdk/gateway provider labels Mar 26, 2026
@shaper shaper requested review from R-Taneja and felixarntz March 26, 2026 07:36
@shaper shaper marked this pull request as ready for review March 26, 2026 18:48
@shaper shaper force-pushed the 03-25-feat_provider_gateway_add_spend_reporting_support branch from 6f6e9bf to fdaacea Compare March 26, 2026 22:01
@shaper shaper force-pushed the 03-26-feat_provider_gateway_add_get-generation_support branch from b14d7ed to 5464080 Compare March 26, 2026 22:01
@shaper shaper force-pushed the 03-26-feat_provider_gateway_add_get-generation_support branch from 5464080 to 7bc628c Compare March 26, 2026 22:05
@shaper shaper force-pushed the 03-26-feat_provider_gateway_add_get-generation_support branch from 7bc628c to 6d9113b Compare March 26, 2026 22:32
@shaper shaper force-pushed the 03-25-feat_provider_gateway_add_spend_reporting_support branch from d574905 to f01df16 Compare March 26, 2026 22:32
@shaper shaper changed the base branch from 03-25-feat_provider_gateway_add_spend_reporting_support to graphite-base/13842 March 26, 2026 22:37
@shaper shaper force-pushed the graphite-base/13842 branch from f01df16 to d30466c Compare March 26, 2026 22:37
@shaper shaper force-pushed the 03-26-feat_provider_gateway_add_get-generation_support branch from 6d9113b to 4a68f65 Compare March 26, 2026 22:37
@graphite-app graphite-app bot changed the base branch from graphite-base/13842 to main March 26, 2026 22:37
@shaper shaper force-pushed the 03-26-feat_provider_gateway_add_get-generation_support branch from 4a68f65 to 2fd9eab Compare March 26, 2026 22:37
@shaper shaper merged commit 435895b into main Mar 27, 2026
19 checks passed
@shaper shaper deleted the 03-26-feat_provider_gateway_add_get-generation_support branch March 27, 2026 07:09
@shaper shaper added the backport Admins only: add this label to a pull request in order to backport it to the prior version label Mar 27, 2026
vercel-ai-sdk bot pushed a commit that referenced this pull request Mar 27, 2026
@shaper @vercel-ai-sdk
feat (provider/gateway): add get-generation support (#13842)
d7e7d76 
## Background

The AI Gateway provider was missing the ability to look up detailed information about specific generations after they were created. Users needed a way to retrieve cost, token usage, latency, and provider details for individual generations using their IDs.

## Changes

This PR adds generation lookup functionality to the AI Gateway provider:

- **New `getGenerationInfo()` method**: Allows fetching detailed information about a specific generation by its ID
- **Generation ID access**: Generation IDs are now available in `providerMetadata.gateway.generationId` on both `generateText` and `streamText` responses
- **Comprehensive generation data**: Returns cost breakdown, token usage, latency metrics, provider details, and more
- **Error handling**: Proper error handling for authentication, server errors, and malformed responses
- **Documentation**: Added complete documentation with examples showing how to use the new functionality
- **Example implementation**: Added a working example demonstrating the feature

The `getGenerationInfo()` method returns detailed information including:
- Cost breakdown (total cost, upstream inference cost for BYOK)
- Token usage (prompt, completion, reasoning, cached tokens)
- Performance metrics (latency, generation time)
- Provider and model information
- Generation metadata (streaming, finish reason, etc.)

## Manual Verification

Tested the feature by:
1. Making a generation request with the gateway provider
2. Extracting the generation ID from the provider metadata
3. Using `getGenerationInfo()` to fetch detailed generation information
4. Verifying all returned fields contain expected data types and values
5. Testing error scenarios (invalid IDs, authentication failures)

## Checklist

- [x] Tests have been added / updated (for bug fixes / features)
- [x] Documentation has been added / updated (for bug fixes / features)
- [x] A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root)
- [x] I have reviewed this pull request (self-review)
@vercel-ai-sdk vercel-ai-sdk bot mentioned this pull request Mar 27, 2026
Merged
@vercel-ai-sdk vercel-ai-sdk bot removed the backport Admins only: add this label to a pull request in order to backport it to the prior version label Mar 27, 2026
@vercel-ai-sdk
Copy link
Copy Markdown
Contributor

vercel-ai-sdk bot commented Mar 27, 2026

✅ Backport PR created: #13870

@vercel-ai-sdk
Copy link
Copy Markdown
Contributor

vercel-ai-sdk bot commented Mar 27, 2026

🚀 Published in:

Package Version
ai 7.0.0-beta.45
@ai-sdk/angular 3.0.0-beta.45
@ai-sdk/gateway 4.0.0-beta.25
@ai-sdk/langchain 3.0.0-beta.45
@ai-sdk/llamaindex 3.0.0-beta.45
@ai-sdk/react 4.0.0-beta.45
@ai-sdk/rsc 3.0.0-beta.46
@ai-sdk/svelte 5.0.0-beta.45
@ai-sdk/vue 4.0.0-beta.45

vercel-ai-sdk bot added a commit that referenced this pull request Mar 27, 2026
@vercel-ai-sdk @shaper
Backport: feat (provider/gateway): add get-generation support (#13870)
768a9d6 
This is an automated backport of #13842 to the release-v6.0 branch. FYI
@shaper

Co-authored-by: Walter Korman <shaper@vercel.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@felixarntz felixarntz felixarntz approved these changes

@R-Taneja R-Taneja Awaiting requested review from R-Taneja

Assignees

@shaper shaper

Labels

ai/provider related to a provider package. Must be assigned together with at least one `provider/*` label feature New feature or request provider/gateway Issues related to the @ai-sdk/gateway provider

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@shaper @felixarntz