feat(provider): add reasoning-file support#13352
Conversation
| providerMetadata?: ProviderMetadata; | ||
| } | ||
|
|
||
| export function convertFromReasoningOutputs( |
There was a problem hiding this comment.
I don't love that we have to have these helpers to convert reasoning outputs to reasoning parts and vice-versa.
Maybe it's needed indeed, but curious why this differentiation exists? @lgrammel
Looks like ContentPart internally uses ReasoningOutput, GenerateTextResult and StreamTextResult use ReasoningOutput too, but then StepResult uses ReasoningPart 🤔
There was a problem hiding this comment.
It is an input / output distinction because inputs and outputs can have different shapes (for DX reasons)
in general, for inputs we want to accept a wide range of formats (so users can just pass in what they need without needing to convert themselves)
for outputs, we want to standardize and offer helpers for conversions, so users can use them however they want (without having to convert or worrying about memory)
files are currently the only case where this distinction is prominent
There was a problem hiding this comment.
re the conversion helpers: we could eliminate it on the input side by allowing generatedfile as input (future work)
There was a problem hiding this comment.
That makes sense. But shouldn't StepResult also use ReasoningOutput rather than ReasoningPart then? It contains reasoning output after all, not input. Wondering whether this is technically a bug?
| '@ai-sdk/provider-utils': patch | ||
| '@ai-sdk/provider': patch | ||
| '@ai-sdk/google': patch | ||
| '@ai-sdk/xai': patch |
There was a problem hiding this comment.
wonder if provider / ai needs to be major (breaking change, see below)
There was a problem hiding this comment.
@gr2m I think you mentioned we still shouldn't use major bumps here because the overall v7 is considered one? Or did I misunderstand that?
| { | ||
| name: 'reasoning', | ||
| type: 'Array<ReasoningPart>', | ||
| type: 'Array<ReasoningPart | ReasoningFilePart>', |
There was a problem hiding this comment.
These are breaking changes because they widen the type that needs to be handled by the users.
| * The full reasoning that the model has generated in the last step. | ||
| */ | ||
| readonly reasoning: Array<ReasoningOutput>; | ||
| readonly reasoning: Array<ReasoningOutput | ReasoningFileOutput>; |
There was a problem hiding this comment.
breaking change; would be good to list in a migration guide so we do not forget
| providerMetadata?: ProviderMetadata; | ||
| } | ||
|
|
||
| export function convertFromReasoningOutputs( |
There was a problem hiding this comment.
It is an input / output distinction because inputs and outputs can have different shapes (for DX reasons)
in general, for inputs we want to accept a wide range of formats (so users can just pass in what they need without needing to convert themselves)
for outputs, we want to standardize and offer helpers for conversions, so users can use them however they want (without having to convert or worrying about memory)
files are currently the only case where this distinction is prominent
| providerMetadata?: ProviderMetadata; | ||
| } | ||
|
|
||
| export function convertFromReasoningOutputs( |
There was a problem hiding this comment.
re the conversion helpers: we could eliminate it on the input side by allowing generatedfile as input (future work)
| @@ -0,0 +1,32 @@ | |||
| import { SharedV3ProviderMetadata } from '../../shared'; | |||
There was a problem hiding this comment.
separate work: v4 models should use v4 provider metadata
## Background The v4 language model spec types were incorrectly importing and using shared v3 types (`SharedV3ProviderOptions`, `SharedV3ProviderMetadata`, `SharedV3Headers`, `SharedV3Warning`) instead of their v4 equivalents. This meant the v4 spec was not self-contained and depended on v3 shared types. Initially pointed out by @lgrammel in #13352 (comment). ## Summary Updated all v4 language model spec files to import from `shared/v4` instead of `shared/v3`, using the correct `SharedV4*` types throughout.
|
🚀 Published in:
|
Background
See #13321
Summary
reasoning-filefor files that are part of reasoning rather than actual contentproviderv4 specaiare expanded accordinglyreasoning-file(and thus breaking it apart fromfile), we fix the bug where reasoning files would before be included in top-levelfiles(e.g. inGenerateTextResultorStepResult), which would lead to a duplicate entry in many scenariosgooglelanguage model to usereasoning-fileinstead offilewith provider metadata where applicablegoogleexamples which were previously based on the provider options workaroundFYI: 9c1c48d shows a glimpse of what supporting this in future 3P code could look like.
Manual Verification
Run the examples modified here.
Specifically, run
examples/ai-functions/src/generate-text/google/image-thinking-multi-step.tsagainst this branch and againstmain: Withmain, you'll see thatfileswill include the same image twice (once as part of reasoning, once as the actual output) - this is the bug this PR effectively fixes.Checklist
pnpm changesetin the project root)Future Work
N/A
Related Issues
Fixes #13321