fix(vertex): add fallback for providerOptions key#12403
Conversation
|
The fix is incomplete as we expect |
|
From a user perspective: Step 1: Tool Call Generation (Output)(i) What I put in: A (ii) What I get back: The reasoning state is returned under (iii) Expected behavior: Per Google Cloud Documentation, the Step 2: Tool Result Resumption (Input)(i) What I put in: I pass the history back to the Vertex provider. Following the standard convention, the signature is provided under the (ii) What I get back: A 500 Internal Server Error from Vertex AI. (iii) Expected behavior: The model should resume reasoning. |
|
i verified the fix for the reproduction code snippet you provided and the pr fixed the error! so assuming the bug only produces in a setup like your repro, the fix should be enough. we specifically allow keeping vertex as provider option key name since there were previous requests of doing so for keeping a track when using vertex provider. under the hood, you can rest assure that the needed providerOptions are passed how the endpoint expects them |
|
If user set I would expect I noticed that |
| (providerOptionsName !== 'google' | ||
| ? part.providerOptions?.google | ||
| : undefined); |
There was a problem hiding this comment.
could this be simplified to just part.providerOptions?.google? (it would double lookup and return undefined in the negative case but i dont think that is an issue)
There was a problem hiding this comment.
hmm i tried that and it leads to a regression with some other tests. it would basically ignore any options set under a key which is not google (which wouldn't work in the case of vertex provider, where we added this change recently)
|
@buibaoanh the mapping isn't 1-1 if you compare it to google's sdk but that is by design of the AI SDK there is a reason for why it's mapped to vertex instead of google and you can see here and here why the ideal dx for setting providerOptions when using vertex provider is that the user defines the key to be |
|
@aayush-kapoor Ok, what about google-vertex docs refer to |
|
@buibaoanh thanks for pointing out, will update the docs |
## Background as a follow up to this comment: #12403 (comment) ## Checklist - [ ] na ~~Tests have been added / updated (for bug fixes / features)~~ - [x] Documentation has been added / updated (for bug fixes / features) - [ ] na ~~A _patch_ changeset for relevant packages has been added (for bug fixes / features - run `pnpm changeset` in the project root)~~ - [x] I have reviewed this pull request (self-review)
Background
multi step tool calls fail because the
thoughtSignaturegets lost due to a key mismatch in providerOptions.if the
thoughtSignatureis passed under the google key, the vertex provider silently drops it. the api then rejects the request and we see the errors as observed in #12351Summary
fallback check to see if
vertexkey doesn't have athoughtSignature, we check if it's passed within the google keyManual Verification
verified via the repro code given in the original issue, which is resolved after the fix
Checklist
pnpm changesetin the project root)Related Issues
fixes #12351