Adding Anthropic messages API support to the Google provider through Google vertex#63282
Conversation
1fa215c to
a714961
Compare
b6d2403 to
9e633b9
Compare
|
looks like you've got a bit of cleanup left to do here, and a few merge conflicts to resolve - can you push commits to fix all my comments before our call tomorrow? |
05fbd36 to
a4f6836
Compare
- Validate that the input messages are not empty - Ensure the first message is a non-empty assistant message - Skip empty assistant messages at the end of the prompt - Disallow consistent speaker role between consecutive messages - Add tests for the various validation cases
a4f6836 to
4c84bcd
Compare
| func getgRPCMethod(stream bool) string { | ||
| func getgRPCMethod(stream bool, modelFamily ModelFamily) string { | ||
| if modelFamily == VertexAnthropic { | ||
| return "streamRawPredict" |
There was a problem hiding this comment.
this seems sus, shouldn't this be under the if stream case below? otherwise streamRawPredict would be used, which is presumably for streaming only(?), even when the non-streaming request is being made
|
Left a few more thoughts - let's go over them and fix them together in a call if that helps |
Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com>
|
The backport to To backport this PR manually, you can either: Via the sg toolUse the sg backport -r 5.4.5099 -p 63282Via your terminalTo backport manually, run these commands in your terminal: # Fetch latest updates from GitHub
git fetch
# Create a new working tree
git worktree add .worktrees/backport-5.4.5099 5.4.5099
# Navigate to the new working tree
cd .worktrees/backport-5.4.5099
# Create a new branch
git switch --create backport-63282-to-5.4.5099
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x --mainline 1 1a6a7f78bf5b545f16e604e4879a56b14f3a12a4
# Push it to GitHub
git push --set-upstream origin backport-63282-to-5.4.5099
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-5.4.5099If you encouter conflict, first resolve the conflict and stage all files, then run the commands below: git cherry-pick --continue
# Push it to GitHub
git push --set-upstream origin backport-63282-to-5.4.5099
# Go back to the original working tree
cd ../..
# Delete the working tree
git worktree remove .worktrees/backport-5.4.5099
Once the pull request has been created, please ensure the following:
|
…Google vertex (#63282) [Linear Issue](https://linear.app/sourcegraph/project/claude-3-on-gcp-8c014e1a3506/overview) This PR adds support for anthropic models in the google provider through google vertex. NOTE: The current code only supported Google Gemini API and had boiler plate code for Google vertex(only for the gemini model) this PR adds Google Vertex for anthropic models properly so this way the google provider can be run in 3 different configurations 1. Google Gemini API(this works but only for chat and not for completions which is the intended behaviour for now) 2. Google Vertex API Anthropic Model(This works perfectly and is added in this PR and tested for both chat and completions and it works great) 3. Google Vertex API Gemini Model (this doesn't work yet and can eventually be added to this codebase but we gotta add a new decoder for the streaming responses of the gemini model through this API we can take care of this later) Sense of Urgency: This is a P0 because of enterprise requirements so I would appreciate a fast approval and merging. <!-- 💡 To write a useful PR description, make sure that your description covers: - WHAT this PR is changing: - How was it PREVIOUSLY. - How it will be from NOW on. - WHY this PR is needed. - CONTEXT, i.e. to which initiative, project or RFC it belongs. The structure of the description doesn't matter as much as covering these points, so use your best judgement based on your context. Learn how to write good pull request description: https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e?pvs=4 --> - Run this branch for Cody instance -> https://github.com/sourcegraph/cody/pull/4606 - Ask @arafatkatze to dm you the siteadmin config to make things work - Check the logs and play with completions and chat <!-- All pull requests REQUIRE a test plan: https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles --> <!-- 1. Ensure your pull request title is formatted as: $type($domain): $what 3. Add bullet list items for each additional detail you want to cover (see example below) 4. You can edit this after the pull request was merged, as long as release shipping it hasn't been promoted to the public. 5. For more information, please see this how-to https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c? Audience: TS/CSE > Customers > Teammates (in that order). Cheat sheet: $type = chore|fix|feat $domain: source|search|ci|release|plg|cody|local|... --> <!-- Example: Title: fix(search): parse quotes with the appropriate context Changelog section: - When a quote is used with regexp pattern type, then ... - Refactored underlying code. --> --------- Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com> Co-authored-by: Beatrix <beatrix@sourcegraph.com> Co-authored-by: Stephen Gutekanst <stephen@sourcegraph.com> (cherry picked from commit 1a6a7f7)
…Google vertex (#63282) [Linear Issue](https://linear.app/sourcegraph/project/claude-3-on-gcp-8c014e1a3506/overview) This PR adds support for anthropic models in the google provider through google vertex. NOTE: The current code only supported Google Gemini API and had boiler plate code for Google vertex(only for the gemini model) this PR adds Google Vertex for anthropic models properly so this way the google provider can be run in 3 different configurations 1. Google Gemini API(this works but only for chat and not for completions which is the intended behaviour for now) 2. Google Vertex API Anthropic Model(This works perfectly and is added in this PR and tested for both chat and completions and it works great) 3. Google Vertex API Gemini Model (this doesn't work yet and can eventually be added to this codebase but we gotta add a new decoder for the streaming responses of the gemini model through this API we can take care of this later) Sense of Urgency: This is a P0 because of enterprise requirements so I would appreciate a fast approval and merging. <!-- 💡 To write a useful PR description, make sure that your description covers: - WHAT this PR is changing: - How was it PREVIOUSLY. - How it will be from NOW on. - WHY this PR is needed. - CONTEXT, i.e. to which initiative, project or RFC it belongs. The structure of the description doesn't matter as much as covering these points, so use your best judgement based on your context. Learn how to write good pull request description: https://www.notion.so/sourcegraph/Write-a-good-pull-request-description-610a7fd3e613496eb76f450db5a49b6e?pvs=4 --> - Run this branch for Cody instance -> https://github.com/sourcegraph/cody/pull/4606 - Ask @arafatkatze to dm you the siteadmin config to make things work - Check the logs and play with completions and chat <!-- All pull requests REQUIRE a test plan: https://docs-legacy.sourcegraph.com/dev/background-information/testing_principles --> <!-- 1. Ensure your pull request title is formatted as: $type($domain): $what 3. Add bullet list items for each additional detail you want to cover (see example below) 4. You can edit this after the pull request was merged, as long as release shipping it hasn't been promoted to the public. 5. For more information, please see this how-to https://www.notion.so/sourcegraph/Writing-a-changelog-entry-dd997f411d524caabf0d8d38a24a878c? Audience: TS/CSE > Customers > Teammates (in that order). Cheat sheet: $type = chore|fix|feat $domain: source|search|ci|release|plg|cody|local|... --> <!-- Example: Title: fix(search): parse quotes with the appropriate context Changelog section: - When a quote is used with regexp pattern type, then ... - Refactored underlying code. --> --------- Signed-off-by: Stephen Gutekanst <stephen@sourcegraph.com> Co-authored-by: Beatrix <beatrix@sourcegraph.com> Co-authored-by: Stephen Gutekanst <stephen@sourcegraph.com> (cherry picked from commit 1a6a7f7)
Linear Issue
This PR adds support for anthropic models in the google provider through google vertex.
NOTE: The current code only supported Google Gemini API and had boiler plate code for Google vertex(only for the gemini model) this PR adds Google Vertex for anthropic models properly so this way the google provider can be run in 3 different configurations
Sense of Urgency: This is a P0 because of enterprise requirements so I would appreciate a fast approval and merging.
Test plan
Changelog