Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

Change chat defaults to Sonnet and autocomplete defaults to StarCoder or Haiku#62757

Merged
philipp-spiess merged 4 commits into
mainfrom
ps/change-model-defaults
May 17, 2024
Merged

Change chat defaults to Sonnet and autocomplete defaults to StarCoder or Haiku#62757
philipp-spiess merged 4 commits into
mainfrom
ps/change-model-defaults

Conversation

@philipp-spiess

@philipp-spiess philipp-spiess commented May 17, 2024

Copy link
Copy Markdown
Contributor

Closes CODY-1626

This extends the recent changes to our Cloud defaults to on-prem instances. StarCoder is now opt-out. Defaults for Claude have also changed.

Expected new changes:

  • On Cody Gateway:
    • Chat: Claude 3 Sonnet
    • Fast Chat: Claude 3 Haiku
    • Completion: StarCoder
  • BYOK Anthropic:
    • Chat: Claude 3 Sonnet
    • Fast Chat: Claude 3 Haiku
    • Completion: Claude 3 Haiku

Open question: Should/can we make changes to the Bedrock config too?

Test plan

  • Updated unit tests, can rely on CI to check the rest.

@philipp-spiess philipp-spiess requested review from a team, eseliger and rafax May 17, 2024 09:46
@cla-bot cla-bot Bot added the cla-signed label May 17, 2024
@philipp-spiess philipp-spiess merged commit 1e3cb2b into main May 17, 2024
@philipp-spiess philipp-spiess deleted the ps/change-model-defaults branch May 17, 2024 12:04
@aramaraju

Copy link
Copy Markdown

Wonderful! Let's not do the changes for Bedrock, we have learned (the hard way) that Provisioned Throughput might need different modelIDs in the config

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants