Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

Add SourcegraphModelConfig.AccessToken#63619

Merged
chrsmith merged 1 commit into
mainfrom
chrsmith/fix-design-flaw
Jul 3, 2024
Merged

Add SourcegraphModelConfig.AccessToken#63619
chrsmith merged 1 commit into
mainfrom
chrsmith/fix-design-flaw

Conversation

@chrsmith

@chrsmith chrsmith commented Jul 3, 2024

Copy link
Copy Markdown
Contributor

Fixes a small design oversight when creating the SourcegraphModelConfig.

The SourcegraphModelConfig type contains all the configuration knobs for a Sourcegraph admin to control how the "Sourcegraph-supplied LLM models" should be used. For example:

  • If set to null, then Sourcegraph will not provide any LLM models for the instance. And only the admin-supplied providers and models will be used.
  • It can be configured to poll Cody Gateway to automatically include "new models" so they can be made available to users of the instance as soon as they are served from Cody Gateway.

However, there's something that is lost in translation between "site admin configuring how LLM models get picked up" and "generating the types.ModelConfiguration data structure needed at runtime".

In order to actually use Cody Gateway at runtime, the Completions Provider needs to know the URL endpoint and access token for Cody Gateway. This is made available via the types.Provider.ServerSideConfig.SourcegraphProvider field. (So when creating the "google" completions provider, that's how we know to send the request to Cody Gateway, etc.)

But with the data type we have right now, this data is missing.

With the current way you can configure Cody Enterprise, the accessToken field is optional. Because when we load the configuration, the Sourcegraph instance will automatically create a new access token based on the license key.

But as we transition to the new-style configuration data, we will no longer be able to rely on CompletionsConfig.AccessToken.

So specifically:

  1. We need to add this field to the types.SourcegraphModelConfig config data type, so that when we are building the types.ModelConfiguration we know how to create the provider information to contact Cody Gateway as needed.
  2. We can make this optional or remove it completely in how the Sourcegraph admin "enters/encodes" the types.SourcegraphModelConfig in their Site configuration data. (And continue to just generate a token based on the license key.)
// conf/computed.go
func getSourcegraphProviderAccessToken(accessToken string, config schema.SiteConfiguration) string {
	// If an access token is configured, use it.
	if accessToken != "" {
		return accessToken
	}

	// Otherwise, use the current license key to compute an access token.
	if config.LicenseKey == "" {
		return ""
	}

	return license.GenerateLicenseKeyBasedAccessToken(config.LicenseKey)
}

Test plan

NA

Changelog

NA

@emidoots emidoots left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense, yep.

@chrsmith chrsmith merged commit 93a5f0c into main Jul 3, 2024
@chrsmith chrsmith deleted the chrsmith/fix-design-flaw branch July 3, 2024 21:08
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants