Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

fix(cody): remove client check for context filters#63855

Merged
olafurpg merged 1 commit into
mainfrom
olafurpg-cody-2888-fix-version-compatibility-issue-in-enterprise-instances
Jul 18, 2024
Merged

fix(cody): remove client check for context filters#63855
olafurpg merged 1 commit into
mainfrom
olafurpg-cody-2888-fix-version-compatibility-issue-in-enterprise-instances

Conversation

@olafurpg

@olafurpg olafurpg commented Jul 16, 2024

Copy link
Copy Markdown
Contributor

Fixes CODY-2888

Previously, Sourcegraph Enterprise instances with context filters enabled rejected requests from all unknown clients out of concern that they might not respect context filters. This behavior makes it incredibly impractical to release now agent-based clients (CLI, Eclipse, Visual Studio, Neovim, ..) that do respect context filters out of the box thanks to the reused logic in the Cody agent.

This logic suffers from both false positives and false negatives:

  • False negatives: upcoming Cody clients (CLI, Eclipse, Visual Studio) already support context filters out of the box thanks to using the Cody agent but they can't send requests unless we add a special case to them. It may require months for these clients to wait for all Enterprise instances to upgrade to a version that adds exceptions for their name.
  • False positive: a malicious client can always fake that it's "jetbrains" with a valid version number even if the client doesn't respect context filters. This gives a false sense of security because it doesn't prevent malicious traffic from bypassing context filters. In fact, I am leaning towards using the

Now, with this change, Sourcegraph Enterprise instances only reject requests from old versions of Cody clients that are known to not support context filters. This ensures we never have false positives or false negatives.

Test plan

See updated test case which accepts a request from an unknown "sublime" client.

Changelog

@olafurpg olafurpg requested review from a team and taras-yemets July 16, 2024 14:26
@cla-bot cla-bot Bot added the cla-signed label Jul 16, 2024
Base automatically changed from olafurpg/revert-bad-commit to main July 16, 2024 14:27
@olafurpg olafurpg force-pushed the olafurpg-cody-2888-fix-version-compatibility-issue-in-enterprise-instances branch from 3496c1b to f4e978c Compare July 16, 2024 14:29
olafurpg referenced this pull request in sourcegraph/cody-public-snapshot Jul 16, 2024
Previously, the `--context-repo` option did not work with the CLI
because of two issues:

- The CLI used the client name "cody-cli", which Sourcegraph Enterprise
  rejects due to a problem that is described in the PR https://github.com/sourcegraph/sourcegraph/pull/63855
  To work around this problem, the CLI will use the client name
  "jetbrains" for time being.
- The `addEnhancedOptions` was set to false in `chat/submitMessage`. We
  now set this option to `true` when `--context-repo` or
  `--context-file` is enabled.
olafurpg referenced this pull request in sourcegraph/cody-public-snapshot Jul 16, 2024
Previously, the `--context-repo` option did not work with the CLI
because of two issues:

- The CLI used the client name "cody-cli", which Sourcegraph Enterprise
  rejects due to a problem that is described in the PR https://github.com/sourcegraph/sourcegraph/pull/63855
  To work around this problem, the CLI will use the client name
  "jetbrains" for time being.
- The `addEnhancedOptions` was set to false in `chat/submitMessage`. We
  now set this option to `true` when `--context-repo` or
  `--context-file` is enabled.
olafurpg referenced this pull request in sourcegraph/cody-public-snapshot Jul 16, 2024
We have to bump the version to 5.5.9 to bypass the server checks that
will (most likely) be removed in
https://github.com/sourcegraph/sourcegraph/pull/63855
olafurpg referenced this pull request in sourcegraph/cody-public-snapshot Jul 16, 2024
We have to bump the version to 5.5.9 to bypass the server checks that
will (most likely) be removed in
https://github.com/sourcegraph/sourcegraph/pull/63855
Comment thread cmd/frontend/internal/httpapi/completions/handler.go Outdated
@taras-yemets taras-yemets self-requested a review July 17, 2024 14:44
@@ -83,10 +83,7 @@ func TestCheckClientCodyIgnoreCompatibility(t *testing.T) {
q: url.Values{
"client-name": []string{"sublime"},

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we require both clientName and clientVersion params but check the version only for VS Code and JetBrains

This test still should fail based on this requirement (thread)

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. Fixed.

// (JetBrains, Eclipse, Visual Studio) support context filters out of
// the box since the original support was added for JetBrains GA in May
// 2024.
return nil

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to check if the client version is provided. If it's not, the function should still return an error (thread).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I've updated the logic to accept an empty version for web only, and require semver for other clients. Added relevant tests

@olafurpg olafurpg force-pushed the olafurpg-cody-2888-fix-version-compatibility-issue-in-enterprise-instances branch from 6cd6ac5 to 2da103e Compare July 18, 2024 11:26
@olafurpg olafurpg requested a review from taras-yemets July 18, 2024 11:27
@olafurpg olafurpg force-pushed the olafurpg-cody-2888-fix-version-compatibility-issue-in-enterprise-instances branch from 2da103e to 0ff6d30 Compare July 18, 2024 11:28
@olafurpg olafurpg enabled auto-merge (squash) July 18, 2024 11:34
@olafurpg

Copy link
Copy Markdown
Contributor Author

Looks like the meat of this PR got merged via https://github.com/sourcegraph/sourcegraph/commit/277648821ccf929574b910ac027b1dfcf1235bf2 This PR now only includes changes to address the review comments

// All agent-based clients (JetBrains, Eclipse, Visual Studio) support
// context filters out of the box since the original support was added
// for JetBrains GA in May 2024.
cvc = clientVersionConstraint{client: clientName, constraint: ">= 0.0.0"}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should also consider the value of the cody-context-filters-clients-test-mode feature flag (refer to the logic a few lines above). Pre-release versions might not satisfy the ">= 0.0.0" condition. I'm not completely sure about this, so it's worth double-checking.

@taras-yemets

Copy link
Copy Markdown
Contributor

Looks like the meat of this PR got merged via https://github.com/sourcegraph/sourcegraph/commit/277648821ccf929574b910ac027b1dfcf1235bf2 This PR now only includes changes to address the review comments

@olafurpg, The referenced PR only removed the feature flag that gated the Cody context filters usage before client support was in place. Without this flag enabled, admins would receive an error when trying to define "cody.contextFilters" in the site config.

Your PR is necessary to allow support for clients other than vscode, jetbrains, and web clients.

@olafurpg olafurpg merged commit d4653a8 into main Jul 18, 2024
@olafurpg olafurpg deleted the olafurpg-cody-2888-fix-version-compatibility-issue-in-enterprise-instances branch July 18, 2024 11:44
olafurpg referenced this pull request in sourcegraph/eclipse Jul 19, 2024
Previously, we send an appropriate Cody agent client name "eclipse".
This was problematic because it meant that the plugin didn't work with
Enterprise server instances that have enabled context filters due to a
check against old versions of clients that may not support context
filters. The server checks have been fixed in https://github.com/sourcegraph/sourcegraph/pull/63855
but we need to wait a few months before being able to send "Eclipse" as
a client name when Enterprise users have completed upgrading to a newer
Sourcegraph version.

The big downside with this solution is that it means all the telemetry
in the plugin indicates that events are coming from JetBrains. We can
fix this problem by extending the Cody agent server protocol to accept
a dedicated setting to override only the HTTP client name for LLM
completion requests. This work is tracked in the issue CODY-2978.
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants