Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

Filters candidate matches against syntactic SCIP documents#63268

Merged
kritzcreek merged 20 commits into
mainfrom
christoph/fetch-syntactic-scip-documents
Jun 19, 2024
Merged

Filters candidate matches against syntactic SCIP documents#63268
kritzcreek merged 20 commits into
mainfrom
christoph/fetch-syntactic-scip-documents

Conversation

@kritzcreek

@kritzcreek kritzcreek commented Jun 14, 2024

Copy link
Copy Markdown
Contributor

Closes https://linear.app/sourcegraph/issue/GRAPH-657/fetch-and-filter-scip-documents-based-on-candidate-files

Test plan

Currently only testable via the site-admin API console

@cla-bot cla-bot Bot added the cla-signed label Jun 14, 2024
@github-actions github-actions Bot added team/graph Graph Team (previously Code Intel/Language Tools/Language Platform) team/product-platform labels Jun 14, 2024
@kritzcreek kritzcreek force-pushed the christoph/fetch-syntactic-scip-documents branch 3 times, most recently from c29d4cc to 1ec73fd Compare June 17, 2024 10:27
@kritzcreek kritzcreek requested a review from a team June 17, 2024 12:51
@kritzcreek kritzcreek marked this pull request as ready for review June 17, 2024 12:51
@kritzcreek kritzcreek force-pushed the christoph/fetch-syntactic-scip-documents branch from be81f9b to 428b19c Compare June 17, 2024 12:59
Comment thread internal/codeintel/codenav/service.go Outdated

@varungandhi-src varungandhi-src left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some initial comments.

Comment thread internal/codeintel/codenav/service.go Outdated
Comment thread internal/codeintel/codenav/service.go Outdated
Comment thread internal/codeintel/codenav/service.go Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. Let's create a dedicated type for holding both results. That way the value can be passed around, and the relationship between the values is clear when the return value is passed around ("these symbols were obtained in this uploadID").
  2. nit: Go convention would be to use "ID", not "Id"

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't agree with 1.

Once we also allow getting the symbol from the request arg rather than by reading a range in a syntactic index, we'll be getting an "unrelated" uploadId instead.

The uploadId is then also used to look up a bunch of other documents that have nothing to do with the symbols.

Comment thread internal/codeintel/codenav/service.go Outdated
Comment thread internal/codeintel/codenav/service.go Outdated
Comment thread internal/codeintel/codenav/syntactic.go Outdated
Comment thread internal/codeintel/codenav/syntactic.go Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  1. I know the logging code internally does some de-duplication, but I think since this loop (over stream.Results) is all in one function, it would be better to record these errors separately and emit a single log message after the loop is complete. E.g. you can collect the files seen multiple times in a collections.Set.
  2. I think all of these should be Warn, and not Error. Is the code behaving in an unexpected way? Yes. Can we continue operating without worrying about some functionality not working properly? I'd say so, we can skip over these unexpected outputs.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly at that point I'd rather just drop these asserts. I don't want to clutter our logic here with checking invariants in the search client/service. Sounds good?

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have tests for enry's code so that if something changes there in a way that would break us, we get to know sooner rather than later. I see these warnings in the same vein, especially since these are not documented post-conditions of the functions that we're calling.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wrt https://github.com/sourcegraph/sourcegraph/pull/63268/commits/5ccd3a1deb97c949859896098cf2b6b5b2106014, I would prefer that we catch bugs sooner rather than later, so weakly against removing those checks, but leaving the final decision up to you.

@kritzcreek kritzcreek Jun 19, 2024

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

8638879 looking good? I just assumed .String() on a set does something reasonable

Comment thread internal/codeintel/codenav/syntactic.go Outdated

@varungandhi-src varungandhi-src left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving to unblock; feel free to merge as I don't have major outstanding concerns.

@kritzcreek kritzcreek force-pushed the christoph/fetch-syntactic-scip-documents branch from d2c61ec to 793c24b Compare June 19, 2024 13:39
@kritzcreek kritzcreek merged commit 7ef56a6 into main Jun 19, 2024
@kritzcreek kritzcreek deleted the christoph/fetch-syntactic-scip-documents branch June 19, 2024 13:58
kritzcreek pushed a commit that referenced this pull request Jun 21, 2024
Fixes
https://linear.app/sourcegraph/issue/GRAPH-667/wrap-up-usage-results-in-graphql-resolvers

This allows us to actually return syntactic usages from the occurrences
API. It does not implement pagination yet, but I think _that_ PR will be
nicer to review after this one landed.

Requires #63268

## Test plan

Running queries against the site-admin api console
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

cla-signed team/graph Graph Team (previously Code Intel/Language Tools/Language Platform) team/product-platform

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants