Skip to content
This repository was archived by the owner on Sep 30, 2024. It is now read-only.

gitserver: Implement Stat and ReadDir in gRPC API#62107

Merged
eseliger merged 1 commit into
mainfrom
es/ls-tree-grpc-2
May 10, 2024
Merged

gitserver: Implement Stat and ReadDir in gRPC API#62107
eseliger merged 1 commit into
mainfrom
es/ls-tree-grpc-2

Conversation

@eseliger

@eseliger eseliger commented Apr 23, 2024

Copy link
Copy Markdown
Member

This PR migrates the calls to git ls-tree to gRPC.

Since ReadDir and Stat used the same git command under the hood, I migrated those two together.

Stat and ReadDir are now simply moved to the backend with the usual layering we did so far.

ListDirectoryChildren effectively was a ReadDir with some post-request processing into a map, so I moved that logic to the only caller instead and made it just use ReadDir.

LsFiles uses the ls-files command which is meant for worktrees and is not a plumbing command so I tried to best replicate its use-cases using ReadDir instead.
There were three use-cases for LsFiles so far:

  1. Exposing all files in the repo from GQL for the fuzzy finder (this is a terrible API as it'll create potentially humongous responses but that's not a fault of our API), and I made it use ReadDir with recurse: true instead and I filter out dir entries.
  2. The own indexer used it to go over all the files, I made that use ReadDir instead as well.
  3. In inference, it was used to get files matching certain patterns. Git ls-tree does not support globbing while ls-files does, so we can not do that anymore. BUT: the logic already did filtering in-app, it just used that to reduce the result set to iterate over. So we decided that for now we will go with a ReadDir and then later we can try to find a better replacement here.

Closes https://github.com/sourcegraph/sourcegraph/issues/47420

Closes https://github.com/sourcegraph/sourcegraph/issues/60413

Closes https://github.com/sourcegraph/sourcegraph/issues/60414

Test plan

Verified that inference still returns the same results. Also browsed files, opened file trees etc from the web app. Existing E2E test suites also still pass.

@cla-bot cla-bot Bot added the cla-signed label Apr 23, 2024
@github-actions github-actions Bot added team/product-platform team/source Tickets under the purview of Source - the one Source to graph it all labels Apr 23, 2024
@eseliger eseliger force-pushed the es/ls-tree-grpc-2 branch 13 times, most recently from ac42853 to dcd785d Compare May 7, 2024 01:18
Comment thread cmd/gitserver/internal/git/gitcli/odb.go Outdated
Comment thread internal/codeintel/autoindexing/internal/inference/service.go Outdated
@eseliger eseliger force-pushed the es/ls-tree-grpc-2 branch from f2bf72e to ecd1764 Compare May 10, 2024 19:16
@eseliger eseliger force-pushed the es/ls-tree-grpc-2 branch 2 times, most recently from 6d0b40a to 0a08be5 Compare May 10, 2024 19:28
@eseliger eseliger marked this pull request as ready for review May 10, 2024 20:09
@eseliger eseliger requested a review from a team May 10, 2024 20:09
Comment thread cmd/gitserver/internal/git/gitcli/odb.go Outdated
Comment thread cmd/gitserver/internal/git/gitcli/odb_test.go Outdated
Comment thread cmd/gitserver/internal/git/gitcli/odb_test.go Outdated
Comment thread cmd/gitserver/internal/server_grpc.go Outdated
Comment thread cmd/gitserver/internal/server_grpc.go Outdated
Comment thread go.mod Outdated
Comment thread cmd/gitserver/internal/git/iface.go Outdated
@eseliger eseliger force-pushed the es/ls-tree-grpc-2 branch from 4e1c78e to 4efccd4 Compare May 10, 2024 22:29
@eseliger eseliger merged commit 824337a into main May 10, 2024
@eseliger eseliger deleted the es/ls-tree-grpc-2 branch May 10, 2024 22:58

Copy link
Copy Markdown
Member Author

Merge activity

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

cla-signed team/product-platform team/source Tickets under the purview of Source - the one Source to graph it all

Projects

None yet

Development

Successfully merging this pull request may close these issues.

gitserver: Move ReadDir to new gRPC pattern gitserver: Move Stat to new gRPC pattern gRPC: migrate gitserver.LsFiles

2 participants