Optimize pushes for multiple refs#3978
Merged
bk2204 merged 1 commit intogit-lfs:masterfrom Jan 16, 2020
Merged
Conversation
0bd8e6e to
012baa7
Compare
e86c59a to
8e32713
Compare
chrisd8088
approved these changes
Jan 16, 2020
lfs/gitscanner.go
Outdated
| } | ||
|
|
||
| // ScanMultiRangeToRemote scans through all commits starting at the left ref but | ||
| // not including the right ref (if given)that the given remote does not have. |
Member
There was a problem hiding this comment.
Nit: might add a space after (if given), but I notice the preceding comment is the same, so maybe not?
Member
Author
There was a problem hiding this comment.
Yeah, I copied and pasted, so will fix. Thanks for noticing.
When pushing multiple refs, we know any Git objects on the remote side can be excluded from the objects that refer to LFS objects we need to push, since if the remote side already has the Git objects, it should have the corresponding LFS objects as well. However, when traversing Git objects, we traverse them on a per-ref basis, which is required since any LFS objects which spawn a batch request will need the ref to be placed in the batch request as part of the protocol. Let's find a list of all the remote sides that exist before traversing any Git objects, and exclude traversing any of those objects in any traversal. As a result, we can traverse far, far fewer objects, especially when pushing new refs in a large repository. Note that we exclude the case when the left and right sides are the same because our code sets them to the same thing in some cases even though Git does not, so we cannot reason about the values in that case.
8e32713 to
d0e950d
Compare
|
bk2204:multi-left-optimization |
chrisd8088
added a commit
to chrisd8088/git-lfs
that referenced
this pull request
Feb 6, 2022
We can remove some unused code which determined the "left" and "right" refs to pass to ScanMultiRangeToRemote() when uploading, as it was obviated by the change in commit d0e950d in PR git-lfs#3978 to the use of a set of refs ("bases") from a single "right" ref.
chrisd8088
added a commit
to chrisd8088/git-lfs
that referenced
this pull request
Feb 7, 2022
We can remove some unused code which determined the "left" and "right" refs to pass to ScanMultiRangeToRemote() when uploading, as it was obviated by the change in commit d0e950d in PR git-lfs#3978 to the use of a set of refs ("bases") from a single "right" ref.
chrisd8088
added a commit
to chrisd8088/git-lfs
that referenced
this pull request
Feb 7, 2022
We can remove some unused code which determined the "left" and "right" refs to pass to ScanMultiRangeToRemote() when uploading, as it was obviated by the change in commit d0e950d in PR git-lfs#3978 to the use of a set of refs ("bases") from a single "right" ref.
chrisd8088
added a commit
to chrisd8088/git-lfs
that referenced
this pull request
Jun 6, 2023
In commit d0e950d of PR git-lfs#3978 the ScanMultiRangeToRemote() method of the GitScanner structure was added, and the sole caller of the existing ScanRangeToRemote() method was revised to call the new scan method instead. As no additional callers of the existing method have been introduced since then, we can simplify our code by removing the unused method now.
chrisd8088
added a commit
to chrisd8088/git-lfs
that referenced
this pull request
Jun 7, 2023
In commit d0e950d of PR git-lfs#3978 the ScanMultiRangeToRemote() method of the GitScanner structure was added, and the sole caller of the existing ScanRangeToRemote() method was revised to call the new scan method instead. As no additional callers of the existing method have been introduced since then, we can simplify our code by removing the unused method now.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR optimizes pushes for multiple refs by using all the remote sides as bases for crawling
git rev-list.Fixes #3976.