Reduce memory usage when searching for used blobs#2599
Merged
fd0 merged 5 commits intorestic:masterfrom Aug 1, 2020
Merged
Conversation
aawsome
approved these changes
Feb 23, 2020
Contributor
aawsome
left a comment
There was a problem hiding this comment.
Thanks for proposing this PR! I also stumbled over the double BlobSet already and somewhere (I think it was in #2513) already called FindUsedBlobs with the seenBlobs = usedBlobs.
I think in the long term the remaining BlobSet should be integrated into the index - after #2523 is finished.
This PR however can and should be merged independently. Good job 👍
3276544 to
17e3f22
Compare
Member
Author
greatroar
reviewed
Feb 29, 2020
17e3f22 to
363bc2a
Compare
4c063a8 to
fd94a58
Compare
fd94a58 to
ebf444d
Compare
The seen BlobSet always contained a subset of the entries in blobs. Thus use blobs instead and avoid the memory overhead of the second set. Suggested-by: Alexander Weiss <alex@weissfam.de>
The only effective change in behavior is that that toplevel nodes can also be skipped.
This also copies the TreeLoader interface from internal/walker to allow stubbing the repository in the call to `FindUsedBlobs`.
ebf444d to
2d7ab91
Compare
6 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What is the purpose of this change? What does it change?
The seen BlobSet always contained a subset of the entries in blobs.
Thus use blobs instead and avoid the memory overhead of the second set.
Was the change discussed in an issue or in the forum before?
No.
Checklist
[ ] I have added documentation for the changes (in the manual)changelog/unreleased/that describes the changes for our users (template here)gofmton the code in all commits