Conversation
Contributor
Author
|
Tracking: #1764. |
technoweenie
approved these changes
Dec 15, 2016
Contributor
technoweenie
left a comment
There was a problem hiding this comment.
Looks great! I made a note of a few things in #1764 (comment), but those should probably be handled in separate pull requests. Also, there are a few commented out errors in favor of ActionMissingError or ActionExpiredErr. Those should be removed, but could easily be done elsewhere too. I think we can ship this PR now.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This pull-request simplifies the
tq.Transfertype, and removes the usage of the mutableTransferableinterface. Additionally, it enables us to remove*api.ObjectResourcefrom thetq's exported API.Previously, we used three main types:
Transferable- aninterface{}to cover downloadable and uploadable LFS pointers.*api.ObjectResource- A type to hold values from the API.*transfer.Transfer- A combination ofTransferable()and*api.ObjectResource. Used in adapter implementations to grab values both from the API, and values pertaining to the local repository.The bummer is that the
Transferableis cached in amap[string]Transferableon the*TransferQueuefor the entire transfer, which can incur some pretty high memory consumption. unsafe.Sizeof suggests that the we're storing much more than 56 bytes per Transfer, not including the size of the inner-structs. My calculations estimate about ~128bytes per object (for two actions (32 bytes/action), 4 bytes for the pointer), and no error).To simplify, let's store the API data when we need it, and ignore it when we don't. For a local cache, all we need is the Name, Path, Oid and Size, which compacts to 56 bytes as a maximum (we can remove 16 bytes/transfer by dropping the
Namefield). This is, on average, 68.7% less memory consumption per unique Transfer.In addition, the code gets simpler, since we only need to care about the objects retuned back from the API when we're sending them to the adapters.
On top of all of that, we got rid of the mutability in
SetObject(), narrowed down our internal use of the*api.ObjectResourcetype, and only have one remaining external use of theapipackage intq.Since this PR is on the larger side, here are the important groups of commits:
Transferableinterface(q *TransferQueue) Add()signature./cc @git-lfs/core