Skip to content

Api/batch#1827

Merged
technoweenie merged 24 commits intoapi-masterfrom
api/batch
Jan 6, 2017
Merged

Api/batch#1827
technoweenie merged 24 commits intoapi-masterfrom
api/batch

Conversation

@technoweenie
Copy link
Contributor

@technoweenie technoweenie commented Jan 4, 2017

This updates the tq.TransferQueue to use lfsapi.Client to make batch API requests, following the pattern set in #1824.

Changes:

  • Removed the need for access, operation, or remote in tq.Manifest. I initially went with the approach described in Api/tq manifest #1826, but didn't like it. Now, a *tq.Manifest is fully immutable, and can be used across multiple transfer queues for different LFS servers. This was impossible before, since access was scoped to an LFS endpoint.
  • The tq.batchResponse now stores the lfsapi.Endpoint used for the batch request. tq.toAdapterCfg() uses that Endpoint in conjunction with the transfer queue's tq.Manifest to provide the config for adapters. This is where the concurrency will be set to 1 for ntlm authenticated LFS services.
  • tq.NewTransferQueue() now requires a remote argument, since it can't use CurrentRemote in *config.Configuration.
  • Updated all http transfer adapters to use lfsapi.Client.

TODO:

@technoweenie technoweenie mentioned this pull request Jan 4, 2017
2 tasks
@ttaylorr ttaylorr mentioned this pull request Jan 4, 2017
17 tasks
Copy link
Contributor

@ttaylorr ttaylorr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks awesome, just one comment that needs your attention.

func newDownloadCheckQueue(options ...tq.Option) *tq.TransferQueue {
return lfs.NewDownloadCheckQueue(cfg, options...)
func newDownloadCheckQueue(manifest *tq.Manifest, remote string, options ...tq.Option) *tq.TransferQueue {
allOptions := make([]tq.Option, len(options), len(options)+1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be initialized with an empty length, instead of len(options)? Since you're appending below, it'll append after the zero-value elements it creates with the length argument. (From one of my PRs: 7af9281)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, better safe than sorry.

lfsapi/lfsapi.go Outdated
return c, nil
}

func (c *Client) GitEnv() env {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do these need to be public? If so, I think we should also export env, since right now we're returning a private interface, so the caller has no idea what methods are exposed.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not really, since you pass the os and git envs to NewClient(). If transfer adapters get extracted to a separate package, I think it will need to be public. I'll unexport for now, and we can export later if necessary.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Gah, should've looked at the code. This is needed in tq :(

Objects []*api.ObjectResource `json:"objects"`
}

func (c *tqClient) Batch(remote string, bReq *batchRequest) (*batchResponse, *http.Response, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😍 Really clean, this is awesome.

a.workerWait.Done()
}

func (a *adapterBase) newHTTPRequest(method string, rel *Action) (*http.Request, error) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice 👌, so cool that we got rid of the httputil dependency.

@technoweenie technoweenie changed the base branch from api/locking to api-master January 6, 2017 18:22
@ttaylorr
Copy link
Contributor

ttaylorr commented Jan 6, 2017

Pending a 🍏 test-suite, 👍

@technoweenie technoweenie merged commit 2e43f34 into api-master Jan 6, 2017
@technoweenie technoweenie deleted the api/batch branch January 6, 2017 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants