lfsapi Proposal

Today, in order to interact with an API inside git-lfs, you have to import the
api package, which depends on httputil, auth, and of course, a
*config.Configuration. That adds up to a pretty heavy dependency.

Packages should be able to interact with the LFS API without importing such
a large set of dependencies. To do this, we can evolve the same philosophy that
applied to the config package: by reducing the scope of domain knowledge that
the package (api) owns, we can distribute calling code that uses it further
as the dependency will loose weight.

This much more closely matches the philosophy of the net/http package.
Clients would be able to make their own *http.Request instances, passing
them into the *api.Client.Do function.

This package should allow easy access to the following domains, while still keeping things lightweight:

• Endpoint resolving (experimented with in Tq/extract endpoint #1770)
• Git Credentials
• Netrc
• NTLM
• SSH auth
• HTTP request logging and redirection handling

By reducing the set of primitives exposed by an API package, we can remove the
tight coupling that currently exists in the locking package between service
classes and API calls.

Right now, that looks something like:

s, resp := c.apiClient.Locks.Lock(&api.LockRequest{
	Path:               path,
	Committer:          api.NewCommitter(c.cfg.CurrentCommitter()),
	LatestRemoteCommit: latest.Sha,
})

if _, err := c.apiClient.Do(s); err != nil {
	return Lock{}, fmt.Errorf("Error communicating with LFS API: %v", err)
}

if len(resp.Err) > 0 {
	return Lock{}, fmt.Errorf("Server unable to create lock: %v", resp.Err)
}

lock := c.newLockFromApi(*resp.Lock)

But it could be reduced down to:

lockReq := &api.LockRequest{
  Path:               path,
  Committer:          api.NewCommitter(c.cfg.CurrentCommitter()),
  LatestRemoteCommit: latest.Sha,
}

endpoint := apiClient.Endpoint("master", "download")
req, _ := http.NewRequest("POST", "/locks", jsonToIOReader(lockReq))
res, _ := apiClient.Do(endpoint, req)
defer res.Body()

var lockRes LockResponse
unmarshalBody(res.Body, &lockRes)

This enables packages like tq to define their own API interfaces,
perhaps like:

package tq

type Batcher struct {
	c *lfsapi.Client
}

func (b *Batcher) Batch(direction string, oids []string) (*BatchResponse, error) {
	// ...
}

As a critical note: lfsapi would no longer be the single source of all LFS
APIs. Instead, it will be a package that is depended on by other packages which
themselves implement the API. This corrects our package structure to be much more
in-line with Go's packaging philosphy.

In addition, the code is much closer to how Go HTTP requests look. However, how
do we deal with non-http protocols? We can use Go's ability to
register custom protocols in net/http:

t := &http.Transport{}
t.RegisterProtocol("ssh", sshRoundTripper())
c := &http.Client{Transport: t}

Implementing a custom http roundtripper is easy. Implementing SSH
could be as simple as converting a request that looks like:

ssh://git@gitserver.com/git-lfs-authenticate?args=foo/bar%20download

To an SSH command like:

ssh git@gitserver.com git-lfs-authenticate foo/bar download

A Go roundtripper function would then convert the SSH command results
into an *http.Response that the client code expects.

We could also use this to support file:// natively, letting LFS
work across local repositories.

• Batch API Api/batch #1827
• BatchSingle API
• Verify API
• Locking API api refactor: locking #1824
• TUS Adapter Api/batch #1827
• Basic Download Adapter Api/batch #1827
• Basic Upload Adapter Api/batch #1827
• Per-Host SSL Certs Api/clients #1787
• HTTP Proxy Api/clients #1787
• HTTP timeouts Api/clients #1787
• NTLM Support Api/ntlm #1838
• Netrc Support Api/auth #1784
• Git Credential Support Api/auth #1784
• SSH Credential Support Api/ssh auth #1837
• HTTP Redirections lfsapi: teach (*Client) Do() how to handle 307 redirections #1791
• UserAgent lfsapi: teach (*Client) Do() how to handle 307 redirections #1791
• HTTP Verbose Logging and Stats Api/tracing #1794

/cc @git-lfs/core