Skip to content

support for client certificates#1869

Closed
drCriX wants to merge 9 commits intogit-lfs:masterfrom
drCriX:master
Closed

support for client certificates#1869
drCriX wants to merge 9 commits intogit-lfs:masterfrom
drCriX:master

Conversation

@drCriX
Copy link
Contributor

@drCriX drCriX commented Jan 16, 2017

allow for sslKey and sslCert in gitconfig http subsection

sslKey and sslCert in http subsection
@drCriX
Copy link
Contributor Author

drCriX commented Jan 16, 2017

I have accepted the "Contributor License Agreement" now...

Copy link
Contributor

@technoweenie technoweenie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks good (minus my really minor style issues), but I really want an integration test for this too. Can you provide instructions on building a sample cert for this?

Our integration tests (the bash files in the test dir) do run against a local SSL web server, which is configured in test/cmd/lfstest-gitserver.go, add in #1067.

It'd be great if you could take a stab at this. But I understand if you can't. Can you just make sure the "Allow edits from maintainers" checkbox is checked for this PR?

lfsapi/certs.go Outdated
// isClientCertEnabledForHost returns whether client certificate
// are configured for the given host
func isClientCertEnabledForHost(c *Client, host string) bool {

Copy link
Contributor

@technoweenie technoweenie Jan 17, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I apologize for the nitpicky review comments, but white space after { and before } really bugs me.

lfsapi/certs.go Outdated
// getClientCertForHost returns a client certificate for a specific host (which may
// be "host:port" loaded from the gitconfig
func getClientCertForHost(c *Client, host string) tls.Certificate {

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Another unnecessary blank line.

lfsapi/client.go Outdated

if isClientCertEnabledForHost(c, host) {

tracerx.Printf("using certs ...")
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two things:

  1. Can you remove the blank space above the tracerx line, and below the BuildNameToCertificate() line?
  2. Can you change the tracerx message to: tracerx.Printf("http: client cert for %s", host)

Copy link
Contributor

@ttaylorr ttaylorr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pending mine and @technoweenie's comments, this looks 👍

lfsapi/certs.go Outdated
_, hostSslCertOk := c.gitEnv.Get(fmt.Sprintf("http.https://%v/.sslCert", host))
_, hostSslCaInfoOk := c.gitEnv.Get(fmt.Sprintf("http.https://%v/.sslCaInfo", host))

if hostSslKeyOk && hostSslCertOk && hostSslCaInfoOk {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would recommend simplifying this to:

return hostSslKeyOk && hostSslCertOk && hostSslCaInfoOk

// be "host:port" loaded from the gitconfig
func getClientCertForHost(c *Client, host string) tls.Certificate {

hostSslKey, _ := c.gitEnv.Get(fmt.Sprintf("http.https://%v/.sslKey", host))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before fetching these values, do you think it would be good to call isClientCertEnabledForHost with the c and host args to bail out early if certificates aren't enabled for that given host?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That happens in lfsapi/client.go already. I don't think we need to call isClientCertEnabledForHost() twice.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, great catch -- thanks!

@drCriX
Copy link
Contributor Author

drCriX commented Jan 24, 2017

I added an integration test based on this work:
https://ericchiang.github.io/post/go-tls/

@drCriX
Copy link
Contributor Author

drCriX commented Jan 24, 2017

I have seen this problem before at our real server.I had to go back to release version 1.5.5.
Do we have to extend the push/locking code also?

WARNING: Unable to search for locks contained in this push. Temporarily skipping check ... batch response: http: Post https://127.0.0.1:1231/test-cloneClientCert.git/info/lfs/objects/batch: remote error: tls: bad certificate

@technoweenie
Copy link
Contributor

What are you referring to specifically?

WARNING: Unable to search for locks contained in this push. Temporarily skipping check ...

You can ignore this. It has to do with file locking. More info at #1884

batch response: http: Post https://127.0.0.1:1231/test-cloneClientCert.git/info/lfs/objects/batch: remote error: tls: bad certificate

This should be covered by this PR. I'll do some digging.

Thanks for adding the integration test!

@technoweenie
Copy link
Contributor

I'm unable to get this working locally:

test: clone ClientCert ...                                         FAILED
-- stdout --
    set up remote git repository: test-cloneClientCert
    Initialized empty Git repository in /Users/rick/go/src/github.com/git-lfs/git-lfs/test/remote/test-cloneClientCert.git/
    clone https://127.0.0.1:57439/test-cloneClientCert to test-cloneClientCert
-- stderr --
    + set -e
    + reponame=test-cloneClientCert
    + setup_remote_repo test-cloneClientCert
    + local reponame=test-cloneClientCert
    + echo 'set up remote git repository: test-cloneClientCert'
    + repodir=/Users/rick/go/src/github.com/git-lfs/git-lfs/test/remote/test-cloneClientCert.git
    + mkdir -p /Users/rick/go/src/github.com/git-lfs/git-lfs/test/remote/test-cloneClientCert.git
    + cd /Users/rick/go/src/github.com/git-lfs/git-lfs/test/remote/test-cloneClientCert.git
    + git init --bare
    + git config http.receivepack true
    + git config receive.denyCurrentBranch ignore
    + clone_repo_clientcert test-cloneClientCert test-cloneClientCert
    + cd /Users/rick/p/git-lfs-temp/test-clone.sh-79634
    + local reponame=test-cloneClientCert
    + local dir=test-cloneClientCert
    + echo 'clone https://127.0.0.1:57439/test-cloneClientCert to test-cloneClientCert'
    ++ git clone https://127.0.0.1:57439/test-cloneClientCert test-cloneClientCert
    + out='Cloning into '\''test-cloneClientCert'\''...
    2017-01-24 10:48:03.166 git-remote-https[80553:5519534] *** Terminating app due to uncaught exception '\''NSInvalidArgumentException'\'', reason: '\''*** -[__NSPlaceholderArray initWithObjects:count:]: attempt to insert nil object from objects[0]'\''
    *** First throw call stack:
    (
    	0   CoreFoundation                      0x00007fffd0158e7b __exceptionPreprocess + 171
    	1   libobjc.A.dylib                     0x00007fffe4d43cad objc_exception_throw + 48
    	2   CoreFoundation                      0x00007fffd004e804 -[__NSPlaceholderArray initWithObjects:count:] + 276
    	3   libcurl.4.dylib                     0x00007fffe463c46f darwinssl_connect_common + 2166
    	4   libcurl.4.dylib                     0x00007fffe463aa95 Curl_ssl_connect_nonblocking + 77
    	5   libcurl.4.dylib                     0x00007fffe4602681 https_connecting + 23
    	6   libcurl.4.dylib                     0x00007fffe460264c Curl_http_connect + 71
    	7   libcurl.4.dylib                     0x00007fffe4610c9d Curl_protocol_connect + 127
    	8   libcurl.4.dylib                     0x00007fffe4624f3f multi_runsingle + 921
    	9   libcurl.4.dylib                     0x00007fffe4624b2d curl_multi_perform + 92
    	10  git-remote-https                    0x0000000107f5dac6 step_active_slots + 25
    	11  git-remote-https                    0x0000000107f5db37 run_active_slot + 77
    	12  git-remote-https                    0x0000000107f5de02 run_one_slot + 41
    	13  git-remote-https                    0x0000000107f5fdb9 http_request + 1499
    	14  git-remote-https                    0x0000000107f5dfae http_request_reauth + 34
    	15  git-remote-https                    0x0000000107f5b424 discover_refs + 606
    	16  git-remote-https                    0x0000000107f5a415 cmd_main + 2029
    	17  git-remote-https                    0x0000000107f60ee4 main + 76
    	18  libdyld.dylib                       0x00007fffe5627255 start + 1
    	19  ???                                 0x0000000000000003 0x0 + 3
    )
    libc++abi.dylib: terminating with uncaught exception of type NSException'
    + test_status=128
-- git trace --
   10:48:03.122447 git.c:371               trace: built-in: git 'init' '--bare'
   10:48:03.131329 git.c:371               trace: built-in: git 'config' 'http.receivepack' 'true'
   10:48:03.135461 git.c:371               trace: built-in: git 'config' 'receive.denyCurrentBranch' 'ignore'
   10:48:03.140054 git.c:371               trace: built-in: git 'clone' 'https://127.0.0.1:57439/test-cloneClientCert' 'test-cloneClientCert'
   10:48:03.146377 run-command.c:350       trace: run_command: 'git-remote-https' 'origin' 'https://127.0.0.1:57439/test-cloneClientCert'

This exception is coming from git-remote-https. It looks like Git just passes these values to curl, and it appears that OSX curl does not properly support ssl client certs. I'm okay with this feature being disabled on the Mac.

Good news is that this fails exactly the same way in Linux (the Travis CI builds) and Windows (AppVeyor). The clone test you added is a good one, because that command is testing that both git and git-lfs are reading the same git config keys to setup the ssl client cert.

@drCriX
Copy link
Contributor Author

drCriX commented Jan 24, 2017

on os x you have to install homebrew openssl git and curl versions.

http://stackoverflow.com/a/40019914

@technoweenie
Copy link
Contributor

Not much luck on my end. I did write a test using the new locking API:

diff --git a/test/test-lock.sh b/test/test-lock.sh
index c5bf135e..93cc8695 100755
--- a/test/test-lock.sh
+++ b/test/test-lock.sh
@@ -17,6 +17,21 @@ begin_test "creating a lock"
 )
 end_test

+begin_test "create lock with server using client cert"
+(
+  set -e
+  reponame="lock_create_client_cert"
+  setup_remote_repo_with_file "$reponame" "a.dat"
+
+  git config lfs.url "$CLIENTCERTGITSERVER/$reponame.git/info/lfs"
+  GITLFSLOCKSENABLED=1 git lfs lock "a.dat" | tee lock.log
+  grep "'a.dat' was locked" lock.log
+
+  id=$(grep -oh "\((.*)\)" lock.log | tr -d "()")
+  assert_server_lock "$reponame" "$id"
+)
+end_test
+
 begin_test "creating a lock (--json)"
 (
   set -e

The difference between this and the git lfs clone test, is that it only exercises git lfs. So, it doesn't matter what version of git/curl/openssl I have. Still no luck though:

test: create lock with server using client cert ...                FAILED
-- stdout --
    set up remote git repository: remote_lock_create_client_cert
    Initialized empty Git repository in /Users/rick/go/src/github.com/git-lfs/git-lfs/test/remote/remote_lock_create_client_cert.git/
    clone local git repository remote_lock_create_client_cert to clone_lock_create_client_cert
    Cloning into 'clone_lock_create_client_cert'...
    warning: You appear to have cloned an empty repository.
    Tracking a.dat
    [master (root-commit) 7e380b7] add a.dat
     2 files changed, 4 insertions(+)
     create mode 100644 .gitattributes
     create mode 100644 a.dat
    [master (root-commit) 7e380b7] add a.dat
     2 files changed, 4 insertions(+)
     create mode 100644 a.dat
     create mode 100644 .gitattributes
Git LFS: (1 of 1 files) 6 B / 6 B
    Consider unlocking your own locked file(s): (`git lfs unlock <path>`)
    * a.dat
    To http://127.0.0.1:62536/remote_lock_create_client_cert
     * [new branch]      master -> master
     * [new branch]      master -> master
-- stderr --
    + set -e
    + reponame=lock_create_client_cert
    + setup_remote_repo_with_file lock_create_client_cert a.dat
    + local reponame=lock_create_client_cert
    + local filename=a.dat
    + setup_remote_repo remote_lock_create_client_cert
    + local reponame=remote_lock_create_client_cert
    + echo 'set up remote git repository: remote_lock_create_client_cert'
    + repodir=/Users/rick/go/src/github.com/git-lfs/git-lfs/test/remote/remote_lock_create_client_cert.git
    + mkdir -p /Users/rick/go/src/github.com/git-lfs/git-lfs/test/remote/remote_lock_create_client_cert.git
    + cd /Users/rick/go/src/github.com/git-lfs/git-lfs/test/remote/remote_lock_create_client_cert.git
    + git init --bare
    + git config http.receivepack true
    + git config receive.denyCurrentBranch ignore
    + clone_repo remote_lock_create_client_cert clone_lock_create_client_cert
    + cd /Users/rick/p/git-lfs-temp/test-lock.sh-7403
    + local reponame=remote_lock_create_client_cert
    + local dir=clone_lock_create_client_cert
    + echo 'clone local git repository remote_lock_create_client_cert to clone_lock_create_client_cert'
    ++ git clone http://127.0.0.1:62536/remote_lock_create_client_cert clone_lock_create_client_cert
    + out='Cloning into '\''clone_lock_create_client_cert'\''...
    warning: You appear to have cloned an empty repository.'
    + cd clone_lock_create_client_cert
    + git config credential.helper lfstest
    + echo 'Cloning into '\''clone_lock_create_client_cert'\''...
    warning: You appear to have cloned an empty repository.'
    + echo 'Cloning into '\''clone_lock_create_client_cert'\''...
    warning: You appear to have cloned an empty repository.'
    + git lfs track a.dat
    + echo a.dat
    + git add .gitattributes a.dat
    + git commit -m 'add a.dat'
    + tee commit.log
    + grep 'master (root-commit)' commit.log
    + grep '2 files changed' commit.log
    + grep 'create mode 100644 a.dat' commit.log
    + grep 'create mode 100644 .gitattributes' commit.log
    + git push origin master
    + tee push.log
    + grep 'master -> master' push.log
    + git config lfs.url https://127.0.0.1:62538/lock_create_client_cert.git/info/lfs
    + tee lock.log
    + GITLFSLOCKSENABLED=1
    + git lfs lock a.dat
    Lock failed: api: http: Post https://127.0.0.1:62538/lock_create_client_cert.git/info/lfs/locks: remote error: tls: bad certificate
    + grep ''\''a.dat'\'' was locked' lock.log
    + test_status=1

@drCriX
Copy link
Contributor Author

drCriX commented Jan 24, 2017

I added your locking-test, but it fails:

Lock failed: Server unable to create lock: lock already created

@technoweenie
Copy link
Contributor

Ah, can you change the filename from a.dat to something else? Our test lock server keeps track of locks globally, instead of per-repo. Definitely not ideal, and we have a separate issue tracking that: #1831.

Copy link
Contributor

@technoweenie technoweenie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's interesting that this requires git config --global http.$LFS_CLIENT_CERT_URL/.sslVerify "false". Since it only affects $LFS_CLIENT_CERT_URL, I suppose it's alright. I'm assuming that the config value is disabling server SSL verification in git/git-lfs, but still sending the client cert to $LFS_CLIENT_CERT_URL. Tests seem to fail if I take out the sslKey or sslCert config values.

I'd really like to get rid of the sslVerify setting, but I'm confident that it only affects the 2 tests added in this PR.

Copy link
Contributor

@ttaylorr ttaylorr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'd really like to get rid of the sslVerify setting, but I'm confident that it only affects the 2 tests added in this PR.

I agree. I think this looks good to be merged into master, thanks again for your contribution @schuCriX. 👍

Subject: pkix.Name{Organization: []string{"Yhat, Inc."}},
SignatureAlgorithm: x509.SHA256WithRSA,
NotBefore: time.Now(),
NotAfter: time.Now().Add(time.Hour), // valid for an hour
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could cause issues with DST transitions, but I'm not worried about it since the likelihood of it happening is so low.

@drCriX
Copy link
Contributor Author

drCriX commented Jan 25, 2017

the go http test server uses a hardcoded certificate for https. it is defined in the internal module. the certificate is issued to an empty server name. the newest git client on travis ci test fails on validating this empty name against 127.0.0.1.

so we have several ways to handle this:

  • comment out travis tests.
  • set sslverify to false
  • maybe setting servers cacert. not checked yet
  • running the testserver with an own certificate.

thanks....

@technoweenie
Copy link
Contributor

technoweenie commented Jan 26, 2017

@schuCriX: I've merged this into master. The sslVerify stuff can be looked at in another PR. I'm not too concerned about it.

EDIT: See #1893 for the actual merge.

Thanks again for your work on this!

chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Nov 5, 2024
In commit 9a00eb3 of PR git-lfs#1893 we
altered our "clone ClientCert" test of the Git LFS client's support
for TLS/SSL client certificates to deal with an exception raised by
the version of libcurl provided by macOS at the time.

As was reported in curl/curl#1105, the libcurl installed as part of
macOS 10.12 ("Sierra") failed to find some TLS/SSL client certificates
and instead raised an NSInvalidArgumentException.  See the notes in
our PR git-lfs#1869 for more details:

  git-lfs#1869 (comment)

To work around this issue, we skip the "clone ClientCert" test if any
NSInvalidArgumentException messages are found in the output of the Git
clone operation performed by the clone_repo_clientcert() function
of our t/testhelpers.sh shell library.

The same workaround was then copied into the "clone ClientCert with
homedir certs" test when it was added to our t/t-clone.sh test script
in commit c54c9da of PR git-lfs#5657.

The underlying problem in libcurl was resolved by the changes in commit
curl/curl@7c9b9ad, which was included
with libcurl version 7.52.0 and so is available in all contemporary
versions of macOS.  For instance, by the time of macOS 10.14 ("Mojave"),
the version of libcurl packaged with the OS was v7.54.0.

We can therefore drop our workarounds, as they are no longer applicable
to any supported versions of macOS.

(Note, too, that the GitHub Actions runners we now use for our CI jobs
provide a libcurl version installed by Homebrew, which takes precedence
over the one supplied by macOS.)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants