git lfs push/upload of big files fails with 401 #2753
Description
On a relatively slow connection, a bigger than usual (900 MB) files fails uploading with a 401.
This must be connected to a timeout issue but I have maxed up all settings I could found (proxy_connect_timeout, proxy_send_timeout, proxy_read_timeout, ...)
Not sure if this is a bug or a miss-configuration of mine, but I could really not found any solution to this problem until now...
git-lfs 2.3.4, git 2.15.0
GIT_TRACE=1 git push
18:50:51.794401 git.c:344 trace: built-in: git 'push'
18:50:51.795882 run-command.c:626 trace: run_command: 'ssh' 'git@lab.xyz.com' 'git-receive-pack '\''Projects/PROJECTS-171123_Project-11-2017.git'\'''
18:50:52.620609 run-command.c:626 trace: run_command: '.git/hooks/pre-push' 'origin' 'git@lab.xyz.com:Projects/PROJECTS-171123_Project-11-2017.git'
18:50:52.629081 git.c:576 trace: exec: 'git-lfs' 'pre-push' 'origin' 'git@lab.xyz.com:Projects/PROJECTS-171123_Project-11-2017.git'
18:50:52.629600 run-command.c:626 trace: run_command: 'git-lfs' 'pre-push' 'origin' 'git@lab.xyz.com:Projects/PROJECTS-171123_Project-11-2017.git'
18:50:52.643607 trace git-lfs: run_command: 'git' version
18:50:52.654563 trace git-lfs: run_command: 'git' config -l
18:50:52.669297 trace git-lfs: tq: running as batched queue, batch size of 100
18:50:53.308294 trace git-lfs: pre-push: refs/heads/master 0722ce02f9949938e19c993a3adee266e23aa60c refs/heads/master 76596a51ebb6598bdb2e234da3bd9cffe19da889
18:50:53.308537 trace git-lfs: run_command: git rev-list --objects 0722ce02f9949938e19c993a3adee266e23aa60c --not --remotes=origin --
18:50:53.314983 trace git-lfs: run_command: git cat-file --batch
18:50:53.335141 trace git-lfs: tq: sending batch of size 24
18:50:53.335314 trace git-lfs: run_command: ssh -- git@lab.xyz.com git-lfs-authenticate Projects/PROJECTS-171123_Project-11-2017.git upload
18:50:53.929541 trace git-lfs: api: batch 24 files
18:50:54.007070 trace git-lfs: HTTP: POST https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/info/lfs/objects/batch
18:50:54.451064 trace git-lfs: HTTP: 200
18:50:54.459207 trace git-lfs: HTTP: {"objects":[{"oid":"1903395cebaed6aa6de7201a008d70ca403ee0f49938af14ad3aa55e688622ee","size":880302878,"actions":{"upload":{"href":"https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/1903395cebaed6aa6de7201a008d70ca403ee0f49938af14ad3aa55e688622ee/880302878","header":{"Authorization":"Basic UjNNMVg6aVJUQVV3dWI3dTFFeHlhOUNGWDN6clpUWTcyUVM1LWUzZmhyLXZEQ3F6c3o0elFSQkE="}}}},{"oid":"2b92e459706e3cfb3fdee0fdb68e10dae7560188eb60a3f2f3b16f4c663f
18:50:54.459241 trace git-lfs: HTTP: 1edc","size":835723004,"actions":{"upload":{"href":"https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/2b92e459706e3cfb3fdee0fdb68e10dae7560188eb60a3f2f3b16f4c663f1edc/835723004","header":{"Authorization":"Basic UjNNMVg6aVJUQVV3dWI3dTFFeHlhOUNGWDN6clpUWTcyUVM1LWUzZmhyLXZEQ3F6c3o0elFSQkE="}}}},{"oid":"a7440184845fa845450ac6a9f6a2f8e5b7817fd4861dea566b9621d12993d69a","size":826701523,"actions":{"upload":{"href":"https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/a7440184845fa845450ac6a9f6a2f8e5b7817fd4861dea566b9621d12993d69a/826701523","header":{"Authorization":"Basic UjNNMVg6aVJUQVV3dWI3dTFFeHlhOUNGWDN6clpUWTcyUVM1LWUzZmhyLXZEQ3F6c3o0elFSQkE="}}}},{"oid":"2e934b568e1db69a3b94a6ec9b7e49121265f58d0345b83a7b8ec10fd5ea69aa","size":784058132,"actions":{"upload":{"href":"https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab
18:50:54.459285 trace git-lfs: HTTP: -lfs/objects/2e934b568e1db69a3b94a6ec9b7e49121265f58d0345b83a7b8ec10fd5ea69aa/784058132","header":{"Authorization":"Basic UjNNMVg6aVJUQVV3dWI3dTFFeHlhOUNGWDN6clpUWTcyUVM1LWUzZmhyLXZEQ3F6c3o0elFSQkE="}}}},{"oid":"7dd24dd68f210641dd49de4526497e59a93c3f6b40590018fe19bc7a23647d08","size":342760648,"actions":{"upload":{"href":"https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/7dd24dd68f210641dd49de4526497e59a93c3f6b40590018fe19bc7a23647d08/342760648","header":{"Authorization":"Basic UjNNMVg6aVJUQVV3dWI3dTFFeHlhOUNGWDN6clpUWTcyUVM1LWUzZmhyLXZEQ3F6c3o0elFSQkE="}}}},{"oid":"603a20b89498d4a50bf615977738157c9c74b9bd8e1fea9dc43ffbf4e198f150","size":339552899,"actions":{"upload":{"href":"https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/603a20b89498d4a50bf615977738157c9c74b9bd8e1fea9dc43ffbf4e198f150/339552899","header":{"Authorization":"Basic UjNNMVg6aVJUQVV3dWI3dTFFeHlhOUNGWDN6clpUWTcyUVM1LWUzZmhyLXZEQ3F6c3o0elFSQkE="}}}},{"oid":"2d0bb82ed201bbebb6e93c847cb2c67353504c47a2eeaf6dac5ace6c45380b9b","size":209959087},{"oid":"defcfe25dc8ccb1ba746c66639935f6600667f47a4f203207b8f026ccf163707","size":58605283},{"oid":"373274a0ffadbf5845dfec1e53bf95dc1dcd71c6e26572155dab2f58b47e244b","size":42635522},{"oid":"425218a3af4b921b45bfce0b12ef17fb2e488640a6970048be92d76420cd76e6","size":42108092},{"oid":"3b175111d3a345ac0036dbfc2f50168355fabfa5757c444f1741906010ceb8d9","size":39706905},{"oid":"c20c455c1a44c471269b63725cf67ca2cd510508ec5acb6758dd6d1abadd3397","size":35960501},{"oid":"1f07b228b00f3d52ef770e89f16bd46b559aff6b9b2d1b35b6078225ca555c7a","size":35202506},{"oid":"c2ec1abe8a2956570bca987f76b1928f74936722d9c8b961428ef233cce6cd30","size":10563584},{"oid":"c8cc4caf2db05e1db3aeb637b18ece3acadd7c0d8e235cee37191c06211599c0","size":9789440},{"oid":"9d9e350b92d86cbb1b0f5e1441699b66c283b4a7d7bb523fa781e90ed0212d19","size":9773056},{"oid":"59493fc4b0bb8473cde91ec62ffa1ed711985db4a44b134f932fb5e81469147
18:50:54.459473 trace git-lfs: HTTP: 9","size":7303168},{"oid":"4a7bef4fb65325e14b810947a507ac9562bbb1d5961f36f0e804c1a74268e09d","size":5261027},{"oid":"3090631cc0abbe7a69b64b14800fde283961aa3c3a4197feb55a7644a03fa2e7","size":2841808},{"oid":"e7b0368739f8a2d9fd669b61911fff0d824ac2054cd82bdb0efec7bbb532e710","size":2402837},{"oid":"fc7a6be2f5ed48f7343a6acf472aec705a9615ad4b61e1bf3b182aacd2df0947","size":1197364},{"oid":"67f560583610cf026707fd619966ff4f24afdfed377b27b1fa31e16561835dad","size":1194240},{"oid":"6c2bc1184ad89a5709e2a82f15618d0ee0fef6fb2f1e9bc8bdb801a467c07a75","size":1100336},{"oid":"e210350cf1a91671ec866ee3bce3c4e871b263b6367955d8da660b3f184eb95e","size":945412}]}
18:50:54.459702 trace git-lfs: tq: starting transfer adapter "basic"
Git LFS: (0 of 12 files, 18 skipped) 0 B / 3.73 GB, 492.62 MB skipped 18:50:54.460033 trace git-lfs: HTTP: PUT https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/1903395cebaed6aa6de7201a008d70ca403ee0f49938af14ad3aa55e688622ee/880302878
Git LFS: (0 of 6 files, 18 skipped) 839.52 MB / 3.73 GB, 492.62 MB skipped 19:34:29.238322 trace git-lfs: HTTP: 401
19:34:29.238501 trace git-lfs: tq: retrying object 1903395cebaed6aa6de7201a008d70ca403ee0f49938af14ad3aa55e688622ee: Authentication required: Authorization error: https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/1903395cebaed6aa6de7201a008d70ca403ee0f49938af14ad3aa55e688622ee/880302878
Check that you have proper access to the repository
19:34:29.238556 trace git-lfs: tq: enqueue retry #1 for "1903395cebaed6aa6de7201a008d70ca403ee0f49938af14ad3aa55e688622ee" (size: 880302878)
19:34:29.238709 trace git-lfs: HTTP: PUT https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/2b92e459706e3cfb3fdee0fdb68e10dae7560188eb60a3f2f3b16f4c663f1edc/835723004
Git LFS: (0 of 6 files, 18 skipped) 131.55 KB / 3.73 GB, 492.62 MB skipped 19:34:29.765428 trace git-lfs: HTTP: 401
19:34:29.766124 trace git-lfs: tq: retrying object 2b92e459706e3cfb3fdee0fdb68e10dae7560188eb60a3f2f3b16f4c663f1edc: Authentication required: Authorization error: https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/2b92e459706e3cfb3fdee0fdb68e10dae7560188eb60a3f2f3b16f4c663f1edc/835723004
Check that you have proper access to the repository
19:34:29.766163 trace git-lfs: tq: enqueue retry #1 for "2b92e459706e3cfb3fdee0fdb68e10dae7560188eb60a3f2f3b16f4c663f1edc" (size: 835723004)
19:34:29.766298 trace git-lfs: HTTP: PUT https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/a7440184845fa845450ac6a9f6a2f8e5b7817fd4861dea566b9621d12993d69a/826701523
Git LFS: (0 of 6 files, 18 skipped) 131.55 KB / 3.73 GB, 492.62 MB skipped 19:34:30.550894 trace git-lfs: HTTP: 401
19:34:30.551042 trace git-lfs: tq: retrying object a7440184845fa845450ac6a9f6a2f8e5b7817fd4861dea566b9621d12993d69a: Authentication required: Authorization error: https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/a7440184845fa845450ac6a9f6a2f8e5b7817fd4861dea566b9621d12993d69a/826701523
Check that you have proper access to the repository
19:34:30.551075 trace git-lfs: tq: enqueue retry #1 for "a7440184845fa845450ac6a9f6a2f8e5b7817fd4861dea566b9621d12993d69a" (size: 826701523)
19:34:30.551297 trace git-lfs: HTTP: PUT https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/2e934b568e1db69a3b94a6ec9b7e49121265f58d0345b83a7b8ec10fd5ea69aa/784058132
Git LFS: (0 of 6 files, 18 skipped) 131.55 KB / 3.73 GB, 492.62 MB skipped 19:34:31.354836 trace git-lfs: HTTP: 401
19:34:31.355173 trace git-lfs: tq: retrying object 2e934b568e1db69a3b94a6ec9b7e49121265f58d0345b83a7b8ec10fd5ea69aa: Authentication required: Authorization error: https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/2e934b568e1db69a3b94a6ec9b7e49121265f58d0345b83a7b8ec10fd5ea69aa/784058132
Check that you have proper access to the repository
19:34:31.355250 trace git-lfs: tq: enqueue retry #1 for "2e934b568e1db69a3b94a6ec9b7e49121265f58d0345b83a7b8ec10fd5ea69aa" (size: 784058132)
19:34:31.355431 trace git-lfs: HTTP: PUT https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/7dd24dd68f210641dd49de4526497e59a93c3f6b40590018fe19bc7a23647d08/342760648
Git LFS: (0 of 6 files, 18 skipped) 0 B / 3.73 GB, 492.62 MB skipped 19:34:31.569304 trace git-lfs: HTTP: 401
19:34:31.569483 trace git-lfs: tq: retrying object 7dd24dd68f210641dd49de4526497e59a93c3f6b40590018fe19bc7a23647d08: Authentication required: Authorization error: https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/7dd24dd68f210641dd49de4526497e59a93c3f6b40590018fe19bc7a23647d08/342760648
Check that you have proper access to the repository
19:34:31.569562 trace git-lfs: tq: enqueue retry #1 for "7dd24dd68f210641dd49de4526497e59a93c3f6b40590018fe19bc7a23647d08" (size: 342760648)
19:34:31.569832 trace git-lfs: HTTP: PUT https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/603a20b89498d4a50bf615977738157c9c74b9bd8e1fea9dc43ffbf4e198f150/339552899
Git LFS: (0 of 6 files, 18 skipped) 131.55 KB / 3.73 GB, 492.62 MB skipped 19:34:32.095027 trace git-lfs: HTTP: 401
19:34:32.095219 trace git-lfs: tq: retrying object 603a20b89498d4a50bf615977738157c9c74b9bd8e1fea9dc43ffbf4e198f150: Authentication required: Authorization error: https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/gitlab-lfs/objects/603a20b89498d4a50bf615977738157c9c74b9bd8e1fea9dc43ffbf4e198f150/339552899
Check that you have proper access to the repository
19:34:32.095281 trace git-lfs: tq: enqueue retry #1 for "603a20b89498d4a50bf615977738157c9c74b9bd8e1fea9dc43ffbf4e198f150" (size: 339552899)
19:34:32.095362 trace git-lfs: tq: sending batch of size 6
19:34:32.095636 trace git-lfs: ssh cache: git@lab.xyz.com git-lfs-authenticate Projects/PROJECTS-171123_Project-11-2017.git upload
19:34:32.095760 trace git-lfs: api: batch 6 files
19:34:32.095968 trace git-lfs: HTTP: POST https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/info/lfs/objects/batch
19:34:32.805648 trace git-lfs: HTTP: 401
19:34:32.805738 trace git-lfs: api error: Authentication required: Authorization error: https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/info/lfs/objects/batch
Check that you have proper access to the repository
Git LFS: (0 of 6 files, 18 skipped) 0 B / 3.73 GB, 492.62 MB skipped
batch response: Authentication required: Authorization error: https://lab.xyz.com:7372/Projects/PROJECTS-171123_Project-11-2017.git/info/lfs/objects/batch
Check that you have proper access to the repository
error: failed to push some refs to 'git@lab.xyz.com:Projects/PROJECTS-171123_Project-11-2017.git'