Skip to content

git push <tag> performance slowdown due to scanning the entire commit history for LFS objects #3976

@dghowlett

Description

@dghowlett

Describe the bug
A git push of a tag ref for a git repos with a large history is extremely slow. Runtimes up to 60 minutes are seen for larger histories vs what should be a couple seconds. With GIT_TRACE=1 on, the runtime points to the git lfs pre-push call consuming the cycles. From diving into the lfs code and adding more debug, the push of a tag is resulting in a git rev-list of the full history relative to the SHA id the tag points to. For repos with large histories, this can result in LFS upload validation of 1000's of object ids. Have verified same behavior with 2.5.1, 2.9.0, 2.9.2

To Reproduce
Take any repo with some history spanning LFS objects. Push a git tag with GIT_TRACE on and you'll see it's attempting to iterate over all LFS objects found in the full history. I would think for a tag push, it should be a no-op with no attempt to push Git LFS objects needed.

git-lfs version
git-lfs/2.9.2 (unknown; linux amd64; go 1.13.5; git 0274d85)

time git push origin testing_git_commit/15
Uploading LFS objects: 100% (8237/8237), 222 GB | 0 B/s, done
Total 0 (delta 0), reused 0 (delta 0)0)

  • [new tag] testing_git_commit/15 -> testing_git_commit/15
    1155.190u 339.693s 24:42.09 100.8% 0+0k 128+434407640io 1pf+0w

Above it took 25 minutes to push the tag.

With GIT_TRACE=1

git-lfs version
git-lfs/2.9.2 (unknown; linux amd64; go 1.13.5; git 0274d85)

....
09:55:38.921966 trace git-lfs: exec: git 'version'
09:55:38.927648 trace git-lfs: exec: git '-c' 'filter.lfs.smudge=' '-c' 'filter.lfs.clean=' '-c' 'filter.lfs.process=' '-c' 'filter.lfs.required=false' 'rev-parse' 'HEAD' '--symbolic-full-name' 'HEAD'
09:55:38.932679 trace git-lfs: exec: git 'config' '-l'
09:55:38.942267 trace git-lfs: pre-push: refs/tags/testing_git_commit/14 ff7fe17b184a0bcfa3b383db456bd2a30bd37b5d refs/tags/testing_git_commit/14 0000000000000000000000000000000000000000
09:55:39.195240 trace git-lfs: tq: running as batched queue, batch size of 100
09:55:39.195443 trace git-lfs: run_command: git rev-list --stdin --objects --not --remotes=origin

09:55:55.927830 trace git-lfs: tq: sending batch of size 100
09:55:55.928307 trace git-lfs: api: batch 100 files
...
09:56:02.059531 trace git-lfs: HTTP: {
"objects" : [ {
"oid" : "aad5173f613a3ed1f169d80ab3eb2d073eaca2841b42a3e2f656a068382d4b2d",
"size" : 28275405
}, {
"oid" : "9b9bfc3a3f85b22a1b5e28112c5fe028b53c5cc69be97cdf969c6df03b5d5197",
"size" : 28275404
}, {
"oid" : "f777603a0ac03decbced862154ca8071ef0b41eef81c16dd275043940351f73e",
"size" : 28275404
}, {
"oid" : "2b1c49a167430ccd56332064e2462ce4c5c094a3e5a62d94bc8bf5d68ea4199b",
"size" : 28275404
.... for 8000+ OIDS.

Looking at the rev-list call the lfs pre-push makes, the above example would point to an estimate of scanning 8242 OIDS which contain LFS pointers:

% git rev-list ff7fe17b --objects | git cat-file --batch-check='%(objectname) %(objecttype) 8242 24726 513882

If I use --no-verify to bypass the pre-push hook, the tag push completes within a couple seconds.

Expected behavior
No attempt to push Git LFS objects should be needed for tag refs that I can think of. I'd expect the same behavior as it currently works for branch deletions where no attempts to push Git LFS objects will be made.

Looking at the LFS code, it seems command_pre_push.go needs to account for tag ref types and the remote SHA value being '0000000000000000000000000000000000000000'.
Under those conditions it should continue and do nothing.
I've added some code in my clone of the source to that effect and it resolves the performance issue.

System environment
RHEL7

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions