Skip to content

Memory bloat when repository has big .gitignore/.gitattributes (thousands of entries) #306

@Sebazzz

Description

@Sebazzz

Mentioned in issue #66:

Every revision seems to get a little bit slower (as if there is some kind of O(n²) problem here). I've tried it with -Xmx4g and a repo with 28000 commits - I believe the repository comparable to dotnet/coreclr. The checked out full repository size just below 1GB to give an idea of the size.

The startup process is slow for such repository, and every revision that is processed (9538 in my case) seems to be processed a little bit slower. Eventually you get almost 100% CPU usage (VM with 4 available virtual processors). This was around revision 5500 of 9538. I attempted to restart the process, after restarting it started over.

I took a short look at the code, and the computed information seems to be cached but I don't think the cached is flushed so if the startup process is interrupted (or would crash) the revisions processed so far are gone.

Running on:

  • Windows Server 2016
  • 4x cpu core
  • 16GB internal memory
  • jdk1.8.0_202

Thread dump of current process (revision 2331 of 9538): git-as-svn-threaddump-rev-2331of9538.txt. I took a couple of thread dumps and every time the stack up from and including collectRename appears to be same.

I will let it run the night (it is 22:33 here) and see in the morning how far it went.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions