Skip to content

Replace localstorage#2689

Merged
technoweenie merged 18 commits into
masterfrom
replace-localstorage
Oct 26, 2017
Merged

Replace localstorage#2689
technoweenie merged 18 commits into
masterfrom
replace-localstorage

Conversation

@technoweenie

Copy link
Copy Markdown
Contributor

Replaces the localstorage with a new, slightly better designed fs package. More of the static vars/funcs have been removed too.

@technoweenie technoweenie merged commit 9c3733e into master Oct 26, 2017
@technoweenie technoweenie deleted the replace-localstorage branch December 7, 2017 19:51
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Mar 16, 2026
In commit 13a8af6 of PR git-lfs#1616 we
added a FastWalkGitRepo() function to our "tools" package for the
purpose of improving the performance of the "git lfs track" command.
At the time, this command used the Walk() function of the "filepath"
package from the Go standard library to traverse the contents of the
current Git working tree and locate all ".gitattributes" files.

Then in commit f1fdc85 of the same
PR git-lfs#1616 we updated the "git lfs track" command's findAttributeFiles()
function to use our new FastWalkGitRepo() function instead of the
"filepath" package's Walk() function.  This change made searches for
".gitattributes" files in large repositories faster for several reasons.
Unlike the Walk() function from the "filepath" package, the functions
called by the FastWalkGitRepo() function to traverse a directory
hierarchy did not sort the entries in each directory, and also ignored
all ".git" directories and all entries which matched any pattern found
in a ".gitignore" file.

Later, in PRs git-lfs#1870 and git-lfs#2689, we expanded the number of callers of
the FastWalkGitRepo() function.  In particular, in PR git-lfs#2689 we began
to make use of the function during the final phase of all Git LFS
commands where we find and delete any stale temporary files stored in
our ".lfs/tmp" directory.  This PR introduced our "fs" package whose
cleanupTmp() method calls the FastWalkGitRepo() function, passing
the path to the ".lfs/tmp" directory and an anonymous callback function
which removes any temporary object data files that are more than an
hour old.

We then added a FastWalkGitRepoAll() function to our "fs" package in
PR git-lfs#3190, which operated in a similar fashion as the FastWalkGitRepo()
function but did not read ".gitignore" files and so also did not skip
directory entries matching any patterns found in those files.

Next, in PR git-lfs#3686, we updated the internal implementation of the
FastWalkGitRepo() and FastWalkGitRepoAll() functions to avoid entering
submodules when traversing a Git working tree.  To make this change,
we added a check to the Walk() method of the "fastWalker" structure
so it would return immediately when processing a directory if the
directory contained an entry named ".git", unless the directory was
the root of the working tree.  Note that the Walk() method already
ignored any directory entries with the name ".git", but this only
meant it would traverse through the contents of a submodule checkout
in a working tree while skipping the submodule's ".git" directory.

Then in commit 83d7f76 of PR git-lfs#3823
we first implemented the NewLsFiles() function in our "git" package,
which executes a "git ls-files" command and returns list of files it
outputs.  As well, we updated the findAttributeFiles() function of our
"git" package and the fixFileWriteFlags() function in our "locking"
package to both call the NewLsFiles() function instead of calling
either the FastWalkGitRepo() or FastWalkGitRepoAll() functions.

Since these were the only instances where we actually used the
FastWalkGitRepo() or FastWalkGitRepoAll() functions to traverse a Git
working tree, in the same commit of PR git-lfs#3823 we removed the
FastWalkGitRepo() function and renamed the FastWalkGitRepoAll() function
to FastWalkDir().  We also simplified some of the internal functions
called by the FastWalkDir() function because they no longer needed to
detect or parse ".gitignore" files, or skip directory entries matching
the patterns from those files.

Although the two remaining use cases for the FastWalkDir() function
also did not need the function to detect and skip submodules or
directories named ".git", the logic to do so was retained in the
internal functions of the "tools" package.

Specifically, the fastWalkWithExcludeFiles() function, which is called
by the FastWalkDir() function, establishes two file path filters with
the patterns ".git" and "**/.git", which are then passed down to the
Walk() method of the "fastWalker" structure.  That method checks whether
the current directory entry's name matches either of the filter patterns,
and if it does returns immediately without processing the entry any
further.  In addition, the Walk() method still also performs the check
added in PR git-lfs#3686 to try to avoid traversing into Git submodules.

As noted above, however, neither of the two remaining callers of the
FastWalkDir() function require these checks, because they only need
the function to traverse directory hierarchies within the ".git/lfs"
directory.

The cleanupTmp() method of the Filesystem structure in our "fs" package
uses the FastWalkDir() function to find stale temporary files within the
".git/lfs/tmp" directory.  The EachObject() method of the same structure,
meanwhile, uses the FastWalkDir() function to invoke a callback function
for each object file under the ".git/lfs/objects" directory.

In both cases, the directory hierarchy traversed by the FastWalkDir()
function is entirely within the ".git/lfs" directory, so there is no
value in trying to exclude ".git" directories while performing the
traversal.

We therefore now simplify the FastWalkDir() function and the internal
functions it invokes by removing the unnecessary checks for submodules
and for directory entries named ".git".

First, we update the fastWalkWithExcludeFiles() function so that does
not initialize any file path filters, and we rename the function to
fastWalkDir().

Next, we remove the Walk() method's "excludePaths" parameter, and alter
the method so that it no longer skips directory entries based on whether
their names match the patterns in that parameter's file path filters.

We also eliminate the check in the Walk() method for directories
containing ".git" directory entries, as this check's only purpose was
to skip Git submodules within a working tree.

As well, we revise the code comments relating to all of these functions
and methods to reflect their new names and their simplified behaviours.
To minimize the changes in this commit, however, we leave the names of
the functions' parameters and internal variables intact, even though
some of them still reflect the original design and the expectation that
the functions would be used with Git working trees.  In a subsequent
commit in this PR we will then rename these variables and parameters,
along with the "rootDir" field of the "fastWalker" structure, so that
they more accurately represent the functions' current purpose and
implementation.

Finally, note that the "cleans only temp files and directories older
than an hour" test in our "t/t-tempfile.sh" shell test script verifies
the behaviour of cleanupTmp() function, which employs the FastWalkDir()
function, while the TestFastWalkBasic() test function in our Go test
suite directly exercises the fastWalkDir() internal function.  Both
tests thus provide some assurance that our changes in this commit have
not introduced any unexpected regressions.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Mar 16, 2026
In commit 13a8af6 of PR git-lfs#1616 we
added a FastWalkGitRepo() function to our "tools" package for the
purpose of improving the performance of the "git lfs track" command.
At the time, this command used the Walk() function of the "filepath"
package from the Go standard library to traverse the contents of the
current Git working tree and locate all ".gitattributes" files.

Then in commit f1fdc85 of the same
PR git-lfs#1616 we updated the "git lfs track" command's findAttributeFiles()
function to use our new FastWalkGitRepo() function instead of the
"filepath" package's Walk() function.  This change made searches for
".gitattributes" files in large repositories faster for several reasons.
Unlike the Walk() function from the "filepath" package, the functions
called by the FastWalkGitRepo() function to traverse a directory
hierarchy did not sort the entries in each directory, and also ignored
all ".git" directories and all entries which matched any pattern found
in a ".gitignore" file.

Later, in PRs git-lfs#1870 and git-lfs#2689, we expanded the number of callers of
the FastWalkGitRepo() function.  In particular, in PR git-lfs#2689 we began
to make use of the function during the final phase of all Git LFS
commands where we find and delete any stale temporary files stored in
our ".lfs/tmp" directory.  This PR introduced our "fs" package whose
cleanupTmp() method calls the FastWalkGitRepo() function, passing
the path to the ".lfs/tmp" directory and an anonymous callback function
which removes any temporary object data files that are more than an
hour old.

We then added a FastWalkGitRepoAll() function to our "fs" package in
PR git-lfs#3190, which operated in a similar fashion as the FastWalkGitRepo()
function but did not read ".gitignore" files and so also did not skip
directory entries matching any patterns found in those files.

Next, in PR git-lfs#3686, we updated the internal implementation of the
FastWalkGitRepo() and FastWalkGitRepoAll() functions to avoid entering
submodules when traversing a Git working tree.  To make this change,
we added a check to the Walk() method of the "fastWalker" structure
so it would return immediately when processing a directory if the
directory contained an entry named ".git", unless the directory was
the root of the working tree.  Note that the Walk() method already
ignored any directory entries with the name ".git", but this only
meant it would traverse through the contents of a submodule checkout
in a working tree while skipping the submodule's ".git" directory.

Then in commit 83d7f76 of PR git-lfs#3823
we first implemented the NewLsFiles() function in our "git" package,
which executes a "git ls-files" command and returns list of files it
outputs.  As well, we updated the findAttributeFiles() function of our
"git" package and the fixFileWriteFlags() function in our "locking"
package to both call the NewLsFiles() function instead of calling
either the FastWalkGitRepo() or FastWalkGitRepoAll() functions.

Since these were the only instances where we actually used the
FastWalkGitRepo() or FastWalkGitRepoAll() functions to traverse a Git
working tree, in the same commit of PR git-lfs#3823 we removed the
FastWalkGitRepo() function and renamed the FastWalkGitRepoAll() function
to FastWalkDir().  We also simplified some of the internal functions
called by the FastWalkDir() function because they no longer needed to
detect or parse ".gitignore" files, or skip directory entries matching
the patterns from those files.

Although the two remaining use cases for the FastWalkDir() function
also did not need the function to detect and skip submodules or
directories named ".git", the logic to do so was retained in the
internal functions of the "tools" package.

Specifically, the fastWalkWithExcludeFiles() function, which is called
by the FastWalkDir() function, establishes two file path filters with
the patterns ".git" and "**/.git", which are then passed down to the
Walk() method of the "fastWalker" structure.  That method checks whether
the current directory entry's name matches either of the filter patterns,
and if it does returns immediately without processing the entry any
further.  In addition, the Walk() method still also performs the check
added in PR git-lfs#3686 to try to avoid traversing into Git submodules.

As noted above, however, neither of the two remaining callers of the
FastWalkDir() function require these checks, because they only need
the function to traverse directory hierarchies within the ".git/lfs"
directory.

The cleanupTmp() method of the Filesystem structure in our "fs" package
uses the FastWalkDir() function to find stale temporary files within the
".git/lfs/tmp" directory.  The EachObject() method of the same structure,
meanwhile, uses the FastWalkDir() function to invoke a callback function
for each object file under the ".git/lfs/objects" directory.

In both cases, the directory hierarchy traversed by the FastWalkDir()
function is entirely within the ".git/lfs" directory, so there is no
value in trying to exclude ".git" directories while performing the
traversal.

We therefore now simplify the FastWalkDir() function and the internal
functions it invokes by removing the unnecessary checks for submodules
and for directory entries named ".git".

First, we update the fastWalkWithExcludeFiles() function so that does
not initialize any file path filters, and we rename the function to
fastWalkDir().

Next, we remove the Walk() method's "excludePaths" parameter, and alter
the method so that it no longer skips directory entries based on whether
their names match the patterns in that parameter's file path filters.

We also eliminate the check in the Walk() method for directories
containing ".git" directory entries, as this check's only purpose was
to skip Git submodules within a working tree.

As well, we revise the code comments relating to all of these functions
and methods to reflect their new names and their simplified behaviours.
To minimize the changes in this commit, however, we leave the names of
the functions' parameters and internal variables intact, even though
some of them still reflect the original design and the expectation that
the functions would be used with Git working trees.  In a subsequent
commit in this PR we will then rename these variables and parameters,
along with the "rootDir" field of the "fastWalker" structure, so that
they more accurately represent the functions' current purpose and
implementation.

Finally, note that the "cleans only temp files and directories older
than an hour" test in our "t/t-tempfile.sh" shell test script verifies
the behaviour of cleanupTmp() function, which employs the FastWalkDir()
function, while the TestFastWalkBasic() test function in our Go test
suite directly exercises the fastWalkDir() internal function.  Both
tests thus provide some assurance that our changes in this commit have
not introduced any unexpected regressions.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Mar 17, 2026
In commit 13a8af6 of PR git-lfs#1616 we
added a FastWalkGitRepo() function to our "tools" package for the
purpose of improving the performance of the "git lfs track" command.
At the time, this command used the Walk() function of the "filepath"
package from the Go standard library to traverse the contents of the
current Git working tree and locate all ".gitattributes" files.

Then in commit f1fdc85 of the same
PR git-lfs#1616 we updated the "git lfs track" command's findAttributeFiles()
function to use our new FastWalkGitRepo() function instead of the
"filepath" package's Walk() function.  This change made searches for
".gitattributes" files in large repositories faster for several reasons.
Unlike the Walk() function from the "filepath" package, the functions
called by the FastWalkGitRepo() function to traverse a directory
hierarchy did not sort the entries in each directory, and also ignored
all ".git" directories and all entries which matched any pattern found
in a ".gitignore" file.

Later, in PRs git-lfs#1870 and git-lfs#2689, we expanded the number of callers of
the FastWalkGitRepo() function.  In particular, in PR git-lfs#2689 we began
to make use of the function during the final phase of all Git LFS
commands where we find and delete any stale temporary files stored in
our ".lfs/tmp" directory.  This PR introduced our "fs" package whose
cleanupTmp() method calls the FastWalkGitRepo() function, passing
the path to the ".lfs/tmp" directory and an anonymous callback function
which removes any temporary object data files that are more than an
hour old.

We then added a FastWalkGitRepoAll() function to our "fs" package in
PR git-lfs#3190, which operated in a similar fashion as the FastWalkGitRepo()
function but did not read ".gitignore" files and so also did not skip
directory entries matching any patterns found in those files.

Next, in PR git-lfs#3686, we updated the internal implementation of the
FastWalkGitRepo() and FastWalkGitRepoAll() functions to avoid entering
submodules when traversing a Git working tree.  To make this change,
we added a check to the Walk() method of the "fastWalker" structure
so it would return immediately when processing a directory if the
directory contained an entry named ".git", unless the directory was
the root of the working tree.  Note that the Walk() method already
ignored any directory entries with the name ".git", but this only
meant it would traverse through the contents of a submodule checkout
in a working tree while skipping the submodule's ".git" directory.

Then in commit 83d7f76 of PR git-lfs#3823
we first implemented the NewLsFiles() function in our "git" package,
which executes a "git ls-files" command and returns list of files it
outputs.  As well, we updated the findAttributeFiles() function of our
"git" package and the fixFileWriteFlags() function in our "locking"
package to both call the NewLsFiles() function instead of calling
either the FastWalkGitRepo() or FastWalkGitRepoAll() functions.

Since these were the only instances where we actually used the
FastWalkGitRepo() or FastWalkGitRepoAll() functions to traverse a Git
working tree, in the same commit of PR git-lfs#3823 we removed the
FastWalkGitRepo() function and renamed the FastWalkGitRepoAll() function
to FastWalkDir().  We also simplified some of the internal functions
called by the FastWalkDir() function because they no longer needed to
detect or parse ".gitignore" files, or skip directory entries matching
the patterns from those files.

Although the two remaining use cases for the FastWalkDir() function
also did not need the function to detect and skip submodules or
directories named ".git", the logic to do so was retained in the
internal functions of the "tools" package.

Specifically, the fastWalkWithExcludeFiles() function, which is called
by the FastWalkDir() function, establishes two file path filters with
the patterns ".git" and "**/.git", which are then passed down to the
Walk() method of the "fastWalker" structure.  That method checks whether
the current directory entry's name matches either of the filter patterns,
and if it does returns immediately without processing the entry any
further.  In addition, the Walk() method still also performs the check
added in PR git-lfs#3686 to try to avoid traversing into Git submodules.

As noted above, however, neither of the two remaining callers of the
FastWalkDir() function require these checks, because they only need
the function to traverse directory hierarchies within the ".git/lfs"
directory.

The cleanupTmp() method of the Filesystem structure in our "fs" package
uses the FastWalkDir() function to find stale temporary files within the
".git/lfs/tmp" directory.  The EachObject() method of the same structure,
meanwhile, uses the FastWalkDir() function to invoke a callback function
for each object file under the ".git/lfs/objects" directory.

In both cases, the directory hierarchy traversed by the FastWalkDir()
function is entirely within the ".git/lfs" directory, so there is no
value in trying to exclude ".git" directories while performing the
traversal.

We therefore now simplify the FastWalkDir() function and the internal
functions it invokes by removing the unnecessary checks for submodules
and for directory entries named ".git".

First, we update the fastWalkWithExcludeFiles() function so that does
not initialize any file path filters, and we rename the function to
fastWalkDir().

Next, we remove the Walk() method's "excludePaths" parameter, and alter
the method so that it no longer skips directory entries based on whether
their names match the patterns in that parameter's file path filters.

We also eliminate the check in the Walk() method for directories
containing ".git" directory entries, as this check's only purpose was
to skip Git submodules within a working tree.

As well, we revise the code comments relating to all of these functions
and methods to reflect their new names and their simplified behaviours.
To minimize the changes in this commit, however, we leave the names of
the functions' parameters and internal variables intact, even though
some of them still reflect the original design and the expectation that
the functions would be used with Git working trees.  In a subsequent
commit in this PR we will then rename these variables and parameters,
along with the "rootDir" field of the "fastWalker" structure, so that
they more accurately represent the functions' current purpose and
implementation.

Finally, note that the "cleans only temp files and directories older
than an hour" test in our "t/t-tempfile.sh" shell test script verifies
the behaviour of cleanupTmp() function, which employs the FastWalkDir()
function, while the TestFastWalkBasic() test function in our Go test
suite directly exercises the fastWalkDir() internal function.  Both
tests thus provide some assurance that our changes in this commit have
not introduced any unexpected regressions.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request Mar 17, 2026
In commit 13a8af6 of PR git-lfs#1616 we
added a FastWalkGitRepo() function to our "tools" package for the
purpose of improving the performance of the "git lfs track" command.
At the time, this command used the Walk() function of the "filepath"
package from the Go standard library to traverse the contents of the
current Git working tree and locate all ".gitattributes" files.

Then in commit f1fdc85 of the same
PR git-lfs#1616 we updated the "git lfs track" command's findAttributeFiles()
function to use our new FastWalkGitRepo() function instead of the
"filepath" package's Walk() function.  This change made searches for
".gitattributes" files in large repositories faster for several reasons.
Unlike the Walk() function from the "filepath" package, the functions
called by the FastWalkGitRepo() function to traverse a directory
hierarchy did not sort the entries in each directory, and also ignored
all ".git" directories and all entries which matched any pattern found
in a ".gitignore" file.

Later, in PRs git-lfs#1870 and git-lfs#2689, we expanded the number of callers of
the FastWalkGitRepo() function.  In particular, in PR git-lfs#2689 we began
to make use of the function in our "git lfs prune" command to locate
all the object files in our local storage directories, which by default
are located within the ".git/lfs/objects" directory.  As well, with
this PR we began to use the FastWalkGitRepo() function during the
final phase of all Git LFS commands where we find and delete any
stale temporary files from the directory where we store such files,
which by default is the ".git/lfs/tmp" directory.

PR git-lfs#2689 introduced our "fs" package, whose Filesystem structure's
EachObject() and cleanupTmp() methods both called the FastWalkGitRepo()
function, passing the paths to the local object storage directory
and the local temporary file storage directory, respectively.

We then added a FastWalkGitRepoAll() function to our "fs" package in
PR git-lfs#3190.  This function operated in a similar fashion as the
FastWalkGitRepo() function but did not read ".gitignore" files and so
also did not skip directory entries matching any patterns found in
those files.

Next, in PR git-lfs#3686, we updated the internal implementation of the
FastWalkGitRepo() and FastWalkGitRepoAll() functions to avoid entering
submodules when traversing a Git working tree.  To make this change,
we added a check to the "fastWalker" structure's Walk() method so it
would return immediately when processing a directory if the directory
contained an entry named ".git", unless the directory was the root of
the working tree.  Note that the Walk() method already ignored any
directory entries with the name ".git", but this only meant it would
traverse through the contents of a submodule checkout in a working
tree while skipping the submodule's ".git" directory.

Then in commit 83d7f76 of PR git-lfs#3823
we first implemented the NewLsFiles() function in our "git" package,
which executes a "git ls-files" command and returns the list of files
it outputs.  As well, we updated the findAttributeFiles() function of
our "git" package and the fixFileWriteFlags() function in our "locking"
package to both call the NewLsFiles() function instead of calling
either the FastWalkGitRepo() or FastWalkGitRepoAll() functions.

Since these were the only instances where we actually used the
FastWalkGitRepo() or FastWalkGitRepoAll() functions to traverse a Git
working tree, in the same commit of PR git-lfs#3823 we removed the
FastWalkGitRepo() function and renamed the FastWalkGitRepoAll() function
to FastWalkDir().  We also simplified some of the internal functions
called by the FastWalkDir() function because they no longer needed to
detect or parse ".gitignore" files, or skip directory entries matching
the patterns from those files.

Although the two remaining use cases for the FastWalkDir() function
also did not need the function to detect and skip submodules or
directories named ".git", the logic to do so was retained in the
internal functions of the "tools" package.

Specifically, the fastWalkWithExcludeFiles() function, which is called
by the FastWalkDir() function, establishes two file path filters with
the patterns ".git" and "**/.git", which are then passed down to the
Walk() method of the "fastWalker" structure.  That method checks whether
the current directory entry's name matches either of the filter patterns,
and if it does returns immediately without processing the entry any
further.  In addition, the Walk() method still also performs the check
added in PR git-lfs#3686 to try to avoid traversing into Git submodules.

As noted above, however, neither of the two remaining callers of the
FastWalkDir() function require these checks, because they only need
the function to traverse directory hierarchies which the Git LFS client
has created, and which are specifically not Git working trees.

The cleanupTmp() method of the Filesystem structure in our "fs" package
uses the FastWalkDir() function to find stale files within the local
temporary file storage directory.  The EachObject() method of the same
structure, meanwhile, uses the FastWalkDir() function to invoke a
callback function for each object file in the local object storage
directory.

By default, these local storage directories are the ".git/lfs/tmp" and
".git/lfs/objects" directories.  If the "lfs.storage" configuration
option is set to a relative path, then these directories will be
located somewhere within the ".git" directory, while if the option is
set to an absolute path, our local storage directories will be located
under that arbitrary location.

In all these cases the Git LFS client creates and manages these local
storage directories, so we can expect them not to contain ".git"
directories or submodules.  This is true even when a user has
configured the "lfs.storage" option with an absolute path, since
the client is still responsible for creating and managing "tmp" and
"objects" directories within that arbitrary location.

We therefore now simplify the FastWalkDir() function and the internal
functions it invokes by removing the unnecessary checks for submodules
and for directory entries named ".git".

First, we update the fastWalkWithExcludeFiles() function so that does
not initialize any file path filters, and we rename the function to
fastWalkDir().

Next, we remove the Walk() method's "excludePaths" parameter, and alter
the method so that it no longer skips directory entries based on whether
their names match the patterns in that parameter's file path filters.

We also eliminate the check in the Walk() method for directories
containing ".git" directory entries, as this check's only purpose was
to skip Git submodules within a working tree.

As well, we revise the code comments relating to all of these functions
and methods to reflect their new names and their simplified behaviours.
To minimize the changes in this commit, however, we leave the names of
the functions' parameters and internal variables intact, even though
some of them still reflect the original design and the expectation that
the functions would be used with Git working trees.  In a subsequent
commit in this PR we will then rename these variables and parameters,
along with the "rootDir" field of the "fastWalker" structure, so that
they more accurately represent the functions' current purpose and
implementation.

Finally, note that the "cleans only temp files and directories older
than an hour" test in our "t/t-tempfile.sh" shell test script verifies
the behaviour of cleanupTmp() function, which employs the FastWalkDir()
function, while the TestFastWalkBasic() test function in our Go test
suite directly exercises the fastWalkDir() internal function.  Both
tests thus provide some assurance that our changes in this commit have
not introduced any unexpected regressions.
chrisd8088 added a commit that referenced this pull request May 21, 2026
In PR #763 we revised the way the Git LFS client created and deleted
temporary files, and as part of this change we updated our top-level
main() function so that upon exit it invokes a function to delete any
stale temporary files and always does so exactly one time only.

Specifically, in commit a0704be we
updated the main() function to call the ClearTempObjects() function
in our "lfs" package using the Do() method of a Once structure from
the "sync" package in the Go standard library.  The main() function
invoked the ClearTempObjects() function via the Do() method in two
instances, when the main() function was exiting normally, and when
it received a signal from the operating system.  Then in commit
a3a2369 of the same PR we refined
the list of signals for which the client will be notified to exit
and restricted them to just the SIGINT and SIGKILL signals.

Later, in commit b9c5a10 of PR #1390
we created a Cleanup() function in our "commands" package and changed
the Do() methods in our main() function to call this new Cleanup()
function instead.  In turn, the Cleanup() function then invoked the
ClearTempObjects() function.

Finally, in commit e9121fd of
PR #2689 we replaced the ClearTempObjects() function with the
Cleanup() method of the Filesystem structure in a new "fs" package.
In the same commit we also added a Cleanup() method for the
Configuration structure in our "config" package, which just invokes
the Cleanup() method on the Configuration structure's "fs" field.
We then updated the Cleanup() function in the "commands" package to
call the Cleanup() method of the global "cfg" variable, whose type
is that of a Configuration structure from our "config" package.

These refactoring steps did not alter the basic design from PR #763,
though, in which stale temporary files are removed when the Git LFS
client either receives a SIGINT and SIGKILL signal or otherwise
exits normally.

Note that these conditions include the case where the main() function
will return a non-zero exit code of 127 because the Run() function in
our "commands" package has returned this value.  However, this case
occurs only when a user provides an unknown Git LFS command name or
invalid set of command-line flag options and arguments.  When that
occurs, the Execute() method of the Command structure from the
"github.com/spf13/cobra" package returns a non-nil "error" value to
our Run() function, which then returns the value 127 to the main()
function.

In other instances when the Git LFS client encounters an unrecoverable
error condition, though, our command implementation functions call
one of the exit wrapper functions in our "commands" package, such as
Exit() or ExitWithCode().  In a recent set of changes in PR #6255 we
ensured that one of these wrapper functions is always used when the
client needs to halt immediately.

However, these wrapper functions do not consistently perform the
cleanup steps that are otherwise executed by the Run() and main()
functions when the client exits normally.  In subsequent commits in
this PR we expect to revise our exit wrapper functions so that when
the client halts it always performs the same set of cleanup actions.

As an initial step toward this goal, we begin by moving the Once
structure from the "main" package to our "commands" package.  We
also refactor the Cleanup() function in the "commands" package so
that it just invokes a new doCleanup() function using the Do()
method of the Once structure.

The doCleanup() function then performs the actual removal of any
stale temporary files by calling the Cleanup() method of the global
"cfg" variable, which in turn executes the Cleanup() method of a
Filesystem structure from our "fs" package.

Next, we revise the main() function so that it calls the Cleanup()
function of the "commands" package directly, both at the end of
the main() function and when a SIGINT or SIGKILL signal has been
received.  As before, though, the temporary file cleanup operation
itself should occur at most one time only, as it is exclusively
invoked via the Once structure's Do() method.

With these changes in place, we will now be able to make further
refinements to ensure that the Git LFS client always performs
cleanup operations when exiting.  In subsequent commits in this PR
we will update the exit wrapper functions in our "commands" package
so they all call the Cleanup() function, which can now guarantee
on its own that it will only execute the cleanup activities a
single time.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant