Skip to content

git/githistory/log: implement ListTask type#2335

Merged
ttaylorr merged 10 commits intomasterfrom
githistory-log-list-task
Jun 19, 2017
Merged

git/githistory/log: implement ListTask type#2335
ttaylorr merged 10 commits intomasterfrom
githistory-log-list-task

Conversation

@ttaylorr
Copy link
Contributor

This pull request provides a new Task implementation, ListTask, which is used to log line-delimited lists of data using the git/githistory/log.Logger type.

This is required work for the import subcommand of git-lfs-migrate(1) which will print out one line for each ref updated by the migrator.


/cc @git-lfs/core
/refs #2146 #2329 #2334

@ttaylorr ttaylorr added this to the v2.2.0 milestone Jun 16, 2017
@ttaylorr ttaylorr requested a review from technoweenie June 16, 2017 21:22
@ttaylorr
Copy link
Contributor Author

Updated from the feedback in #2334 (comment), so this should be ready for 👀.

}

// Durable implements the Task.Durable function and ensures that all log updates
// are printed to the sink.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍 This is a better comment than the other Durable() ones in #2334 like:

Durable implements Task.Durable and returns false, indicating that this task is not durable.

@ttaylorr ttaylorr changed the base branch from githistory-log-durable-tasks to master June 19, 2017 21:17
@ttaylorr ttaylorr merged commit 42661cd into master Jun 19, 2017
@ttaylorr ttaylorr deleted the githistory-log-list-task branch June 19, 2017 21:40
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request May 26, 2023
In commit e90d54d of PR git-lfs#2329 the
"git/githistory/log" package was added, including the PercentageTask
structure and associated methods, but unlike the WaitingTask which
was added at the same time, no Complete() method was defined for
the PercentageTask structure.

(Note that the "git/githistory/log" package was later renamed to the
"tasklog" package in commits b9ab79e
and 45c580e of PR git-lfs#2747.)

Other task types such as the ListTask and SimpleTask structures
(introduced in commit 31ffeb9 of
PR git-lfs#2335 and commit 7a760b6 of PR git-lfs#2756,
respectively) provide a Complete() method, and the Meter task type
of the "tq" package, which implemented the Task interface in commit
7c0f9e2 of PR git-lfs#2732, provides an
equivalent Finish() method.  These methods allow the caller to
explicitly close the channel returned by the Updates() method,
on which the anonymous goroutine started by the Logger.consume() method
for the task is waiting, in a "range" loop on the channel in the
Logger.logTask() method.

One key use of the PercentageTask structure is in the methods of
the Rewriter structure from the "git/githistory" package.  It is
initialized with the number of commits to be rewritten during
a "git lfs migrate" command's traversal of a Git repository's history.
As each commit is rewritten, the Count() method is called to
increment the percentage completed value.  When every commit has
been rewritten, the count reaches the expected total, and the
Count() method closes the channel, thus allowing the task receiving
updates to finish and exit its goroutine.

Under exceptional circumstances, though, the Rewrite() method of
the Rewriter structure may never finish iterating through all
the expected commits, and return in error or via a panic call.
When this happens, the goroutine waiting on the PercentageTask's
updates channel can never exit, leading to a hung process which
never fully exits.

In order to allow the Rewriter's Rewrite() method to define a
deferred function which will always be called when it returns,
under any circumstances, we add a Complete() method for the
PercentageTask structure which sets the number of completed elements
to the expected total in an atomic swap, and then closes the
updates channel if the number of completed elements was previously
less than the expected total.

We also add a test to validate this new method's behaviour, and
update the existing TestPercentageTaskCallsDoneWhenComplete()
function to also confirm that calling the new Complete() method
after the number of completed elements has reached the expected
total does not cause a second attempt to close the updates channel.
chrisd8088 added a commit to chrisd8088/git-lfs that referenced this pull request May 26, 2023
In commit e90d54d of PR git-lfs#2329 the
"git/githistory/log" package was added, including the PercentageTask
structure and associated methods, but unlike the WaitingTask which
was added at the same time, no Complete() method was defined for
the PercentageTask structure.

(Note that the "git/githistory/log" package was later renamed to the
"tasklog" package in commits b9ab79e
and 45c580e of PR git-lfs#2747.)

Other task types such as the ListTask and SimpleTask structures
(introduced in commit 31ffeb9 of
PR git-lfs#2335 and commit 7a760b6 of PR git-lfs#2756,
respectively) provide a Complete() method, and the Meter task type
of the "tq" package, which implemented the Task interface in commit
7c0f9e2 of PR git-lfs#2732, provides an
equivalent Finish() method.  These methods allow the caller to
explicitly close the channel returned by the Updates() method,
on which the anonymous goroutine started by the Logger.consume() method
for the task is waiting, in a "range" loop on the channel in the
Logger.logTask() method.

One key use of the PercentageTask structure is in the methods of
the Rewriter structure from the "git/githistory" package.  It is
initialized with the number of commits to be rewritten during
a "git lfs migrate" command's traversal of a Git repository's history.
As each commit is rewritten, the Count() method is called to
increment the percentage completed value.  When every commit has
been rewritten, the count reaches the expected total, and the
Count() method closes the channel, thus allowing the task receiving
updates to finish and exit its goroutine.

Under exceptional circumstances, though, the Rewrite() method of
the Rewriter structure may never finish iterating through all
the expected commits, and return in error or via a panic call.
When this happens, the goroutine waiting on the PercentageTask's
updates channel can never exit, leading to a hung process which
never fully exits.

In order to allow the Rewriter's Rewrite() method to define a
deferred function which will always be called when it returns,
under any circumstances, we add a Complete() method for the
PercentageTask structure which sets the number of completed elements
to the expected total in an atomic swap, and then closes the
updates channel if the number of completed elements was previously
less than the expected total.

We also add a test to validate this new method's behaviour, and
update the existing TestPercentageTaskCallsDoneWhenComplete()
function to also confirm that calling the new Complete() method
after the number of completed elements has reached the expected
total does not cause a second attempt to close the updates channel.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants