Skip to content

enable FetchStepTests on Mac/Linux and remove MacTODO test categories#426

Merged
chrisd8088 merged 2 commits intomicrosoft:mainfrom
chrisd8088:fix-fetch-lock-test
Sep 8, 2020
Merged

enable FetchStepTests on Mac/Linux and remove MacTODO test categories#426
chrisd8088 merged 2 commits intomicrosoft:mainfrom
chrisd8088:fix-fetch-lock-test

Conversation

@chrisd8088
Copy link
Contributor

@chrisd8088 chrisd8088 commented Sep 5, 2020

The FetchStepCleansUpStaleFetchLock() functional test currently only succeeds on Windows (and is therefore marked with the MacTODO.TestNeedsToLockFile category) because it checks whether the FetchStep command has released its lock file by testing if the lock file has been deleted.

This works with our WindowsFileBasedLock implementation, which opens the lock file with FileOptions.DeleteOnClose, so the file is removed when the lock is released. However, the MacFileBasedLock version leaves the file in place and uses flock(2) and LOCK_EX to acquire and release the lock atomically; there is no simple way to delete the file while still holding the open file descriptor needed for flock().

To make this test succeed on Mac/Linux while still being meaningful, we need to be robust against the lock file existing both at the start of the test and afterwards, while still ensuring that the lock has been released by the FetchStep command.

We therefore simply attempt to open the lock file after the fetch command completes, using the FileShare.None attribute, which will fail on all platforms if the lock file has not been released.

On Windows this is because WindowsFileBasedLock uses FileShare.Read to open the lock file, so if the lock is still held our attempt with FileShare.None will fail.

On macOS (and Linux) this is because our use of flock() in MacFileBasedLock will conflict with the flock() with LOCK_EX which the .NET Core Unix implementation of FileShare.None also uses.

See the discussions in microsoft/VFSForGit#213, dotnet/runtime#24432, as well as the .NET Core library implementation.

We also rename the test to FetchStepReleasesFetchLockFile() to more accurately reflect what it is now testing.

And having now addressed the issues preventing the tests in the MacTODO.TestNeedsToLockFile category from running on macOS, we can now remove this class of categories entirely.

The FetchStepCleansUpStaleFetchLock() functional test currently
only succeeds on Windows (and is therefore marked with the
MacTODO.TestNeedsToLockFile category) because it checks whether
the FetchStep command has released its lock file by testing if
the lock file has been deleted.

This works with our WindowsFileBasedLock implementation, which
opens the lock file with FileOptions.DeleteOnClose, so the file is
removed when the lock is released.  However, the MacFileBasedLock
version leaves the file in place and uses flock(2) and LOCK_EX
to acquire and release the lock atomically; there is no simple
way to delete the file while still holding the open file
descriptor needed for flock().

To make this test succeed on Mac/Linux while still being meaningful,
we need to be robust against the lock file existing both at the
start of the test and afterwards, while still ensuring that the
lock has been released by the FetchStep command.

We therefore simply attempt to open the lock file after the fetch
command completes, using the FileShare.None attribute, which will
fail on all platforms if the lock file has not been released.

On Windows this is because WindowsFileBasedLock uses FileShare.Read
to open the lock file, so if the lock is still held our attempt
with FileShare.None will fail.

On macOS (and Linux) this is because our use of flock() in
MacFileBasedLock will conflict with the flock() with LOCK_EX which
the .NET Core Unix implementation of FileShare.None also uses.

See the discussions in microsoft/VFSForGit#213, dotnet/runtime#24432,
as well as the .NET Core library implementation in:

https://github.com/dotnet/runtime/blob/341b8b22280e297daa7d96b4b8a9fa1d40d28aa9/src/libraries/System.Private.CoreLib/src/System/IO/FileStream.Unix.cs#L73-L88

We also rename the test to FetchStepReleasesFetchLockFile() to
more accurately reflect what it is now testing.
Having addressed the issues preventing the tests in the
MacTODO.TestNeedsToLockFile category from running on macOS,
we can now remove this class of categories entirely.
Copy link
Contributor

@derrickstolee derrickstolee left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Making a mental note to test that we properly acquire the lock on macOS when the file-based lock exists on disk but our open permissions give us access.

@chrisd8088 chrisd8088 merged commit 5c5cf30 into microsoft:main Sep 8, 2020
@chrisd8088 chrisd8088 deleted the fix-fetch-lock-test branch September 8, 2020 16:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants