[Linux] Add GVFS platform and build configuration for Linux#1125
[Linux] Add GVFS platform and build configuration for Linux#1125chrisd8088 merged 24 commits intomicrosoft:features/linuxprototypefrom
Conversation
0fa217b to
5288077
Compare
deacb1d to
02fc9d4
Compare
5288077 to
47fb515
Compare
02fc9d4 to
6e457bc
Compare
47fb515 to
972f05a
Compare
6e457bc to
af62389
Compare
6923bb4 to
54dc071
Compare
|
/azp run PR - Linux - Build and Unit Test |
|
Azure Pipelines successfully started running 1 pipeline(s). |
e898f42 to
aa19474
Compare
9f017e7 to
e966c32
Compare
b3da2ce to
9c13a19
Compare
ea1c6df to
d91e98b
Compare
|
@wilbaker -- I realize there's a lot of fresh code here, since we're adding a whole new platform to the GVFS provider. To that end we've tried to keep the sequence of commits as clean and logical as possible (hence all the rebasing), so you might find it easier to review by looking at the commit sequence, at least in part. There are places where we have some code duplication between Mac and Linux, and that'll be the focus of another PR based on |
96512e8 to
e33d831
Compare
A directory containing only ".git" and ".gitattributes" will be an uninitialized GVFS clone, but MirrorProvider will have an empty initial directory without any Git artifacts.
In order to prevent exceptions triggered by file projection requests arriving prior to our exit from our wait loop in our ProjFS.Linux StartVirtualizationInstance() method, we only set the projfs variable to non-null at the tail end of the startup sequence, and then in the HandleProjEvent() callback (as well as several the placeholder update methods), we always acquire a local copy and return immediately ENODEV if it is null. The placeholder update methods should, in theory, never be called with a null this.projfs, because they are only executed by threads started after StartVirtualizationInstance() and stopped before StopVirtualizationInstance(). However, we defensely check for a null projfs to be on the safe side. Similarly, using a local copy ensures we can not trigger a NullReferenceException should projfs happen to become null after our check, although we do not expect this to be possible, since the libprojfs/libfuse threads which execute projection requests should all be stopped before our StopVirtualizationInstance() method sets projfs to null. Also remove a probably-impossible-but-who-knows race if StopVirtualizationInstance was called twice concurrently. Co-authored-by: Ashe Connor <ashe@kivikakk.ee>
We split the "Unix" methods into both Mac and Linux (as their magic consts differ), and change the build process so we can rely on conditionally-included projects, viz. the respective GVFS.Platform.* projects.
These are unused, and aren't portable between macOS/Linux and so shouldn't be used.
On both Linux and Mac/Darwin, the uid_t type is an unsigned int, not a signed int.
Remove Windows builds of Linux projects where we don't build them for Mac, and equivalently remove some Windows builds of the corresponding Mac projects, specifically: GVFS.Linux and GVFS.Mac GVFS.Hooks.Linux and GVFS.Hooks.Mac GVFS.Mount.Linux and GVFS.Mount.Mac Co-authored-by: Chris Darroch <chrisd8088@github.com>
e33d831 to
b8b89b5
Compare
| { | ||
| excludeCategories.Add(Categories.LinuxOnly); | ||
| } | ||
|
|
There was a problem hiding this comment.
- On Linux should we also be excluding
Categories.MacOnly? - Could you incorporate these new exclusions in this
if/elseblock just below?
if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX))
{
excludeCategories.Add(Categories.MacTODO.FailsOnBuildAgent);
excludeCategories.Add(Categories.MacTODO.NeedsNewFolderCreateNotification);
excludeCategories.Add(Categories.MacTODO.NeedsGVFSConfig);
excludeCategories.Add(Categories.MacTODO.NeedsDehydrate);
excludeCategories.Add(Categories.MacTODO.NeedsServiceVerb);
excludeCategories.Add(Categories.MacTODO.NeedsStatusCache);
excludeCategories.Add(Categories.MacTODO.NeedsCorruptObjectFix);
excludeCategories.Add(Categories.MacTODO.TestNeedsToLockFile);
excludeCategories.Add(Categories.WindowsOnly);
}
else
{
excludeCategories.Add(Categories.MacOnly);
}
There was a problem hiding this comment.
- We already exclude
MacOnlybecause of that same following block, which addsMacOnlyto the exclusion list if the platform is notOSPlatform.OSX. - I'm unsure on what you're asking with the second question here. Incorporate what new exclusions where?
There was a problem hiding this comment.
- I'm unsure on what you're asking with the second question here. Incorporate what new exclusions where?
(To be clear, not trying to be obtuse! Just not parsing the question well.)
There was a problem hiding this comment.
My reading of @wilbaker's suggestion here is that because many of these functional tests have not yet been ported to the Mac (and are therefore Windows-only not by design, but only because of lack of time), we will want to, for now, also exclude these on Linux. I'll make that change shortly!
There was a problem hiding this comment.
I actually think I'm going to leave this as-is for now, because we haven't yet run the functional tests on Linux (see github#16) and so we may find that we need a different, likely larger, set of LinuxTODO tests. Until we investigate, it's a little hard to know which ones from the MacTODO category will really be POSIX-TODO and apply to Linux as well, and which are Mac-specific.
But as I'm 100% certain we'll be revisiting these tests and categoroes very soon, I think I'll go ahead and merge with this as it stands, as a placeholder (projecting filesystem pun intended! :-)
There was a problem hiding this comment.
@kivikakk @chrisd8088 apologies for the confusion.
The idea in the second point was to combine the new if/else with the existing one below so that it would look something like:
if (RuntimeInformation.IsOSPlatform(OSPlatform.OSX))
{
excludeCategories.Add(... stuff excluded on Mac ...);
}
else if (RuntimeInformation.IsOSPlatform(Linux))
{
excludeCategories.Add(... stuff excluded on Linux ...);
}
else
{
excludeCategories.Add(... stuff excluded on Windows ...);
}
It can get tricky to keep track of what we're excluding or including, and it might be easier to keep track if we set them all in one spot.
There was a problem hiding this comment.
To clarify, when I wrote excludeCategories.Add(... stuff excluded on <Platform> ...); I meant call excludeCategories.Add as many times as needed in each block to exclude what should be excluded on each platform
There was a problem hiding this comment.
Ah, makes perfect sense. I'll do a PR for this -- thanks!
GVFS/GVFS.FunctionalTests/FileSystemRunners/FileSystemRunner.cs
Outdated
Show resolved
Hide resolved
| EndProject | ||
| Global | ||
| GlobalSection(SolutionConfigurationPlatforms) = preSolution | ||
| Debug.Linux|x64 = Debug.Linux|x64 |
There was a problem hiding this comment.
The good news is I was able to build your branch on Mac with Scripts/Mac/BuildGVFSForMac.sh. The bad news is that something is configured in a way that VS for Mac doesn't like, but that can get sorted out before this is merged to master.
There was a problem hiding this comment.
Hm, can you describe it at all? I just tried a reasonably fresh build with VS for Mac of this tree and everything worked OK.
I trashed BuildOutput, packages, Publish, ran enough of BuildGVFSForMac.sh to generate the constants and assembly version files, then built the solution. Everything looks fine.
I will note that I encounter issues regularly if I try to build with VS and then build script (or vice versa) without cleaning everything in between — VS gets confused, red-underlines everything, says System is undefined, etc. etc. Just to see if I could trigger it in this case, after building with VS per above, I ran the full BuildGVFSForMac.sh, and then again in VS, and this one time it actually works.
(Well, actually, I get the incompatible SDK error thing on two projects, but that's not what I'm talking about and is already known!)
There was a problem hiding this comment.
Hi, these are the errors I'm seeing:
Error: NuGet packages need to be restored before building. NuGet MSBuild targets are missing and are needed for building. The NuGet MSBuild targets are generated when the NuGet packages are restored. (GVFS.Common)
Error: NuGet packages need to be restored before building. NuGet MSBuild targets are missing and are needed for building. The NuGet MSBuild targets are generated when the NuGet packages are restored. (GVFS.Hooks.Mac)
Error: NuGet packages need to be restored before building. NuGet MSBuild targets are missing and are needed for building. The NuGet MSBuild targets are generated when the NuGet packages are restored. (GVFS.Tests)
I tried deleting BuildOutput, packages, and Publish but it didn't help.
There was a problem hiding this comment.
Trying to make my steps reproducible:
- Start with VS closed.
- Deleted
BuildOutput,packages,Publish. The clone is at~/microsoft/VFSForGit/src, and theVFSForGitdirectory is empty except for the clone atsrc. - Performed
git clean -fdxin the clone. - Opened
GVFS.slnin Visual Studio for Mac (Community, 8.0.4 build 0). - It automatically restored packages when opening the solution. (I'll omit the entire Package Console log, but the last line reads
Packages successfully restored.) - Tried to build the solution. Failed because I have
Debug.Windowsselected and a bunch of "This project type is not supported by Visual Studio Community 2019 for Mac." messages appeared. - Switched to
Debug.Mac. Failed becauseGVFSConstants.GitVersion.csis not found. - Made these changes to
BuildGVFSForMac.shand ran the script to create the two generated.csfiles. - Built the solution again.
-
========== Build: 16 succeeded, 0 failed, 2 up-to-date, 7 skipped ========== Build successful.
It's possible the git clean step was necessary? I'm not sure.
There was a problem hiding this comment.
Ahh thanks, I did forget to run git clean, let me give that a try.
[Linux] Add GVFS platform and build configuration for Linux We add the core GVFS.Platform.Linux classes, derived from their GVFS.Platform.Mac equivalents but with appropriate changes for syscall argument signatures and flag values, type definitions, and structure fields (e.g., mode_t is a uint, and the layout of struct stat is quite different). In order to avoid conflicting with the GNOME Virtual file system (GVfs), we use .vfsforgit as the name of our per-repository private directories, instead of .gvfs, as is used on Windows and Mac. The build tooling is largely a duplicate of the corresponding Mac configuration, with some changes to ensure we build successfully on Linux (e.g., testing for != Windows is no longer sufficient to determine that the platform is OS X, so we add distinct IsLinux and IsOSX build conditions to the .csproj file for GVFS.Upgrader, among other subprojects). The GVFS.sln configuration changes should ensure that the ProjFS.Linux.Managed and GVFS.Platform.Linux subprojects are built on all platforms (and their Mac and Windows equivalents are built on Linux), and that the GVFS.Service.Mac classes are built on Linux, so as to allow the GVFS.UnitTests subproject to build and run successfully on all three platforms. We also build the GVFS.Hooks.Linux and GVFS.Mount.Linux subprojects on both Linux and Windows, just as their Mac counterparts are built on both Mac and Windows. The GVFS.Service binary is, for the time being, only a stub on Linux, as we have yet to determine how (or if) we should handle auto-mounting repositories. Note that for now the BuildGVFSForLinux.sh script does not perform any installation or packaging.
[Linux] Add GVFS platform and build configuration for Linux We add the core GVFS.Platform.Linux classes, derived from their GVFS.Platform.Mac equivalents but with appropriate changes for syscall argument signatures and flag values, type definitions, and structure fields (e.g., mode_t is a uint, and the layout of struct stat is quite different). In order to avoid conflicting with the GNOME Virtual file system (GVfs), we use .vfsforgit as the name of our per-repository private directories, instead of .gvfs, as is used on Windows and Mac. The build tooling is largely a duplicate of the corresponding Mac configuration, with some changes to ensure we build successfully on Linux (e.g., testing for != Windows is no longer sufficient to determine that the platform is OS X, so we add distinct IsLinux and IsOSX build conditions to the .csproj file for GVFS.Upgrader, among other subprojects). The GVFS.sln configuration changes should ensure that the ProjFS.Linux.Managed and GVFS.Platform.Linux subprojects are built on all platforms (and their Mac and Windows equivalents are built on Linux), and that the GVFS.Service.Mac classes are built on Linux, so as to allow the GVFS.UnitTests subproject to build and run successfully on all three platforms. We also build the GVFS.Hooks.Linux and GVFS.Mount.Linux subprojects on both Linux and Windows, just as their Mac counterparts are built on both Mac and Windows. The GVFS.Service binary is, for the time being, only a stub on Linux, as we have yet to determine how (or if) we should handle auto-mounting repositories. Note that for now the BuildGVFSForLinux.sh script does not perform any installation or packaging.
We add the core
GVFS.Platform.Linuxclasses, derived from theirGVFS.Platform.Macequivalents but with appropriate changes for syscall argument signatures and flag values, type definitions, and structure fields (e.g.,mode_tis auint, and the layout ofstruct statis quite different).In order to avoid conflicting with the GNOME Virtual file system (GVfs), we use
.vfsforgitas the name of our per-repository private directories, instead of.gvfs, as is used on Windows and Mac.The build tooling is largely a duplicate of the corresponding Mac configuration, with some changes to ensure we build successfully on Linux (e.g., testing for
!= Windowsis no longer sufficient to determine that the platform is OS X, so we add distinctIsLinuxandIsOSXbuild conditions to the.csproj file forGVFS.Upgrader, among other subprojects).The
GVFS.slnconfiguration changes should ensure that theProjFS.Linux.ManagedandGVFS.Platform.Linuxsubprojects are built on all platforms (and their Mac and Windows equivalentsare built on Linux), and that the
GVFS.Service.Macclasses are built on Linux, so as to allow theGVFS.UnitTestssubproject to build and run successfully on all three platforms.We also build the
GVFS.Hooks.LinuxandGVFS.Mount.Linuxsubprojects on both Linux and Windows, just as their Mac counterparts are built on both Mac and Windows.The
GVFS.Servicebinary is, for the time being, only a stub on Linux, as we have yet to determine how (or if) we should handle auto-mounting repositories.Note that for now the
BuildGVFSForLinux.shscript does not perform any installation or packaging./cc @jrbriggs, @wilbaker