UnGAC MSBuild Assemblies on Install or Update by benvillalobos · Pull Request #5626 · dotnet/msbuild

benvillalobos · 2020-08-07T17:44:04Z

Forgind

Just initial feedback

Forgind · 2020-08-07T21:09:26Z

src/Microsoft.Build.UnGAC/Program.cs

+                string s = output.ReadLine();
+
+                // Store all instances of gacutil for now.
+                while (!String.IsNullOrEmpty(s))
+                {
+                    allInstancesOfGACUtil.Add(s);
+                    s = output.ReadLine();
+                }


Suggested change

string s = output.ReadLine();

// Store all instances of gacutil for now.

while (!String.IsNullOrEmpty(s))

{

allInstancesOfGACUtil.Add(s);

s = output.ReadLine();

}

// Store all instances of gacutil for now.

for (string loc = output.ReadLine(); !String.IsNullOrEmpty(loc); loc = output.ReadLine())

{

allInstancesOfGACUtil.Add(s);

}

Does this terminate appropriately? If so, I think there's a ReadAllLines function that might do the same. (I might have made that up 😄.)

Also, do we really care about any gacutils after the first?

Does this terminate appropriately? If so, I think there's a ReadAllLines function that might do the same. (I might have made that up 😄.)

It does and has always, which kind of worries me. I would do ReadAllLines but have experienced lockups from that as well. Perhaps I can clear the buffer (as I do below) so I can safely pull everything.

Also, do we really care about any gacutils after the first?

Honestly I don't think so. I did it because "why not" while I had them there. The final version will almost certainly not store all instances.

Forgind · 2020-08-07T21:09:55Z

src/Microsoft.Build.UnGAC/Program.cs

+                input.WriteLine("where /F /R \"C:\\Program Files (x86)\\Microsoft SDKs\\Windows\" gacutil.exe");
+
+                // Throw away the first line - it's the command we just used
+                output.ReadLine();


This would probably make it messier rather than cleaner, but you should be able to set echo to off to suppress this.

Forgind · 2020-08-07T21:10:25Z

src/Microsoft.Build.UnGAC/Program.cs

+                StreamReader output;
+                StreamWriter input;
+
+                Process proc = new Process


This seems heavy. Is there a way to run it in-proc?

I'm not aware of one. I considered using an exec task but we need this to be an exe. If someone knows I'd love to know!

@ladipro?
He knows everything.

You can access the GAC via the Fusion interface, for example:
https://docs.microsoft.com/en-us/dotnet/framework/unmanaged-api/fusion/iassemblycache-uninstallassembly-method

Not sure if a COM interop wrapper exists or you'll have to build one yourself. I guess a big reason to go this route would be if gacutil.exe is not guaranteed to be present on the machine. Otherwise, if you know for sure there is gacutil.exe and you can find it, it may be easier to just execute it. I'm assuming we don't care about performance here.

Good idea. There are a few that exist, but either the links to them are dead or they're not supported anymore. I found https://github.com/EasyHook/EasyHook
http://blog.wibeck.org/2008/10/working-with-gac-from-code/ points to an API wrapper that unfortunately 404's today.
There's also https://www.codeproject.com/Articles/8285/GAC-API-Interface

This is likely of interest:

msbuild/src/Tasks/NativeMethods.cs

Lines 344 to 351 in f2c4bfd

[ComImport, Guid("E707DCDE-D1CD-11D2-BAB9-00C04F8ECEAE"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]

internal interface IAssemblyCache

{

/* Unused.

[PreserveSig]

int UninstallAssembly(uint dwFlags, [MarshalAs(UnmanagedType.LPWStr)] string pszAssemblyName, IntPtr pvReserved, int pulDisposition);

*/

int UninstallAssembly();

I don't think we can depend on gacutil.exe existing: the VS install may be what installs it. We could maybe express a dependency on "some .NET SDK" and use that, but it's probably easier to use the COM object.

it's probably easier to use the COM object

😲

I just spent a good chunk of time trying to get https://docs.microsoft.com/en-us/dotnet/api/system.enterpriseservices.internal.publish.gacremove?view=netframework-4.8 working to no avail (passing in file path, assembly name, pointing directly to dll, etc). MattCav suggested it was preferable to use some sort of API and keep the code simple so that's the route I'll go.

Reusing NativeMethods was the way to go. The program does need to be running as admin for it to work, however.

Forgind · 2020-08-07T21:12:38Z

src/Microsoft.Build.UnGAC/Program.cs

+
+                proc.Close();
+                proc.Dispose();
+                Console.ReadKey();


What is this?

The readkey is temporary so I can see the console output before the window closes.

😕 --> 😄 --> 👍

Forgind · 2020-08-07T21:14:58Z

src/Microsoft.Build.UnGAC/Program.cs

+
+namespace Microsoft.Build.UnGAC
+{
+    class Program


Does this work, and is this the right place for this? I'm surprised first that this runs at install time rather than running every time the user builds and second that you had to make a new project to make something that runs automatically. Could you explain that a little more?

What will allow it to run at install time is the exe being pointed to from src/Package/MSBuild.VSSetup/exe.swr, which src/Package/MSBuild.VSSetup/files.swr will point at. And files.swr is already being picked up by microbuild.

benvillalobos · 2020-08-11T00:44:21Z

Another issue I've been seeing since adding the Microsoft.Build.UnGAC project:

I've tried removing the nuget packages, but can't seem to from this project.

Looks like they're taken from src/Directory.Build.targets

  <ItemGroup Condition="'$(IsUnitTestProject)' == 'true' And '$(TargetFrameworkIdentifier)' != '.NETFramework' ">
    <PackageReference Include="xunit.console" />
  </ItemGroup>

Presumably I shouldn't have this project under src, or somehow override that import for this particular project.

benvillalobos · 2020-08-21T22:53:53Z

Current issue:

.packages\microsoft.dotnet.arcade.sdk\1.0.0-beta.20411.9\tools\VisualStudio.BuildIbcTrainingInputs.targets#L21
.packages\microsoft.dotnet.arcade.sdk\1.0.0-beta.20411.9\tools\VisualStudio.BuildIbcTrainingInputs.targets(21,5): error MSB4018: The "GenerateTrainingInputFiles" task failed unexpectedly.
System.ArgumentNullException: Value cannot be null.
Parameter name: source
   at System.Linq.Enumerable.Select[TSource,TResult](IEnumerable`1 source, Func`2 selector)
   at Microsoft.DotNet.Build.Tasks.VisualStudio.IbcEntry.GetEntriesFromVsixJsonManifest(JObject json) in /_/src/Microsoft.DotNet.Build.Tasks.VisualStudio/OptProf/IbcEntry.cs:line 85
   at Microsoft.DotNet.Build.Tasks.VisualStudio.GenerateTrainingInputFiles.ExecuteImpl() in /_/src/Microsoft.DotNet.Build.Tasks.VisualStudio/OptProf/GenerateTrainingInputFiles.cs:line 84
   at Microsoft.DotNet.Build.Tasks.VisualStudio.GenerateTrainingInputFiles.Execute() in /_/src/Microsoft.DotNet.Build.Tasks.VisualStudio/OptProf/GenerateTrainingInputFiles.cs:line 42
   at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute()
   at Microsoft.Build.BackEnd.TaskBuilder.<ExecuteInstantiatedTask>d__26.MoveNext()

VisualStudio.BuildIbcTrainingInputs.targets is imported in AfterSigning.proj.

rainersigwald · 2020-09-02T14:18:36Z

MSBuild.Dev.sln

+# Visual Studio Version 16
+VisualStudioVersion = 16.0.30320.27


Nit: Go ahead and revert this entirely since there's no meaningful change.

src/Package/MSBuild.UnGAC.Setup/Program.cs

src/Package/MSBuild.UnGAC.Setup/MSBuild.UnGAC.Setup.csproj

rainersigwald · 2020-09-02T14:21:19Z

src/Package/MSBuild.UnGAC.Setup/MSBuild.UnGAC.Setup.csproj

+    <TargetFramework>net45</TargetFramework>
+    <!-- Set as an exe because this project publishes its own output. -->
+    <OutputType>Exe</OutputType>
+    <!-- Forcing the 'vsix' output to be json output. -->


Add

... workaround for dotnet/arcade#xxxx

(and file the "support exe packages" bug there ;))

Filed: dotnet/arcade#6120

src/Package/MSBuild.UnGAC.Setup/Program.cs

src/Package/MSBuild.UnGAC.Setup/MSBuild.UnGAC.Setup.csproj

…nd thus prevent MSBuild from installing

Much cleaner!

…sed to arcade as a semicolon delimited list, which arcade can't deal with

Reverted MSBuild.Dev.sln

Forgind

Looking good!

Forgind · 2020-09-18T16:48:34Z

eng/Signing.props

 		<FileSignInfo Include="MessagePack.dll" CertificateName="3PartySHA2" /> 
-		<FileSignInfo Include="MessagePack.Annotations.dll" CertificateName="3PartySHA2" /> 
-		<ItemsToSign Include="$(VisualStudioSetupOutputPath)DevDivPackages\*.nupkg" />
+		<FileSignInfo Include="MessagePack.Annotations.dll" CertificateName="3PartySHA2" />


I'm a little confused by this; do we not need to sign DevDivPackages*.nupkg anymore?

We didn't sign it in the first place; this line was just in error.

Forgind · 2020-09-18T16:49:53Z

src/Package/Microsoft.Build.UnGAC/Microsoft.Build.UnGAC.csproj

+<Project Sdk="Microsoft.NET.Sdk">
+  <PropertyGroup>
+    <!-- Setup runs on net45 and may not have installed newer yet. -->
+    <TargetFramework>net45</TargetFramework>


Does that mean it doesn't work on mac/linux? Is there a way we can make it work on mac/linux?

The GAC is a thing only on .NET Framework (on Windows). Things do not break because of the GAC on Mac/Linux, because there's no GAC.

Forgind · 2020-09-18T16:52:14Z

src/Package/Microsoft.Build.UnGAC/Program.cs

+    /// It runs at VS install-time as well as repair-time.
+    /// It is intended to run as best effort. Meaning that if it fails, we avoid throwing and instead log it.
+    /// </summary>
+    class Program


nit: generic name

Forgind · 2020-09-18T16:56:44Z

src/Package/Microsoft.Build.UnGAC/Program.cs

+            {
+                string[] assembliesToUnGAC =
+                {
+                    "Microsoft.Build, Version=15.1.0.0",


Are these the only versions people GAC?

They're the only versions that we use, so they're the only ones that need to be removed from the GAC. Prior to VS 15, our assemblies were GACed by design; after they're not and they're all 15.1.0.0.

Forgind · 2020-09-18T16:57:17Z

src/Package/Microsoft.Build.UnGAC/Program.cs

+                    // If we hit an error with an assembly, keep trying the others.
+                    if ((hresult >> 31) == 1)
+                    {
+                        Console.WriteLine($"Could not remove {assembly} from the GAC. HResult: {hresult}");


Does it write this if the assembly just wasn't in the GAC?

It should, but I'm modifying this now to not check whether it was successful. It will just print out that it tried, and what the result was.

rainersigwald

I'd like to see your insertion-automation changes before signing off.

rainersigwald · 2020-09-18T17:09:50Z

eng/Signing.props

 		<FileSignInfo Include="MessagePack.dll" CertificateName="3PartySHA2" /> 
-		<FileSignInfo Include="MessagePack.Annotations.dll" CertificateName="3PartySHA2" /> 
-		<ItemsToSign Include="$(VisualStudioSetupOutputPath)DevDivPackages\*.nupkg" />
+		<FileSignInfo Include="MessagePack.Annotations.dll" CertificateName="3PartySHA2" />


We didn't sign it in the first place; this line was just in error.

src/Package/Microsoft.Build.UnGAC/NativeMethods.cs

src/Package/Microsoft.Build.UnGAC/Program.cs

benvillalobos · 2020-09-21T20:37:12Z

@rainersigwald you can see it here: https://dev.azure.com/devdiv/DevDiv/_releaseProgress?releaseId=815300&_a=release-pipeline-progress
The only change was under Variables/InsertJsonValues. Changed

- Microsoft.Build.vsman{$(MSBuild_ExtApisPackageVersion)+$(Build.SourceVersion)}=https://vsdrop.corp.microsoft.com/file/v1/Products/DevDiv/dotnet/msbuild/$(Build.SourceBranchName)/$(Build.BuildNumber);Microsoft.Build.vsman
+ Microsoft.Build.vsman{$(MSBuild_ExtApisPackageVersion)+$(Build.SourceVersion)}=https://vsdrop.corp.microsoft.com/file/v1/Products/DevDiv/dotnet/msbuild/$(Build.SourceBranchName)/$(Build.BuildNumber);Microsoft.Build.vsman,Microsoft.Build.UnGAC.vsman=https://vsdrop.corp.microsoft.com/file/v1/Products/DevDiv/dotnet/msbuild/$(Build.SourceBranchName)/$(Build.BuildNumber);Microsoft.Build.UnGAC.vsman

Note that I removed the version from the ungac vsman, and there comma separating Microsoft.Build.vsman and Microsoft.Build.UnGAC. If you modify the Insert VS Payload step in the release and find the Component.json values field, there's a little guide there (see Example 2) that shows you can do this.

rainersigwald · 2020-09-21T21:27:23Z

@benvillalobos that change looks good. I think this is good to go. Please change the real insertion job after you merge it.

* Added .swr file. Try catching the program to ensure it doesn't fail and thus prevent MSBuild from installing * Moved exe.swr to its own directory. Otherwise both .swr files are passed to arcade as a semicolon delimited list, which arcade can't deal with * Sign UnGAC.exe

benvillalobos added the WIP label Aug 7, 2020

benvillalobos marked this pull request as draft August 7, 2020 17:45

Forgind reviewed Aug 7, 2020

View reviewed changes

elachlan mentioned this pull request Aug 11, 2020

Broken CI Builds #5646

Closed

benvillalobos marked this pull request as ready for review August 21, 2020 22:52

Forgind mentioned this pull request Aug 24, 2020

A warning causes build to fail, but still says "0 Errors" #5100

Closed

benvillalobos removed the WIP label Sep 1, 2020

rainersigwald requested changes Sep 2, 2020

View reviewed changes

benvillalobos mentioned this pull request Sep 3, 2020

Add Support For .exe Packages dotnet/arcade#6120

Open

benvillalobos added 19 commits September 3, 2020 23:33

First implementation of Microsoft.Build.UnGAC. Workarounds galore!

3b39127

comments

ae20d69

Added .swr file. Try catching the program to ensure it doesn't fail a…

cbbd763

…nd thus prevent MSBuild from installing

catching my own nits

b032a19

nits!

46155cb

minor write update

cacd72c

Minor PR feedback

93d9f78

Testing build without packageinternalrevision

42a8bf5

Using unmanaged code.

3857ba1

Much cleaner!

Fixing build error with xunit.console

3488360

Moved exe.swr to its own directory. Otherwise both .swr files are pas…

034d84d

…sed to arcade as a semicolon delimited list, which arcade can't deal with

Transitioning UnGAC.Setup project to output its own .exe

13042b8

Working state! (hopefully)

b52892f

Remove unused ungac project

be4ea05

Cleanup

c53a3c0

csproj cleanup

1c944fc

Basic documentation

d1137ae

Remove readkey!

f2d4c9e

.exe now flows

87e9bab

benvillalobos added 3 commits September 3, 2020 23:33

PR Feedback, documentation.

a63a2ad

Reverted MSBuild.Dev.sln

Comment linking to the arcade bug.

3cfb13a

Add logging for removed assemblies.

49900b2

benvillalobos force-pushed the ungac-on-install branch from 3d48453 to 49900b2 Compare September 4, 2020 06:37

Sign UnGAC.exe

e89501e

Forgind approved these changes Sep 18, 2020

View reviewed changes

rainersigwald reviewed Sep 18, 2020

View reviewed changes

Changes from review

60039b6

rainersigwald approved these changes Sep 21, 2020

View reviewed changes

benvillalobos merged commit 4bbee8f into dotnet:master Sep 21, 2020

benvillalobos mentioned this pull request Nov 18, 2020

Bug when using MSBuild with custom target since NET5.0 update #5875

Closed

rainersigwald mentioned this pull request Jan 31, 2022

Can't build the repo locally: swc.exe exited with code -532462766 #7352

Closed

	[ComImport, Guid("E707DCDE-D1CD-11D2-BAB9-00C04F8ECEAE"), InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
	internal interface IAssemblyCache
	{
	/* Unused.
	[PreserveSig]
	int UninstallAssembly(uint dwFlags, [MarshalAs(UnmanagedType.LPWStr)] string pszAssemblyName, IntPtr pvReserved, int pulDisposition);
	*/
	int UninstallAssembly();

		# Visual Studio Version 16
		VisualStudioVersion = 16.0.30320.27

Conversation

benvillalobos commented Aug 7, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Forgind left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benvillalobos Aug 10, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benvillalobos commented Aug 11, 2020

Uh oh!

benvillalobos commented Aug 21, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Forgind left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

benvillalobos commented Aug 7, 2020 •

edited

Loading

benvillalobos Aug 10, 2020 •

edited

Loading

benvillalobos commented Sep 21, 2020 •

edited by rainersigwald

Loading