Skip to content

When we're loading dependencies, don't require exact versions#56432

Merged
jasonmalinowski merged 13 commits intodotnet:mainfrom
jasonmalinowski:allow-loading-of-higher-assembly-versions
Nov 30, 2021
Merged

When we're loading dependencies, don't require exact versions#56432
jasonmalinowski merged 13 commits intodotnet:mainfrom
jasonmalinowski:allow-loading-of-higher-assembly-versions

Conversation

@jasonmalinowski
Copy link
Copy Markdown
Member

@jasonmalinowski jasonmalinowski commented Sep 16, 2021

When we're loading analyzer dependencies, our current logic was to require the exact version that was requested from the loading assembly. This is problematic in a lot of cases where you would normally have a binding redirect to ensure assemblies load cleanly; by requiring exact versions we'd end up in diamond dependency cases where you couldn't practically produce an analyzer package that would cleanly load.

While doing this change, @RikkiGibson and I noticed several other problems:

  1. There was an errant cache that was left in the .NET Core implementation, which meant that we'd sometimes reuse assemblies even when we were able to load them correctly. This has also been fixed as a prerequisite to the main fix, since otherwise it was difficult to prove that the behavior changes here weren't also going to impact the .NET Core implementation.
  2. There were some race conditions in the implementation.

REVIEWING COMMIT-AT-A-TIME IS HIGHLY RECOMMENDED, since those changes are broken out to separate commits.

Fixes #53672
Fixes #56216

@ghost ghost added the Area-Analyzers label Sep 16, 2021
@jasonmalinowski jasonmalinowski self-assigned this Sep 16, 2021
@jasonmalinowski jasonmalinowski force-pushed the allow-loading-of-higher-assembly-versions branch from d66bbd2 to 3db6ee0 Compare September 16, 2021 22:55
jasonmalinowski added a commit to jasonmalinowski/omnisharp-roslyn that referenced this pull request Sep 17, 2021
Roslyn has made some improvements to better handle the case where
analyzers or generators have a mix of assembly versions, such as
in microsoft/CsWin32#218. The Roslyn
changes were made in dotnet/roslyn#56432, and
this consumes it.
Comment thread src/Tools/ExternalAccess/OmniSharp/Analyzers/OmnisharpAnalyzerLoaderFactory.cs Outdated
@jasonmalinowski jasonmalinowski force-pushed the allow-loading-of-higher-assembly-versions branch from acbf323 to 1d06f5d Compare September 17, 2021 23:18
@jasonmalinowski jasonmalinowski marked this pull request as ready for review September 17, 2021 23:19
@jasonmalinowski jasonmalinowski requested review from a team as code owners September 17, 2021 23:19
Comment thread src/Compilers/Test/Core/AssemblyLoadTestFixture.cs Outdated
Comment thread src/Compilers/Test/Core/AssemblyLoadTestFixture.cs Outdated
Comment thread src/Workspaces/CoreTest/ShadowCopyAndAllowRedirectsAssemblyLoaderTests.cs Outdated
Comment thread src/Compilers/Test/Core/AssemblyLoadTestFixture.cs
Comment thread src/Compilers/Test/Core/AssemblyLoadTestFixture.cs Outdated
genlu
genlu previously approved these changes Sep 21, 2021
@sharwell
Copy link
Copy Markdown
Contributor

If I have a solution with two different projects, each with their own version of StyleCop.Analyzers installed, will the IDE still load both versions and match them with the corresponding project after this change?

@jasonmalinowski
Copy link
Copy Markdown
Member Author

@sharwell, hmm, good point. Will add tests.

@jasonmalinowski jasonmalinowski marked this pull request as draft September 21, 2021 18:55
TheBlubb14 added a commit to TheBlubb14/BlubbiChanged that referenced this pull request Oct 10, 2021
@jasonmalinowski jasonmalinowski force-pushed the allow-loading-of-higher-assembly-versions branch 3 times, most recently from 67120b6 to eedcff7 Compare November 2, 2021 01:38
@jasonmalinowski jasonmalinowski changed the title When we're loading dependencies in VS, don't require exact versions When we're loading dependencies, don't require exact versions Nov 2, 2021
@jasonmalinowski jasonmalinowski force-pushed the allow-loading-of-higher-assembly-versions branch from eedcff7 to 15522e3 Compare November 2, 2021 18:33
@jasonmalinowski jasonmalinowski force-pushed the allow-loading-of-higher-assembly-versions branch from 15522e3 to 879bb90 Compare November 2, 2021 19:06
There's a bunch of similarly named methods here, this just clarifies one
of them in particular.
Copy link
Copy Markdown
Member

@RikkiGibson RikkiGibson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change looks good to me! Thanks so much for the refactoring of desktop-specific stuff and for making the test fixture much more clear.

Comment thread src/Compilers/Test/Core/AssemblyLoadTestFixture.cs Outdated
}

var identity = AssemblyIdentityUtils.TryGetAssemblyIdentity(fullPath);
return AddToCache(fullPath, identity);
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is just going to call _loadedAssemblyIdentitiesByPath again and i'm not sure I see the purpose.

Consider that this implementation needs to decide if it requires the invariant that references do or do not change on disk. My assumption is that the invariant required here is references don't change on disk (otherwise there are other issues with the PR). If that is the case then why do we need to re-read from _loadedAsseblyIdentitiesByPath here? Sholudn't we just grab a lock and assign in?

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jaredpar: this is unchanged from the current implementation:

private AssemblyIdentity GetOrAddAssemblyIdentity(string fullPath)
{
Debug.Assert(PathUtilities.IsAbsolute(fullPath));
lock (_guard)
{
if (_loadedAssemblyIdentitiesByPath.TryGetValue(fullPath, out var existingIdentity))
{
return existingIdentity;
}
}
var identity = AssemblyIdentityUtils.TryGetAssemblyIdentity(fullPath);
return AddToCache(fullPath, identity);
}

I don't know why it's done that way either.

_loadedAssembliesByPath[fullPath] = assembly;

return assembly;
_loadedAssembliesByPath[fullPath] = loadedAssembly;
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider this sequence in the context of my other comment about invariants. There is no attempt to see if the state of the collection changed between lock calls, it's just a simple write operation.

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This appears to be a bug if the goal is to ensure that two loads for the same path always return the same thing per the interface documentation, but it's been around since #11956. @mavasani or @tmat any chance if this is a load-bearing bug?

Comment thread src/Compilers/Core/Portable/DiagnosticAnalyzer/AnalyzerAssemblyLoader.cs Outdated
We had a cache in AnalyzerAssemblyLoader that tried to cache and reuse
the same assembly, keyed by assembly identity. This code makes some
sense to keep around for the .NET full framework case where we can't
load multiple assemblies with the same identity, but doesn't make
sense to keep around for our AssemblyLoadContext implementation.
The AssemblyLoadContext code correctly isolated dependencies and didn't
use this cache when resolving dependencies, but it was still active when
the public LoadFromPath entrypoint was called -- we'd hit the cache
there before doing anything else.

To ensure we don't have any identity-related caching logic reappearing
in the AssemblyLoadContext implementation, the identity-related logic
has been moved to DefaultAnalyzerAssemblyLoader.Desktop.cs, which
greatly simplifies the base type.
@jasonmalinowski jasonmalinowski force-pushed the allow-loading-of-higher-assembly-versions branch from ebb7048 to 7d8471a Compare November 23, 2021 00:02
If you had loading happening at the same time as a call to
AddDependencyLocation, bad things might happen:

- GetPaths wasn't taking a lock before looking at the dictionary
- Code used the returned HashSet outside of the guarding lock
@jasonmalinowski jasonmalinowski force-pushed the allow-loading-of-higher-assembly-versions branch from 7d8471a to 00db8bd Compare November 23, 2021 00:03
When we're loading analyzer dependencies, our current logic was to
require the exact version that was requested from the loading
assembly. This is problematic in a lot of cases where you would normally
have a binding redirect to ensure assemblies load cleanly in diamond
dependency scenarios. Without this, if you depend on B.dll and A.dll,
but B.dll depends on an older version of A.dll, you can't produce a
package that actually loads because there's no sane way to package both
A.dlls at the same time.

We avoid adding any logic here where we'll still try to use an older
version if that's still available. Consider a more complicated example
of:

1. Analyzer 1 depends on and ships B.dll (version 2) and A.dll
   (version 2); B.dll (version 2) depends on A.dll (version 1).
2. Analyzer 2 depends on A.dll (version 1) and ships that.

When loading Analyzer 1, we need a redirect to ensure the A that it
consumes and the A that B consumes are unified. We don't want the
presence of Analyzer 2 to change that. We could satisfy B.dll's request
with Ananlyer 2's dependency, but if Analyzer 1 expects type unification
then adding Analyzer 2 to the build would break it.

Fixes dotnet#53672
@jasonmalinowski jasonmalinowski force-pushed the allow-loading-of-higher-assembly-versions branch from a727f51 to dca430c Compare November 23, 2021 01:17
@jasonmalinowski jasonmalinowski merged commit 0bdc853 into dotnet:main Nov 30, 2021
@ghost ghost added this to the Next milestone Nov 30, 2021
@jasonmalinowski jasonmalinowski deleted the allow-loading-of-higher-assembly-versions branch November 30, 2021 03:29
@allisonchou allisonchou modified the milestones: Next, 17.1.P2 Nov 30, 2021
jasonmalinowski added a commit to jasonmalinowski/omnisharp-roslyn that referenced this pull request Dec 2, 2021
Roslyn has made some improvements to better handle the case where
analyzers or generators have a mix of assembly versions, such as
in microsoft/CsWin32#218. The Roslyn
changes were made in dotnet/roslyn#56432, and
this consumes it.
jasonmalinowski added a commit to jasonmalinowski/omnisharp-roslyn that referenced this pull request Jan 4, 2022
Roslyn has made some improvements to better handle the case where
analyzers or generators have a mix of assembly versions, such as
in microsoft/CsWin32#218. The Roslyn
changes were made in dotnet/roslyn#56432, and
this consumes it.
jasonmalinowski added a commit to jasonmalinowski/omnisharp-roslyn that referenced this pull request Jan 26, 2022
Roslyn has made some improvements to better handle the case where
analyzers or generators have a mix of assembly versions, such as
in microsoft/CsWin32#218. The Roslyn
changes were made in dotnet/roslyn#56432, and
this consumes it.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

7 participants