Skip to content

Determinism test needs to use bootstrap compiler#49483

Merged
jaredpar merged 2 commits intodotnet:masterfrom
jaredpar:det
Nov 19, 2020
Merged

Determinism test needs to use bootstrap compiler#49483
jaredpar merged 2 commits intodotnet:masterfrom
jaredpar:det

Conversation

@jaredpar
Copy link
Member

@jaredpar jaredpar commented Nov 18, 2020

For the determinism test to be effective it should be using the
bootstrap compiler. That was the intent of this job but at some point
the bootstrap got disabled. This fixes that and excludes some binaries
that are now failing determinism.

Tracking issue for the failed binaries #49482

This also fixes the issue where we should not shut down the server for missing dependencies. The logic for that is covered in the issue #46684

closes #46684

@jaredpar jaredpar requested a review from a team as a code owner November 18, 2020 22:49
@jaredpar
Copy link
Member Author

@dotnet/roslyn-infrastructure PTAL

Comment on lines 26 to 27
Copy link
Member

@Youssef1313 Youssef1313 Nov 19, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one was already fixed in #49370.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That change only fixed the field emit, it missed the parameters which have the same problem.

For the determinism test to be effective it should be using the
bootstrap compiler. That was the intent of this job but at some point
the bootstrap got disabled. This fixes that and excludes some binaries
that are now failing determinism.

Tracking issue for the failed binaries dotnet#49482
@sharwell
Copy link
Contributor

@AArnott this fixes the performance problem where the compiler server does not work for projects with your analyzers installed.

@AArnott
Copy link
Contributor

AArnott commented Nov 19, 2020

@sharwell I don't recall the problem you're referring to. Can you job my memory?

@jaredpar
Copy link
Member Author

@dotnet/roslyn-compiler PTAL. The logic for why we should not restart the server for missing dependencies is covered in #46684.

@sharwell
Copy link
Contributor

sharwell commented Nov 19, 2020

@AArnott the context is #46671. We had to include custom build logic because the vs-threading analyzers package was tanking build performance at no fault of its own (on my machine, builds went from ~2 minutes without the code fixes assembly to ~6 minutes with it, even though the assembly provided no build features). After this change, the code fixes assembly in that package will no longer negatively impact build throughput.

@jaredpar
Copy link
Member Author

@AArnott

Not sure if this applies to your analyzers but any analyzer DLL that also packaged a code fixer inside it effectively never used the compiler server. The compiler server is very sensitive about analyzer load errors because it has to deal with real problems around analyzers polluting the assembly load space of the process. That is because the server sees all of the analyzers for all compilations that go through it where as the command line csc just sees the analyzers for the current build.

Quick example of where this can cause a problem. Consider two projects (UtilA and UtilB) which depend on two versions of analyzer FindTheBugz which is not strong name signed. The compiler server can handle either of them just fine but not both. The first one to build loads their version of FindTheBugz one the second just gets the version the first loaded. The compiler server though detects this problem, redirects the build request to the command line compiler and restarts the server.

One of the load errors the server would shut down on was missing dependencies. That was written originally under the guise of "better safe than sorry". Over time though it's caused subsantial perf and usability problems. After some discussion decided this is actually an unnecessary restriction and this is lifting it.


$script:skipList = @(
# Added to work around https://github.com/dotnet/roslyn/issues/48417
"Microsoft.CodeAnalysis.EditorFeatures2.UnitTests.dll"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Microsoft.CodeAnalysis.EditorFeatures2.UnitTests.dll" [](start = 2, length = 54)

The fix for parameters got merged.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great. I'll either get this changed here or submit a follow up PR to remove this line.

{
private static readonly ImmutableArray<string> s_defaultIgnorableReferenceNames = ImmutableArray.Create("mscorlib", "System", "Microsoft.CodeAnalysis", "netstandard");

public static bool Check(string baseDirectory, IEnumerable<CommandLineAnalyzerReference> analyzerReferences, IAnalyzerAssemblyLoader loader, IEnumerable<string>? ignorableReferenceNames = null)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ignorableReferenceNames [](start = 170, length = 23)

Is this parameter still used?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No. Thought I had deleted that. Will fix.

Copy link
Contributor

@AlekseyTs AlekseyTs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (iteration 2), modulo a couple of comments.

@jaredpar jaredpar merged commit d46e72f into dotnet:master Nov 19, 2020
@ghost ghost added this to the Next milestone Nov 19, 2020
@jaredpar jaredpar deleted the det branch November 19, 2020 23:32
333fred added a commit to 333fred/roslyn that referenced this pull request Nov 20, 2020
* upstream/master: (47 commits)
  Make compiler server logging explicit (dotnet#48557)
  [master] Update dependencies from dotnet/arcade (dotnet#48274)
  Handle removed project in ExternalErrorDiagnosticUpdateSource
  Report an error for ref-returning auto-properties declared in an interface. (dotnet#49510)
  Add usings on paste (dotnet#48501)
  Determinism test needs to use bootstrap compiler (dotnet#49483)
  Simplify and reduce amount of work performed by SourcePropertySymbolBase constructor. (dotnet#49360)
  Updates test
  Simplify
  Split Document/Workspace handler into only handling open/closed documents respectively.
  only report watson once.
  Additional usage of a PooledHashset. (dotnet#49459)
  Loc checkin
  Update src/Features/CSharp/Portable/CodeRefactorings/ConvertLocalFunctionToMethod/CSharpConvertLocalFunctionToMethodCodeRefactoringProvider.cs
  Preserve annotation on comment trivia when performing formatting.
  Validate arguments to IAsyncCompletionSource methods
  Determinism fixes for AnonymousTypes in VB (dotnet#49467)
  Collect nfw information for a crash we're seeing.
  Make sure to not discard text changes when no reference changes are present
  Create an unsafe method from a local function when necessary (dotnet#49389)
  ...
@allisonchou allisonchou modified the milestones: Next, 16.9.P2 Nov 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Consider if compiler server should exit on missing analyzer dependency

6 participants