Skip to content

Remove IBuiltInAnalyzer.OpenFileOnly#26590

Closed
sharwell wants to merge 1 commit intodotnet:masterfrom
sharwell:fix-open-file-only
Closed

Remove IBuiltInAnalyzer.OpenFileOnly#26590
sharwell wants to merge 1 commit intodotnet:masterfrom
sharwell:fix-open-file-only

Conversation

@sharwell
Copy link
Copy Markdown
Contributor

@sharwell sharwell commented May 3, 2018

This internal method overlaps the behavior of DiagnosticSeverity.Hidden. The the few analyzers relying on it for correctness already reported Hidden diagnostics by default. These analyzers are updated to include NotConfigurable, ensuring that Hidden severity will be used for the diagnostics in all configurations.

Fixes #15760
Fixes #16026
Closes #16700

Ask Mode template not completed

Customer scenario

What does the customer do to get into this situation, and why do we think this
is common enough to address for this release. (Granted, sometimes this will be
obvious "Open project, VS crashes" but in general, I need to understand how
common a scenario is)

Bugs this fixes

(either VSO or GitHub links)

Workarounds, if any

Also, why we think they are insufficient for RC vs. RC2, RC3, or RTW

Risk

This is generally a measure our how central the affected code is to adjacent
scenarios and thus how likely your fix is to destabilize a broader area of code

Performance impact

(with a brief justification for that assessment (e.g. "Low perf impact because no extra allocations/no complexity changes" vs. "Low")

Is this a regression from a previous update?

Root cause analysis

How did we miss it? What tests are we adding to guard against it in the future?

How was the bug found?

(E.g. customer reported it vs. ad hoc testing)

Test documentation updated?

If this is a new non-compiler feature or a significant improvement to an existing feature, update https://github.com/dotnet/roslyn/wiki/Manual-Testing once you know which release it is targeting.

@sharwell sharwell requested a review from a team as a code owner May 3, 2018 15:57
@sharwell sharwell added this to the 15.8 milestone May 3, 2018
@jcouv jcouv added the Area-IDE label May 3, 2018
@sharwell sharwell force-pushed the fix-open-file-only branch 2 times, most recently from 768c4a9 to 59f5b85 Compare May 4, 2018 03:04
@davkean
Copy link
Copy Markdown
Member

davkean commented May 7, 2018

Help me understand the behavior change here?

@sharwell
Copy link
Copy Markdown
Contributor Author

sharwell commented May 7, 2018

When it returned true, the IBuiltInAnalyzer.OpenFileOnly method indicated that an analyzer would only run for files known to be open in the editor. This had two primary impacts:

  1. Closed files would not be analyzed (a possible performance improvement)
  2. The analyzer would only run inside the Visual Studio process

This pull request recognizes that an analyzer containing only diagnostics with severity Hidden also provides these guarantees. During review, I found that all analyzers returning OpenFileOnly fell into two categories:

  1. The analyzer only needed to be a Hidden diagnostic. For this case, the diagnostic is marked as NotConfigurable, meaning the default severity cannot be overridden by a user (e.g. via a rule set file). This configuration guarantees that the diagnostic continues to behave as though OpenFileOnly was provided and returned true.
  2. The analyzer should have been running on closed files, and failure to do so was causing usability bugs. For these analyzers, the previous limitation was lifted by no longer requiring the analyzer to run only on open files.

@CyrusNajmabadi
Copy link
Copy Markdown
Contributor

CyrusNajmabadi commented May 7, 2018

Note: with this change, the only purpose of IBuiltInAnalyzer is to support the 'GetAnalyzerCategory' call. It would be very good if we could remove that last call somehow (or make it public). It seems to serve a very important purpose for performance, but at hte same time is kept completely internal preventing 3rd party analyzers from benefitting from that perf entrypoint.

I reviewed our existing use of this API and even discovered a bunch of cases where we get it wrong. I opened up #26671 to address this.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this mean users can never enforce squiggles in the IDE for unnecessary usings? I presume the project system requires this behavior. They configure RemoveUnnecessaryImportsDiagnosticId to be an error over here: <Rule Id="IDE0005" Action="Error" /> <!-- Using directive is unnecessary using System.Text;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tagging @davkean too

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, why is this tied to this change?

This comment was marked as outdated.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I read that and still don't understand context. Assume I know nothing about how this works; why is it broken? It works in my limited testing.

Copy link
Copy Markdown
Contributor Author

@sharwell sharwell May 8, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mavasani @davkean IDE0005 needs two changes before it can be configured:

  1. Needs to avoid reporting the errors in generated code (but keep reporting a hidden diagnostic for fading unnecessary code)
  2. Needs to be able to run OOP (it may already, but this needs to be confirmed)

In the meantime the configuration will be silently ignored, and then when the diagnostic is properly enabled it will become active again.

@mavasani
Copy link
Copy Markdown
Contributor

mavasani commented May 8, 2018

@sharwell I like the intent of the PR, thanks for starting cleanup in this area. Finally we are working towards getting rid of this internal (hacky) interface!!

However, I am not sure about the NotConfigurable restriction being enforced on the IDE analyzer diagnostics that are hidden by default. Project system repo demonstrates a valid use case of showing squiggles for open file only diagnostics - they don't want to pollute the error list with unused usings errors for closed files, but do want a stronger visual indication of unused usings. This was one of the primary scenarios that forced us to not use severity to decide open file only behavior. @heejaechang can probably fill in if there were any additional user scenarios/perf implifications of coupling these two.

@davkean
Copy link
Copy Markdown
Member

davkean commented May 8, 2018

Project system repo demonstrates a valid use case of showing squiggles for open file only diagnostics - they don't want to pollute the error list with unused usings errors for closed files, but do want a stronger visual indication of unused usings.

No, that is not a correct representation of our position. I want the Error List to always show the correct state, I don't care if I have a file open or not. Showing errors only when a file is open is very confusing.

@mavasani
Copy link
Copy Markdown
Contributor

mavasani commented May 8, 2018

I want the Error List to always show the correct state, I don't care if I have a file open or not. Showing errors only when a file is open is very confusing.

Ah, sorry to have misinterpreted. Regardless, I feel the current change will mean unused usings can never be configured to be an error/warning. @sharwell Is it necessary to add these NotConfigurable tags to the IDE diagnostics?

@sharwell
Copy link
Copy Markdown
Contributor Author

sharwell commented May 8, 2018

@mavasani IDE0005 is impacted by this, but there is definitely intent to fix. I moved #22579 back into the 15.8 milestone to accompany this. There is one other scenario impacted:

For users with Full Solution Analysis enabled, the error list will now show Suggestion messages for diagnostics at severity Info for a subset of our diagnostics that enabled Open File Only mode for severities less than Warning (with Info being the only impacted one). This is arguably the correct behavior anyway, especially since it aligns with the behavior of all third-party analyzers.

@mavasani
Copy link
Copy Markdown
Contributor

mavasani commented May 8, 2018

@sharwell I am still slightly uncomfortable about tying 2 concepts here (severity and scope of execution) to one data point (severity). For example, if someone now bumps up the severity of any diagnostic ID reported by SimplifyTypeNameAnalyzer to be warning/error, we will start running them for closed files and it will make the IDE unusable, see this comment. There are certainly times when users desire a strong visual indication (squiggle) for certain issues, but don't really want to get the performance hit from closed file analysis and/or usability hit of polluting the error list.

I personally prefer keeping these two things separate, possibly by implementing #12713 or by introducing a new IDE specific API to DiagnosticAnalyzer type.

@sharwell
Copy link
Copy Markdown
Contributor Author

sharwell commented May 8, 2018

For example, if someone now bumps up the severity of any diagnostic ID reported by SimplifyTypeNameAnalyzer to be warning/error, we will start running them for closed files and it will make the IDE unusable, see this comment.

It already does this. In addition, the default severity of this analyzer is Hidden, so it will not be impacted by this change unless users have customized the severity specifically to Info and have FSA enabled.

I personally prefer keeping these two things separate, possibly by implementing #12713 or by introducing a new IDE specific API to DiagnosticAnalyzer type.

I'm not fundamentally opposed to expanding the API for this. However, I do not believe we are currently using IBuiltInAnalyzer.OpenFileOnly to an extent that would block this PR.

This internal method overlaps the behavior of DiagnosticSeverity.Hidden. The
the few analyzers relying on it for correctness already reported Hidden
diagnostics by default. These analyzers are updated to include NotConfigurable,
ensuring that Hidden severity will be used for the diagnostics in all
configurations.

Fixes dotnet#15760
Fixes dotnet#16026
Closes dotnet#16700
@heejaechang
Copy link
Copy Markdown
Contributor

well, I like things to be explicit since it is easy to check. when things are implicitly implied, things get fragile and require knowledge on implementation detail. for example, right now only closed file analysis is happening on OOP, so hidden has kind of same meaning of Open file only. but that's just how we implemented it and our current optimization.

unless this remove some issue, not sure why we want to do this.. especially if IBuildInAnalyzer is still there, and there are more issue we need to resolve to remove this.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

so at some point, we want to move open file only analyzers to OOP as well. but this one shouldn't move even at that point since it require rename service which will not exist in OOP side.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

➡️ This one will not run OOP even after this pull request.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How is that controlled? Thanks!

Copy link
Copy Markdown
Contributor Author

@sharwell sharwell May 10, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CyrusNajmabadi It's described in the initial comment for this pull request.

@sharwell sharwell force-pushed the fix-open-file-only branch from 59f5b85 to b2d9e9b Compare May 9, 2018 20:12
public override DiagnosticAnalyzerCategory GetAnalyzerCategory() => DiagnosticAnalyzerCategory.SemanticSpanAnalysis;

public override bool OpenFileOnly(Workspace workspace)
{
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code style analyzer is weird one I believe, since engine is specifically rewritten to ignore code style analyzer's descriptor. it is special ability grant to only IDE code style analyzer which we want to fix at some point. since they return dynamic descriptor that are not allowed to any other analyzers.

basically they can return diagnostic that is not from descriptor they return.

so, some of these analyzer used to not run on OOP when this return true. will all these run on OOP now? or never. I think code style analyzer always return fake hidden descriptor? (since it doesn't matter for them)


public bool OpenFileOnly(Workspace workspace)
{
var preferTypeKeywordInDeclarationOption = workspace.Options.GetOption(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

all these analyzers are special. can't I think their descriptor has no meaning.


public bool OpenFileOnly(Workspace workspace)
{
var preferTypeKeywordInDeclarationOption = workspace.Options.GetOption(
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

same here. not sure what the reason was that IDE refuse to use regular diagnostic system but use this system. @mavasani do you remember? was it due to naming convention analyzer? which they say they don't know what diagnostic they have either since user can configure those?

var existingData = await ProjectAnalysisData.CreateAsync(project, stateSets, avoidLoadingData, cancellationToken).ConfigureAwait(false);

// we can't return here if we have open file only analyzers sine saved data for open file only analyzer
// is wrong. (since it only contains info on open files rather than whole project)
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you should do something here I believe. otherwise, fix all on open file only analyzer will be broken. not sure whether there is a test cover this or not though.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see why you removed it here now.

{
if (existing.TryGetValue(analyzer, out var analysisResult) &&
analysisResult.Version == version &&
!analyzer.IsOpenFileOnly(project.Solution.Workspace))
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmmm.. we might filter out all hidden analyzer for closed file one, so this basically assume it will never be here I guess.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we might filter out all hidden analyzer for closed file one

No, hidden diagnostics work for Fix All even if the documents are closed.

}

// due to OpenFileOnly analyzer, we need to run inproc as well for such analyzers
var inProcResultTask = AnalyzeInProcAsync(CreateAnalyzerDriver(analyzerDriver, a => a.IsOpenFileOnly(project.Solution.Workspace)), project, remoteHostClient, cancellationToken);
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I doubt this is okay due to style analyzers. you want this, probably need to fix code style analyzers to behave same as regular analyzers first.

@heejaechang
Copy link
Copy Markdown
Contributor

heejaechang commented May 9, 2018

what is driving force of this PR? having issue with OpenFiles or just clean up? I think if It wants to remove OpenFiles, it should first fix the reason it was added at the first place.

it should first make all IDE analyzers to behave same as third party analyzers that don't require information that third party analyzer can't access. (which we wanted but didn't have time to do so)

if I am remembering correctly.

first is options, (which was one of reason we started editorconfig in compiler)
second is analyzer returning diagnostics with diagnostics descriptor analyzer doesn't report as supported (even severity that is not from editor config or rule set file)
last is IDE internal service (static service I believe) that OOP doesn't have.

@sharwell
Copy link
Copy Markdown
Contributor Author

sharwell commented May 10, 2018

first is options, (which was one of reason we started editorconfig in compiler)

This is broken before and after the change in this PR (i.e. this PR doesn't change behavior for this).

second is analyzer returning diagnostics with diagnostics descriptor analyzer doesn't report as supported (even severity that is not from editor config or rule set file)

We already run these analyzers OOP, so I'm assuming they work in this scenario. I can do some more manual validation if it would help.

last is IDE internal service (static service I believe) that OOP doesn't have.

The only two analyzers potentially impacted are unnecessary imports and the rename tracking analyzers. This PR forces both of these to run in process so they will not break.

@CyrusNajmabadi
Copy link
Copy Markdown
Contributor

CyrusNajmabadi commented May 10, 2018

The only two analyzers potentially impacted are unnecessary imports

What IDE services does this need?

N/M. I see now. It accesses the workspace to find all linked documents to decide what is ok to remove or not.

@sharwell
Copy link
Copy Markdown
Contributor Author

What IDE services does this need?

I don't know of any, but I haven't conclusively ruled it out either.

@sharwell
Copy link
Copy Markdown
Contributor Author

Closing this for now; will file an issue to revisit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

6 participants