Skip to content

Support negative null-checks when we offer a fix to use pattern-matching#20246

Merged
sharwell merged 1 commit intodotnet:dev15.7.xfrom
alrz:smart-as-p3
Jan 31, 2018
Merged

Support negative null-checks when we offer a fix to use pattern-matching#20246
sharwell merged 1 commit intodotnet:dev15.7.xfrom
alrz:smart-as-p3

Conversation

@alrz
Copy link
Member

@alrz alrz commented Jun 15, 2017

@alrz
Copy link
Member Author

alrz commented Jun 27, 2017

@Pilchie this is actually IDE related. same for #20240, all of which blocked by #19907 awaiting your review.

@Pilchie
Copy link
Member

Pilchie commented Jun 27, 2017

Thanks and sorry!

Tagging the rest of @dotnet/roslyn-ide for review too.

@Pilchie Pilchie added the Community The pull request was submitted by a contributor who is not a Microsoft employee. label Jun 27, 2017
@alrz
Copy link
Member Author

alrz commented Jun 28, 2017

Please hold off the review untill the linked PR goes in. Thanks.

@shyamnamboodiripad shyamnamboodiripad changed the base branch from dev15.6 to master June 29, 2017 22:14
@alrz alrz changed the title Support negative null-checks when we offer a fix to use pattern-matching [WIP] Support negative null-checks when we offer a fix to use pattern-matching Jul 26, 2017
@sharwell sharwell added Community The pull request was submitted by a contributor who is not a Microsoft employee. and removed Community The pull request was submitted by a contributor who is not a Microsoft employee. labels Aug 31, 2017
@Pilchie Pilchie requested a review from a team September 8, 2017 15:08
@alrz alrz force-pushed the smart-as-p3 branch 2 times, most recently from 71c0bcf to d56b078 Compare November 17, 2017 22:20
@alrz alrz changed the title [WIP] Support negative null-checks when we offer a fix to use pattern-matching Support negative null-checks when we offer a fix to use pattern-matching Nov 17, 2017
@alrz
Copy link
Member Author

alrz commented Nov 17, 2017

@sharwell this is green, ready for review. thanks.

@alrz
Copy link
Member Author

alrz commented Nov 22, 2017

@sharwell any chance we can have this for 15.6?

Copy link
Contributor

@sharwell sharwell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly questions

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❗️ (Unless not possible) it would be good to retain the original if statement as well as the new one.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just changed this test to cover negative case.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

➡️ Sounds good, thanks

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ Is the ability to handle a cast to object new for both the normal and negated forms? If so, we should add a test for the newly supported normal form.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, it's new, I'll add the test for (object)x != null as well.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💭 This will rule out the feature for many types. We may be able to improve the experience by allowing user-defined operators, but ensuring a single Fix All operation only corrects operations that use the same operator. For example:

  • If all uses of == null and != null for a variable use the default operator, a Fix All on the variable will fix all cases where no user-defined operators are involved
  • If one or more uses of == or != involve a user-defined operator for a variable with the declared type T, then a Fix All on that variable will fix all instances involving variables with declared type T, whether or not they use user-defined operators. Alternately, this scenario could fix all instances involving variables with declared type T which use one or more user-defined operators.

The equivalence ID of the code fix can be used to "bucket" fixes according to the above conditions.

If you like the idea but aren't ready to implement it, feel free to leave this as a follow-up issue by filing a new bug referencing this comment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not part of this PR, the only diff here is notEquals -> comparison

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed #23374

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ To help me get my head around this, can you provide an example of why this block is necessary?

Copy link
Member Author

@alrz alrz Nov 22, 2017

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (!(x is Foo  foo))  {} 
// foo is always definitely assigned (we don't need to check so we return early)
while (!(x is Foo  foo)) {}
// foo is not in scope (so we need to check if it's accessed)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can you put this in as a comment.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Wait, this is only correct if if has an unreachable endpoint. I'll remove this check.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💭 This seems like an unnecessary change

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Relates to #21962

I can revert it if you want.

@alrz
Copy link
Member Author

alrz commented Nov 24, 2017

Also added fix for #21097

@sharwell Do I need to open another PR for that?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is it worth to use Task.WhenAll here?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope. :)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be a candidate for using Theory instead of Fact. With that said, Theory is used sparingly (very rarely currently) and not everyone has agreed that it's a good idea. I'm only showing the difference here as a comparison. The one advantage I've found is a test failure shows more information about the specific failing case, as opposed to just a line number in a method with many tests.

[Theory, Trait(Traits.Feature, Traits.Features.CodeActionsInlineTypeCheck)]
[InlineData("x != null", "o is string x")]
[InlineData("null != x", "o is string x")]
[InlineData("(object)x != null", "o is string x")]
[InlineData("null != (object)x", "o is string x")]
// Other test cases added as additional attributes...
public async Task InlineTypeCheck1(string input, string output)
{
  await TestStatement($"if ({input}) {{ }}", $"if ({output}) {{ }}");
  await TestStatement($"var y = {input};", $"var y = {output};");
  await TestStatement($"return {input};", $"return {output};");
}

@alrz alrz changed the title Support negative null-checks when we offer a fix to use pattern-matching [WIP] Support negative null-checks when we offer a fix to use pattern-matching Jan 18, 2018
@alrz alrz changed the title [WIP] Support negative null-checks when we offer a fix to use pattern-matching Support negative null-checks when we offer a fix to use pattern-matching Jan 18, 2018
Copy link
Contributor

@sharwell sharwell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was hesitant to start with due to test churn, but now that I've been through it in more detail I think this is a great change.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be a candidate for using Theory instead of Fact. With that said, Theory is used sparingly (very rarely currently) and not everyone has agreed that it's a good idea. I'm only showing the difference here as a comparison. The one advantage I've found is a test failure shows more information about the specific failing case, as opposed to just a line number in a method with many tests.

[Theory, Trait(Traits.Feature, Traits.Features.CodeActionsInlineTypeCheck)]
[InlineData("x != null", "o is string x")]
[InlineData("null != x", "o is string x")]
[InlineData("(object)x != null", "o is string x")]
[InlineData("null != (object)x", "o is string x")]
// Other test cases added as additional attributes...
public async Task InlineTypeCheck1(string input, string output)
{
  await TestStatement($"if ({input}) {{ }}", $"if ({output}) {{ }}");
  await TestStatement($"var y = {input};", $"var y = {output};");
  await TestStatement($"return {input};", $"return {output};");
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 This test case covers the original InlineTypeCheck1.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 This test case covers the original InlineTypeCheckInvertedCheck1.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 This test case replaces the original TestMissingOnNullEquality, showing new functionality.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 This test case covers the original TestInlineNullCheck1.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❗️ Use an explicitly-typed variable rather than a cast. This ensures a compilation error occurs for any case where an implicit cast is not available (which in theory ensures the operation is a widening conversion, though we have cases even in this codebase where that is not the case).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 IsKind

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I use Kind() where the code asserts that the variable is never null i.e the path is unreachable if it was. This is also the convention used on the compiler side.

Copy link
Member Author

@alrz alrz Jan 18, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suppose on the IDE side we traditionally use IsKind to not throw if anything ever failed.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 definitely

💡 I find this block pleasantly readable, but it may be because I have context from seeing the recent bug. Consider expanding this comment by adding the following:

// For example:
//
// if ((x = o as string) != null || SomeExpression) {
//   // x would not be definitely assigned here if pattern matching were used
// }

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 A comment regarding the return value would be handy

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Assuming my understanding is correct, the following might be good to add.

// For example:
//
// if (!(o is string x)) {
//   return;
// }
//
// // The 'return' statement above ensures x is definitely assigned here
// Console.WriteLine(x);

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done.

@sharwell
Copy link
Contributor

sharwell commented Jan 18, 2018

Requesting a review from @agocke to verify the use of data flow analysis.

Moving back into to 15.6. @jinujoseph This needs to be rebased if we bring it into 15.6. I can do this easily, but I only want to if it will pass ask mode bar. Overall I think this is a great change that expands the capability of one of our regularly-used features.

@sharwell sharwell requested a review from agocke January 18, 2018 16:58
@sharwell sharwell changed the base branch from master to dev15.6.x January 18, 2018 16:59
@sharwell sharwell changed the base branch from dev15.6.x to master January 18, 2018 16:59
@CyrusNajmabadi
Copy link
Contributor

This would be a candidate for using Theory instead of Fact. With that said, Theory is used sparingly (very rarely currently) and not everyone has agreed that it's a good idea.

I'm usually one how pushes back on Theories. That said, if things stay very simple, and it's clear precisely what the theory is testing/expecting, then i'm totally ok with it. Where i usually don't like them is where people parameterize 6 places in a test and it becomes totally unclear waht's actually happening :)

In this case, theories look fine to me. If we use them judiciously and keep clarity high, then i'm all for them :)

Copy link
Member Author

@alrz alrz Jan 19, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a bug here, should be missing but it's not:

    void M()
    {
        [|var|] x = o as string;
        if (x != null)
        {
        }
        Console.WriteLine(x);
        x = "";
    }

x is always assigned but there is a read before assignment - probably need to revert code to iterate over statements one by one, @CyrusNajmabadi, can you confirm if that was the original purpose of doing so instead of region analysis?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is a bug here, should be missing but it's not:

It's missing for me. I'm not able to see this locally.

Copy link
Member Author

@alrz alrz Jan 19, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: iterating over statements is not safe because assignments could be anywhere in an expression,

Console.WriteLine(x = Read(x));

Copy link
Member Author

@alrz alrz Jan 19, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CyrusNajmabadi I meant in this branch. The previous revision check for nodes statement by statement so doesn't have this bug.

However, with either approaches, this one is not missing.

    string Use(string x) => x;
    void M()
    {
        var x = o as string;
        if (x != null)
        {
        }
        Console.WriteLine(x = Use(x));
    }

From data flow analysis results, we can't see if the read is happened before write..

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@CyrusNajmabadi, can you confirm if that was the original purpose of doing so instead of region analysis?

Yes, i believe that was the intent.

Note: iterating over statements is not safe because assignments could be anywhere in an expression,

I thought we checked assignment on a per-reference basis. So we would specifically check the Use(x) and see the problem. Right?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we return early in the previous code. we see the write first and bail. I'll try a mixed approach, if we are AlwaysAssgined, check if there is a read, if yes, need to walk through tree to see it doesn't come first. early return here could be safe.

Copy link
Contributor

@CyrusNajmabadi CyrusNajmabadi Jan 19, 2018

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, i think i see what you're saying. The logic here is buggy:

private void CheckDefiniteAssignment(
SemanticModel semanticModel, ISymbol localVariable, SyntaxNode node,
out bool isAssigned, out bool isAccessedBeforeAssignment,
CancellationToken cancellationToken)
{
if (node != null)
{
foreach (var id in node.DescendantNodes().OfType<IdentifierNameSyntax>())
{
var symbol = semanticModel.GetSymbolInfo(id, cancellationToken).GetAnySymbol();
if (localVariable.Equals(symbol))
{
isAssigned = id.IsOnlyWrittenTo();
isAccessedBeforeAssignment = !isAssigned;

Specifically, because it sees the x =, it assumes that 'x' is always assigned, and thus the change is safe. But it's not. Because the right side of the assignment executes first, and will do a read of the unassigned variable.

  1. That sucks. But it's also a very corner case.
  2. We could consider fixing it. Namely that if the reference is the left of an assign, that we have to ensure that the right side is fine as well. Annoying, but doable.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: using AlwaysAssigned is necessary to handle branches e.g. #21097

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see. The logic here has changed a bit, and i'll admit i don't really fully understand the algorithm as it stands now, so i'm not sure the exact right fix. But i do think this can be solved.

Also, an efficiency point:

                    var symbol = semanticModel.GetSymbolInfo(id, cancellationToken).GetAnySymbol();
                    if (localVariable.Equals(symbol))
                    {

We should check if the id.ValueText matches the local variable name before bothering to do the binding. That will avoid a lot of expensive semantic checks.

@CyrusNajmabadi
Copy link
Contributor

Note: using AlwaysAssigned is necessary to handle branches e.g. #21097

BTW, something to consider:

Right now the code is very concept/information dense. I.e. it's trying to figure out all the cases and encode it all into these variables that it's working with. You should always feel free to consider decomposing this into the major cases that need to be answered. i.e.:

if (ThereIsReadBeforeAssignmentInFollowingStatements(...) ||
    ThereIsSomethingWithTheElseClause(...) ||
    ThereIsSomeOtherCaseThatsAlsoImportant(...))
{
    return;
}

Then make each of those approaches clear as to what tehy're checking for and how they do it.

I'm not saying this will improve things. But it's something to consider when code starts getting very dense and difficult to understand because it's trying to handle so many cases.

@CyrusNajmabadi
Copy link
Contributor

BTW, is there a reason we don't just do:

  1. Look for cheap and obvious things that would cause a def assignment error. If we find any, bail immediately.
  2. If we don't find anything, then make the actual change, and actually check the compiler to see if it reported any def assignment issues?

I thought we already did this approach for some fix providers. It means not actually having to try to reimplement all the compiler's flow control logic into the IDE.

@CyrusNajmabadi
Copy link
Contributor

CyrusNajmabadi commented Jan 19, 2018

We already have CSharpInlineDeclarationDiagnosticAnalyzer, which has:

private bool WouldCauseDefiniteAssignmentErrors(
SemanticModel semanticModel,
VariableDeclarationSyntax localDeclaration,
VariableDeclaratorSyntax localDeclarator,
BlockSyntax enclosingBlock,
ILocalSymbol outLocalSymbol,
CancellationToken cancellationToken)

This is essentially the exact same problem. We're "inlining" the variable declaration into the pattern. We should just extract out a helper here that can be used for both analyzers. Now, this check is expensive. So ideally we'd just do some quicker checks first (like what existed in V1 of this analyzer). But if those quick checks don't see a problem, we do the expensive work.

@alrz
Copy link
Member Author

alrz commented Jan 23, 2018

The diff doesn't help much as I rewrote the def assignment check part. Please review the final file. Thanks.

@alrz
Copy link
Member Author

alrz commented Jan 24, 2018

tagging @CyrusNajmabadi

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it woudl be helpful to have examples shown in the code of what this is catching.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it woudl be helpful to have examples shown in the code of what this is catching.

@alrz
Copy link
Member Author

alrz commented Jan 24, 2018

@dotnet-bot test windows_release_vs-integration_prtest please

@alrz
Copy link
Member Author

alrz commented Jan 25, 2018

tagging @sharwell, this is ready to go in with 2 approved reviews.

@jinujoseph
Copy link
Contributor

@sharwell lets take this for 15.7
@Pilchie for ask mode barcheck before retargetting .

@Pilchie Pilchie added this to the 15.7 milestone Jan 31, 2018
@Pilchie
Copy link
Member

Pilchie commented Jan 31, 2018

Approved for 15.7 - sorry for the delay. Note that it will need to be re-targeted to the dev15.7.x branch.

@sharwell
Copy link
Contributor

@alrz This needs to be rebased on the dev15.7.x branch. Please let me know if you would like me to do the rebase or if you would like to do it yourself. 😄

@alrz alrz changed the base branch from master to dev15.7.x January 31, 2018 16:42
@alrz
Copy link
Member Author

alrz commented Jan 31, 2018

will rebase when I get home

@alrz
Copy link
Member Author

alrz commented Jan 31, 2018

Had to squash commits, it got muddled up in the middle of rebase

@sharwell sharwell merged commit 4521f66 into dotnet:dev15.7.x Jan 31, 2018
@alrz alrz deleted the smart-as-p3 branch January 31, 2018 20:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Approved to merge Area-IDE cla-already-signed Community The pull request was submitted by a contributor who is not a Microsoft employee.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants