Nullability Rewriter Step#34017
Conversation
…notation and NullableFlowState, and update the the rewriter to use that instead of the Nullability enum.
…e walker and rewriting them.
|
The bootstrap build is failing, but that's not due to anything that will cause massive changes during review as I work on fixing the issue, so I'll ask for reviews from @dotnet/roslyn-compiler while I start working on them. #Closed |
|
|
||
| // PROTOTYPE(nullable-api): Doc Comment | ||
| public Nullability Nullability { get; } | ||
| public NullabilityInfo NullabilityInfo { get; } |
There was a problem hiding this comment.
Info [](start = 42, length = 4)
Consider dropping Info suffix from field names. #Resolved
| && object.Equals(this.Nullability, other.Nullability) | ||
| && object.Equals(this.ConvertedNullability, other.ConvertedNullability); | ||
| && object.Equals(this.NullabilityInfo, other.NullabilityInfo) | ||
| && object.Equals(this.ConvertedNullabilityInfo, other.ConvertedNullabilityInfo); |
There was a problem hiding this comment.
NullabilityInfo should implement IEquatable<NullabilityInfo> and the implementation should be used here to avoid boxing. #Resolved
| } | ||
|
|
||
| /// <summary> | ||
| /// The nullable state of an rvalue computed in NullableWalker. |
There was a problem hiding this comment.
NullableWalker [](start = 52, length = 14)
Doc comment should not refer to internal types. #Resolved
| namespace Microsoft.CodeAnalysis.CSharp.Symbols | ||
| { | ||
| internal enum NullableAnnotation : byte | ||
| internal enum CSharpNullableAnnotation : byte |
There was a problem hiding this comment.
CSharpNullableAnnotation [](start = 18, length = 24)
Please remove the CSharp prefix. #Resolved
| } | ||
|
|
||
| public new Nullability TopLevelNullability | ||
| public virtual new NullabilityInfo TopLevelNullabilityInfo |
There was a problem hiding this comment.
TopLevelNullabilityInfo [](start = 43, length = 23)
Consider dropping the Info suffix. #Resolved
| // Bit 3: Whether nullability has been set | ||
| // Bit 4: 1 if the node is nullable, 0 if the node is not nullable | ||
| // Bits 5 and 6: 01 if the node is not annotated, 10 if the node is annotated, 00 if the node is disabled | ||
| TopLevelNullableInfoSet = 1 << 3, |
There was a problem hiding this comment.
InfoSet [](start = 28, length = 7)
InfoSet prefix seems unnecessary. #Resolved
|
|
||
| case BoundNodeAttributes.TopLevelUnknown: | ||
| return Nullability.Unknown; | ||
| var annotation = (_attributes & BoundNodeAttributes.TopLevelAnnotationMask) switch |
There was a problem hiding this comment.
@jaredpar here's a switch expression for you :). #Closed
| NotAnnotated, // Type is not annotated - string, int, T (including the case when T is unconstrained). | ||
| Annotated, // Type is annotated - string?, T? where T : class; and for int?, T? where T : struct. | ||
| NotNullable, // Explicitly set by flow analysis | ||
| Nullable, // Explicitly set by flow analysis |
There was a problem hiding this comment.
Do we need these two values? #Closed
There was a problem hiding this comment.
Neal is removing them in the master. The next merge will remove them.
In reply to: 264906411 [](ancestors = 264906411)
|
|
||
| // PROTOTYPE(nullable-api): Document | ||
| Nullability ReceiverNullability { get; } | ||
| NullableAnnotation ReceiverNullableAnnotation { get; } |
There was a problem hiding this comment.
ReceiverNullableAnnotation [](start = 27, length = 26)
What's a scenario where receiver could be annotated? #Closed
There was a problem hiding this comment.
| Microsoft.CodeAnalysis.NullableAnnotation.Annotated = 3 -> Microsoft.CodeAnalysis.NullableAnnotation | ||
| Microsoft.CodeAnalysis.NullableAnnotation.Default = 0 -> Microsoft.CodeAnalysis.NullableAnnotation | ||
| Microsoft.CodeAnalysis.NullableAnnotation.NotAnnotated = 2 -> Microsoft.CodeAnalysis.NullableAnnotation | ||
| Microsoft.CodeAnalysis.NullableAnnotation.NotNullable = 4 -> Microsoft.CodeAnalysis.NullableAnnotation |
There was a problem hiding this comment.
NotNullable [](start = 42, length = 11)
We should probably remove Nullable/NotNullable from the annotations enum at this point. #Closed
There was a problem hiding this comment.
Neal has done this in master. When I merge that in, these will go away.
In reply to: 264935207 [](ancestors = 264935207)
There was a problem hiding this comment.
Neal changed the internal enum (Microsoft.CodeAnalysis.CSharp.Symbols.NullableAnnotation), not the public one which you're creating.
Is this PR negatively affected if it's doesn't add public Nullable/NotNulable members?
In reply to: 265363390 [](ancestors = 265363390,264935207)
There was a problem hiding this comment.
I don't want to have to add logic to it to correctly handle converting Nullable to Annotated and NotNullable to NotAnnotated.
In reply to: 265676688 [](ancestors = 265676688,265363390,264935207)
| public readonly struct TypeInfo : IEquatable<TypeInfo> | ||
| { | ||
| internal static readonly TypeInfo None = new TypeInfo(null, null, Nullability.NotComputed, Nullability.NotComputed); | ||
| internal static readonly TypeInfo None = new TypeInfo(null, null, default, default); |
There was a problem hiding this comment.
default [](start = 74, length = 7)
nit: consider naming the arguments #Closed
| bool HasReferenceTypeConstraint { get; } | ||
|
|
||
| Nullability ReferenceTypeConstraintNullability { get; } | ||
| NullableAnnotation ReferenceTypeConstraintNullableAnnotation { get; } |
There was a problem hiding this comment.
ReferenceTypeConstraintNullableAnnotation [](start = 27, length = 41)
// PROTOTYPE(nullable-api): Document #Closed
| public override int GetHashCode() => | ||
| Hash.Combine(Annotation.GetHashCode(), FlowState.GetHashCode()); | ||
|
|
||
| } |
There was a problem hiding this comment.
} [](start = 4, length = 1)
nit: unnecessary empty line above #Closed
| IsSuppressed = 1 << 2, | ||
|
|
||
| // Bit 3: Whether nullability has been set | ||
| // Bit 4: 1 if the node is nullable, 0 if the node is not nullable |
There was a problem hiding this comment.
// Bit 4: 1 if the node is nullable, 0 if the node is not nullable [](start = 12, length = 66)
nit: Perhaps "// Bit 4: 1 if the node has maybe-null state, 0 if the node is not-null" #Closed
| TopLevelFlowStateNullable = 1 << 4, | ||
| TopLevelNotAnnotated = 1 << 5, | ||
| TopLevelAnnotated = 1 << 6, | ||
| TopLevelDisabled = 0, |
There was a problem hiding this comment.
TopLevelDisabled = 0, [](start = 12, length = 21)
Do we need this value? #Closed
| // Bit 3: Whether nullability has been set | ||
| // Bit 4: 1 if the node is nullable, 0 if the node is not nullable | ||
| // Bits 5 and 6: 01 if the node is not annotated, 10 if the node is annotated, 00 if the node is disabled | ||
| TopLevelNullableInfoSet = 1 << 3, |
There was a problem hiding this comment.
TopLevelNullableInfoSet [](start = 12, length = 23)
I don't think we need this extra bit to know if nullability has been set. You can use the two later bits to encode that: 00 is unset, 01 is not-annotated, 10 is annotated, and 11 is disabled. #Closed
| get { return this.ConstantValueOpt; } | ||
| } | ||
|
|
||
| public override NullabilityInfo TopLevelNullabilityInfo |
There was a problem hiding this comment.
TopLevelNullabilityInfo [](start = 40, length = 23)
Can we remove the override and seal the definition in BoundNode?
Can we remove BoundExpression.TopLevelNullabilityInfo and overrides? The implementation in BoundNode should be sufficient. #Resolved
There was a problem hiding this comment.
This is working around an issue that Neal has fixed in master that I haven't checked to ensure is resolved in my branch yet: you couldn't have a TSWA with a null symbol that wasn't considered Unknown. That may be fixed, but I haven't checked it yet.
As to the BoundNode vs BoundExpression, I specifically did this because only things that derive from BoundExpression should be given a nullability. The info needs to be stored in the BoundNode because that's where the flags live, but it shouldn't be possible to set nullability on anything that isn't a BoundExpression.
In reply to: 264938565 [](ancestors = 264938565)
There was a problem hiding this comment.
Please add a PROTOTYPE comment to make the BoundExpression implementation non-virtual.
In reply to: 265247636 [](ancestors = 265247636,264938565)
There was a problem hiding this comment.
nit: comment could be more specific to help remind to remove this property
In reply to: 265667308 [](ancestors = 265667308,265247636,264938565)
There was a problem hiding this comment.
There's a comment on BoundExpression to make the property non-virtual.
In reply to: 265805034 [](ancestors = 265805034,265667308,265247636,264938565)
| { | ||
| internal static class PublicNullableFlowStateExtensions | ||
| { | ||
| public static NullableFlowState ToInternalFlowState(this CodeAnalysis.NullableFlowState flowState) |
There was a problem hiding this comment.
ToInternalFlowState [](start = 40, length = 19)
Is this necessary? When do we need to convert from the public values to internal values? #Resolved
There was a problem hiding this comment.
Currently, it's being used in the SymbolDisplay visitor and the VerifyTypes helpers. I'll work on removing them in a later PR if it's possible, but it will require removing TypeWithState and TypeSymbolWithAnnotations from a few places so I'll hold off on that for now.
In reply to: 265784942 [](ancestors = 265784942)
| public enum NullableAnnotation : byte | ||
| { | ||
| NotApplicable = 0, | ||
| Disabled, // No information. Think oblivious. |
There was a problem hiding this comment.
Disabled [](start = 8, length = 8)
Disabled seems misleading because it seems to imply that annotations were ignored rather than the more likely scenario that annotations were never included. I think this should be None or Unset. #Resolved
There was a problem hiding this comment.
Noted. These APIs will need to go through a formal design review, so I'll bring up your concerns there. This is what the current plan of record has, so it's what I'm going with for now.
In reply to: 266024732 [](ancestors = 266024732)
| /// Sets the analyzed nullability of the expression to be the given result. | ||
| /// </summary> | ||
| private void SetAnalyzedNullability(BoundExpression expr, VisitResult result, bool? isLvalue = null) | ||
| { |
There was a problem hiding this comment.
{ [](start = 8, length = 1)
We should assert the type matches the expected type. (See VisitExpressionWithoutStackGuard.)
Debug.Assert(AreCloseEnough(result.RValueType.Type, expr.Type);
``` #Resolved
There was a problem hiding this comment.
There was a problem hiding this comment.
Would AreCloseEnough() allow enabling the assertion now?
In reply to: 266121801 [](ancestors = 266121801,266096745)
There was a problem hiding this comment.
Unfortunately not currently: how we handle conversions means that this api gets called with the wrong node.
In reply to: 266126506 [](ancestors = 266126506,266121801,266096745)
| NotAnnotated, // Type is not annotated - string, int, T (including the case when T is unconstrained). | ||
| Annotated, // Type is annotated - string?, T? where T : class; and for int?, T? where T : struct. | ||
| NotNullable, // Explicitly set by flow analysis | ||
| Nullable, // Explicitly set by flow analysis |
There was a problem hiding this comment.
Can these be removed? #Resolved
There was a problem hiding this comment.
Not yet. I'll remove them when I do the merge from master.
In reply to: 266097888 [](ancestors = 266097888)
| TopLevelNullabilityMask = TopLevelUnknown, | ||
| IsSuppressed = 1 << 2, | ||
|
|
||
| // Bit 3: 1 if the node is has maybe-null state, 0 if the node is not null |
There was a problem hiding this comment.
is has [](start = 36, length = 6)
Typo? #Closed
| public VisitResult WithType(TypeSymbol newType) => new VisitResult(new TypeWithState(newType, RValueType.State), TypeSymbolWithAnnotations.Create(newType, LValueType.NullableAnnotation)); | ||
| private string GetDebuggerDisplay() => $"{{LValue: {LValueType.GetDebuggerDisplay()}, RValue: {RValueType.GetDebuggerDisplay()}}}"; | ||
|
|
||
| public NullabilityInfo ToNullabilityInfo() => new NullabilityInfo(LValueType.NullableAnnotation.ToPublicAnnotation(), RValueType.State.ToPublicFlowState()); |
There was a problem hiding this comment.
ToNullabilityInfo [](start = 35, length = 17)
Consider inlining in the caller. #Closed
| //if (this.State.Reachable) // Consider reachability: see https://github.com/dotnet/roslyn/issues/28798 | ||
| { | ||
| // Because we use an iterative algorithm for binary operators to avoid blowing the stack, we need to ensure | ||
| // that _currentExpression is correct. |
There was a problem hiding this comment.
that _currentExpression is correct [](start = 19, length = 34)
The comment appears to be stale. Can the entire comment be removed? #Closed
|
|
||
| // For nested binary operators, this can be the only time they're visited due to explicit stack used in AbstractFlowPass.VisitBinaryOperator, | ||
| // so we need to set the flow-analyzed type here. | ||
| SetAnalyzedNullability(binary, _visitResult); |
There was a problem hiding this comment.
SetAnalyzedNullability(binary, _visitResult); [](start = 16, length = 45)
Is _visitResult different than the return value of InferResultNullability() above? If not, this call can be removed and replaced with updateAnalyzedNullability: true above. #Closed
| } | ||
| } | ||
|
|
||
| } |
There was a problem hiding this comment.
Please remove blank line. #Closed
| ImmutableArray<RefKind> refKindsOpt, | ||
| ImmutableArray<ParameterSymbol> parameters, | ||
| ImmutableArray<int> argsToParamsOpt, | ||
| ImmutableArray<BoundExpression> argumentsWithConversions, |
There was a problem hiding this comment.
ImmutableArray argumentsWithConversions [](start = 12, length = 56)
Not used. #Closed
| if (!_disableNullabilityAnalysis) | ||
| { | ||
| var bag = new DiagnosticBag(); | ||
| Analyze(compilation, node, bag, node.Type.GetDelegateType()?.DelegateInvokeMethod, returnTypes: null, initialState: GetVariableState(), _analyzedNullabilityMapOpt); |
There was a problem hiding this comment.
Analyze [](start = 16, length = 7)
Are we visiting the lambda body twice in non-error scenarios? #Resolved
There was a problem hiding this comment.
Yes. The prototype comment above is tracking trying to remove this.
In reply to: 266115689 [](ancestors = 266115689)
| @@ -1260,7 +1527,7 @@ protected override BoundExpression VisitExpressionWithoutStackGuard(BoundExpress | |||
| { | |||
There was a problem hiding this comment.
{ [](start = 8, length = 1)
Please assert _analyzedNullabilityMapOpt does not contain an entry for node to ensure we are not visiting expressions multiple times. #WontFix
There was a problem hiding this comment.
We can't do this: currently, we're explicitly visiting arguments twice.
In reply to: 266115954 [](ancestors = 266115954)
| } | ||
|
|
||
| // VisitTypeExpression is called recursively, so we need to manually set the previous expression before | ||
| // updating the result; |
There was a problem hiding this comment.
Can this comment be removed? #Closed
| { | ||
| var verifier = new DebugVerifier(analyzedNullabilityMap); | ||
| verifier.Visit(node); | ||
| // Can't just remove nodes from _analyzedNullabilityMap and verify no nodes remaining because nodes can be reused. |
There was a problem hiding this comment.
// Can't just remove nodes from _analyzedNullabilityMap and verify no nodes remaining because nodes can be reused. [](start = 16, length = 114)
The comment appears to be stale. Can it be removed? #ByDesign
There was a problem hiding this comment.
I suppose it's more explaining why I didn't just make _analyzedNullabilityMap mutable and remove nodes from it as I visited them, and am instead just doing a count comparison.
In reply to: 266118053 [](ancestors = 266118053)
| #region ITypeParameterTypeSymbol Members | ||
|
|
||
| Nullability ITypeParameterSymbol.ReferenceTypeConstraintNullability | ||
| // PROTOTYPE(nullable-api): Document |
There was a problem hiding this comment.
// PROTOTYPE(nullable-api): Document [](start = 8, length = 36)
Not needed. The documentation belongs on the interface member. #Closed
| } | ||
|
|
||
| internal static TypeSymbolWithAnnotations Create(TypeSymbol typeSymbol, NullableAnnotation nullableAnnotation = NullableAnnotation.Unknown, ImmutableArray<CustomModifier> customModifiers = default) | ||
| internal static TypeSymbolWithAnnotations Create(TypeSymbol typeSymbol, NullableAnnotation CSharpNullableAnnotation = NullableAnnotation.Unknown, ImmutableArray<CustomModifier> customModifiers = default) |
There was a problem hiding this comment.
CSharpNullableAnnotation [](start = 99, length = 24)
Rename can be reverted. #Closed
| } | ||
|
|
||
| private static TypeSymbolWithAnnotations CreateNonLazyType(TypeSymbol typeSymbol, NullableAnnotation nullableAnnotation, ImmutableArray<CustomModifier> customModifiers) | ||
| private static TypeSymbolWithAnnotations CreateNonLazyType(TypeSymbol typeSymbol, NullableAnnotation CSharpNullableAnnotation, ImmutableArray<CustomModifier> customModifiers) |
There was a problem hiding this comment.
CSharpNullableAnnotation [](start = 109, length = 24)
Can be reverted. #Closed
|
|
||
| TypeSymbol typeSymbol = other.TypeSymbol; | ||
| NullableAnnotation nullableAnnotation = MergeNullableAnnotation(this.NullableAnnotation, other.NullableAnnotation, variance); | ||
| NullableAnnotation CSharpNullableAnnotation = MergeNullableAnnotation(this.NullableAnnotation, other.NullableAnnotation, variance); |
There was a problem hiding this comment.
CSharpNullableAnnotation [](start = 31, length = 24)
Can be reverted. #Closed
| } | ||
|
|
||
| [Fact, WorkItem(31370, "https://github.com/dotnet/roslyn/issues/31370")] | ||
| // PROTOTYPE(nullable-api): wants to change the type of the literal 2 to error type from int |
There was a problem hiding this comment.
PROTOTYPE(nullable-api): wants to change the type of the literal 2 to error type from int [](start = 11, length = 89)
Does this comment still apply? #Resolved
There was a problem hiding this comment.
Yes. This is one of the tests that had errors, so I want to leave it here as I've already spend some time to determine what the actual issue is.
In reply to: 266131177 [](ancestors = 266131177)
| base.Visit(node); | ||
| if (node is BoundExpression expr) | ||
| { | ||
| _map[expr.Syntax] = new TypeWithState(expr.Type, expr.TopLevelNullability.FlowState.ToInternalFlowState()).ToTypeSymbolWithAnnotations(); |
There was a problem hiding this comment.
new TypeWithState(expr.Type, expr.TopLevelNullability.FlowState.ToInternalFlowState()).ToTypeSymbolWithAnnotations() [](start = 44, length = 116)
Consider extracting a local function. #WontFix
There was a problem hiding this comment.
This visitor isn't intended to survive past the introduction of the public API. I'll leave this as is.
In reply to: 266133059 [](ancestors = 266133059)
| } | ||
| } | ||
|
|
||
| private sealed class TopLevelNullabilityRetriever : BoundTreeWalker |
There was a problem hiding this comment.
TopLevelNullability [](start = 29, length = 19)
Consider renaming since this returns not just top-level nullability but types with top-level and nested nullability. #Closed
This adds an initial implementation of the NullableWalker part of
rewriting the bound tree with nullability information, and hooks up the
rewriter to this. It additionally adds a debug verifier for the
NullableWalker that will verify that we're actually visiting and
recording results for all BoundExpression nodes in the tree. Some tests
are currently skipped where this does not hold true to allow the review
cycle to start while work continues.
Additionally, the VerifyTypes compilation helper is hooked up to this
rewritten information as a basic smoke test of the information being
produced by the rewriter.
For this PR, I'm most concerned about the general approach of getting
the information from the NullableWalker. If you have specific concerns
about some of the information from the walker being incorrect, please
note it and I'll put a prototype comment to follow up and add a test
to verify the behavior, but I'm not going to correct individual
scenarios in this PR.
To make this PR reviewable, I've separated the changes into 3 commits, and
I suggest going commit by commit. Commit 1 is the public API rename from
Nullability to NullableAnnotation and NullableFlowState. Commit 2 is
separating the public version of NullableAnnotation from the private
version, as that affects what we emit in the Nullable attribute and
we don't want to have a concept of default in that code path. Commit 3
adds the rewriting logic to the NullableWalker and skips some tests
that are currently not passing. There is some bleed-over of the rename
in NullableReferenceTypeTests.cs as I did not reorganize the commit
line-by-line in that file. I was going to clean up the locations
of NullableAnnotation/NullableFlowState in this PR, but no matter what
I do I'm going to have some conflicts with Neal's change so I'll fix
those up after this is merged.