Skip to content

Extensions: nullability analysis for invocations and method groups#78080

Merged
jcouv merged 9 commits intodotnet:mainfrom
jcouv:extensions-nullable
Apr 23, 2025
Merged

Extensions: nullability analysis for invocations and method groups#78080
jcouv merged 9 commits intodotnet:mainfrom
jcouv:extensions-nullable

Conversation

@jcouv
Copy link
Member

@jcouv jcouv commented Apr 9, 2025

Included:

  • invocations
  • method groups conversions
  • deconstruction
  • state of extension parameter

Out-of-scope:

  • other pattern-based scenarios (foreach, etc)
  • properties

For attributes, after some discussion, I updated the PR to maximally support the nullability attributes for compatibility/portability.

Filled #78022 (re-inferred method group not properly shown in semantic model and var)
Fixes #78182

Relates to test plan #76130

@jcouv jcouv added Area-Compilers Feature - Extension Everything The extension everything feature labels Apr 9, 2025
@jcouv jcouv self-assigned this Apr 9, 2025
@ghost ghost added the untriaged Issues and PRs which have not yet been triaged by a lead label Apr 9, 2025
@jcouv jcouv force-pushed the extensions-nullable branch 4 times, most recently from 27ea047 to 44cd555 Compare April 9, 2025 17:15
@jcouv jcouv force-pushed the extensions-nullable branch from 44cd555 to 2f2f70d Compare April 10, 2025 09:19
@jcouv jcouv marked this pull request as ready for review April 10, 2025 11:06
@jcouv jcouv requested a review from a team as a code owner April 10, 2025 11:06
@jcouv jcouv requested review from AlekseyTs and jjonescz April 10, 2025 11:06
@jcouv
Copy link
Member Author

jcouv commented Apr 11, 2025

@jjonescz @AlekseyTs for review. Thanks

in CheckConstraintsArgs args)
{
if (!RequiresChecking(method))
Debug.Assert(!method.GetIsNewExtensionMember()); // Tracked by https://github.com/dotnet/roslyn/issues/76130: Review all remaining callers with regards to new extensions
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review all remaining callers with regards to new extensions

Is the goal to remove the enclosing method or CheckConstraintsIncludingExtension at the end? If so, consider saying this explicitly. If not, could you please elaborate what is the goal? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The question was whether we want any of the callers be affected by extensions. If not, they'll continue to use this method. Otherwise, their logic needs to be updated (lookup logic) and they should call CheckConstraintsIncludingExtension.
After reviewing the callers, I see that none of them accounts for classic extension methods, so I don't think they should be affected by new extension methods either.
Will remove this follow-up comment

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I am sorry, but the response doesn't make sense to me. We don't want the callers to be affected , but we call CheckConstraintsIncludingExtension here. So, why is it a good thing to have two methods after you reviewed all callers?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved offline. Brought back original name

skipParameters);
}

public static bool CheckMethodConstraintsIncludingExtension(
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CheckMethodConstraintsIncludingExtension

Why do we want to add a different method instead of simply adjusting behavior of the original method? #Closed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed offline that we probably want to keep the original name of the method and push the logic that checks constraints on the extension down to the "root" CheckConstraints helper (so that no one could miss that logic no matter what overload is used).

{
return method.GetIsNewExtensionMember()
? method.ContainingType.TypeParameters.Concat(method.TypeParameters)
: method.ConstructedFrom.TypeParameters;
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ConstructedFrom

Why is it necessary to go through ConstructedFrom wouldn't method.TypeParameters return the right thing? #Closed


if (member is PropertySymbol property)
{
Debug.Assert(property.GetIsNewExtensionMember());
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Debug.Assert(property.GetIsNewExtensionMember());

It feels inconsistent that properties and methods get different treatment. For methods we check this condition, but for properties we assume it is true. #Closed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This thread was marked as resolved, but I do not see any related code change and there was no other response.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The resolution is the added assertion at the top of the method: Debug.Assert(member.GetMemberArityIncludingExtension() != 0);

throw ExceptionUtilities.UnexpectedValue(member);
}

internal static PooledDictionary<TypeParameterSymbol, int>? MakeOrdinalsIfNeeded<TMember>(this TMember member, ImmutableArray<TypeParameterSymbol> originalTypeParameters)
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

MakeOrdinalsIfNeeded

"MakeAdjustedTypeParameterOrdinalsIfNeeded"? #Closed


// Since we're concatenating type parameters from the extension and from the method together
// we need to control the ordinals that are used
ordinals = PooledDictionary<TypeParameterSymbol, int>.GetInstance();
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ordinals = PooledDictionary<TypeParameterSymbol, int>.GetInstance();

I think the map that we create here is supposed to use identity comparison. Is this what we get? #Closed

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This thread was resolved, but I do not think changing the key to object changes anything about how keys are compared

// M(object, object) and is therefore the best match.

return !methodSymbol.CheckConstraints(new ConstraintsHelper.CheckConstraintsArgs(this.Compilation, this.Conversions, includeNullability: false, node.Location, diagnostics));
return !methodSymbol.CheckConstraintsIncludingExtension(new ConstraintsHelper.CheckConstraintsArgs(this.Compilation, this.Conversions, includeNullability: false, node.Location, diagnostics));
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CheckConstraintsIncludingExtension

Is this change covered by a test? #Closed

Copy link
Member Author

@jcouv jcouv Apr 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will undo this change and the one in GetCandidatesPassingFinalValidation. In the language versions where we include new extension members in overload resolution, we filter bad candidates (which fail constraint checks) early, so the final validation check doesn't make a difference.
But new extensions can make it here, so the helper with assertion cannot be used.

TMethodOrPropertySymbol member = result.Member;
if (!MemberGroupFinalValidationAccessibilityChecks(receiverOpt, member, syntax, candidateDiagnostics, invokedAsExtensionMethod: isExtensionMethodGroup && !member.GetIsNewExtensionMember()) &&
(typeArgumentsOpt.IsDefault || ((MethodSymbol)(object)result.Member).CheckConstraints(new ConstraintsHelper.CheckConstraintsArgs(this.Compilation, this.Conversions, includeNullability: false, syntax.Location, candidateDiagnostics))))
(typeArgumentsOpt.IsDefault || ((MethodSymbol)(object)result.Member).CheckConstraintsIncludingExtension(new ConstraintsHelper.CheckConstraintsArgs(this.Compilation, this.Conversions, includeNullability: false, syntax.Location, candidateDiagnostics))))
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CheckConstraintsIncludingExtension

Is this change covered by a test? #Closed

foreach (var m in methods)
{
if (!IsUnboundGeneric(m) && m.ParameterCount > 0)
if (!IsUnboundGeneric(m) && m.GetParametersIncludingExtensionParameter().Length > 0)
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GetParametersIncludingExtensionParameter

It might be better to have an API that returns combined count and avoids unnecessary allocations. #Closed

if (!IsUnboundGeneric(m) && m.ParameterCount > 0)
if (!IsUnboundGeneric(m) && m.GetParametersIncludingExtensionParameter().Length > 0)
{
parameterListList.Add(m.Parameters);
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parameterListList.Add(m.Parameters);

Shouldn't we also include extension parameter? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added and tried to observe the change, but I wasn't able to


MethodSymbol method = (MethodSymbol)(Symbol)result.Member;
if (!method.CheckConstraints(new ConstraintsHelper.CheckConstraintsArgs(compilation, conversions, includeNullability: false, location, diagnostics)))
if (!method.CheckConstraintsIncludingExtension(new ConstraintsHelper.CheckConstraintsArgs(compilation, conversions, includeNullability: false, location, diagnostics)))
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CheckConstraintsIncludingExtension

Is this change covered by a test? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don't report detailed overload resolution diagnostics for extensions, so I'll undo this.

{
// The receiver will be analyzed as an argument instead
receiver = null;
return false;
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

return false;

This defeats the purpose of the method to enable handling of long chains of fluent calls. Note that for classic extension methods we return true (line 6388). #Closed

AssignmentKind.Assignment);

bool reportedDiagnostic = enumeratorInfoOpt?.GetEnumeratorInfo.Method is { IsExtensionMethod: true } // Tracked by https://github.com/dotnet/roslyn/issues/76130: Test this code path with new extensions
bool reportedDiagnostic = enumeratorInfoOpt?.GetEnumeratorInfo.Method is { IsExtensionMethod: true } // Tracked by https://github.com/dotnet/roslyn/issues/76130: Test this code path with new extensions // Tracked by https://github.com/dotnet/roslyn/issues/76130: Test this code path with new extensions
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

bool reportedDiagnostic = enumeratorInfoOpt?.GetEnumeratorInfo.Method is { IsExtensionMethod: true } // Tracked by #76130: Test this code path with new extensions // Tracked by #76130: Test this code path with new extensions

Is this change really necessary? #Closed

// (4,1): warning CS8602: Dereference of a possibly null reference.
// s.M();
Diagnostic(ErrorCode.WRN_NullReferenceReceiver, "s").WithLocation(4, 1));
comp.VerifyEmitDiagnostics();
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

comp.VerifyEmitDiagnostics();

Consider calling CompileAndVerify(...).VerifyDiagnostics() instead #Closed

#nullable enable

object? oNull = null;
oNull.M(); // 1
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 11, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oNull

Do we have tests covering nullability of arguments (both for static and instance extensions)? #Closed

// Map of TypeParameterSymbol to ordinal for new extension methods
// When doing type inference on a new extension method, we combine the type parameters
// from the extension declaration and from the method, so we cannot rely on the ordinals from the type parameters.
private readonly Dictionary<TypeParameterSymbol, int> _ordinals;
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TypeParameterSymbol

It is not clear why we are relaxing the type #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In order to get reference equality. We could allocate, we could introduce a new pool, or we could use object as the key type.

if (!IsUnboundGeneric(m) && m.GetParameterCountIncludingExtensionParameter() > 0)
{
parameterListList.Add(m.Parameters);
parameterListList.Add(m.GetParametersIncludingExtensionParameter());
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parameterListList.Add(m.GetParametersIncludingExtensionParameter());

Are we testing this code path with static extensions? #Closed

}

public static bool RequiresChecking(MethodSymbol method)
public static bool RequiresCheckingIncludingExtension(MethodSymbol method)
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

RequiresCheckingIncludingExtension

I don't think we need to change the name of this method. #Closed

{
method = InferMethodTypeArguments(method, GetArgumentsForMethodTypeInference(results, argumentsNoConversions), refKindsOpt, argsToParamsOpt, expanded);
parametersOpt = method.Parameters;
parametersOpt = method.GetParametersIncludingExtensionParameter();
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

parametersOpt = method.GetParametersIncludingExtensionParameter();

Are we testing this code path for static extensions? #Closed

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, Nullability_Invocation_09, the object.M2() invocation

}

private void CheckMethodConstraints(SyntaxNode syntax, MethodSymbol method)
private void CheckMethodConstraintsIncludingExtension(SyntaxNode syntax, MethodSymbol method)
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CheckMethodConstraintsIncludingExtension

It doesn't look like there is a need to rename this method #Closed

skipParameters);
}

if (method.GetIsNewExtensionMember() && method.ContainingType is { Arity: > 0 } extension
Copy link
Contributor

@AlekseyTs AlekseyTs Apr 16, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if (method.GetIsNewExtensionMember() && method.ContainingType is { Arity: > 0 } extension

As we discussed offline, I think this logic should be moved into the method called above, and quite possibly into much lower level helper, the "root". Therefore, I do not expect implementation of the enclosing method to change. #Closed

@AlekseyTs
Copy link
Contributor

Done with review pass (commit 4)

Copy link
Contributor

@AlekseyTs AlekseyTs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (commit 6), assuming CI is passing

@jcouv
Copy link
Member Author

jcouv commented Apr 17, 2025

@jjonescz for another review. Thanks

return;
}

ImmutableArray<ParameterSymbol> parameters = method.IsStatic ? method.Parameters : method.GetParametersIncludingExtensionParameter();
Copy link
Member

@jjonescz jjonescz Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

method.IsStatic ? method.Parameters : method.GetParametersIncludingExtensionParameter()

It feels like there could be a helper for this common pattern. Even better, the existing helper could have a non-optional parameter specifying what should happen in the static case so callers don't forget to handle that properly. #Resolved

if (node.ReceiverOpt is not null)
{
// Tracked by https://github.com/dotnet/roslyn/issues/76130: Do we need to do anything special for new extensions here?
VisitRvalueEpilogue(receiver); // VisitRvalue does this after visiting each node
Copy link
Member

@jjonescz jjonescz Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why don't we need to do anything special for new extensions here? Consider adding a comment. #Resolved

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good catch. I forgot to revisit this when enabling the stack-based optimization.
We need to set extensionReceiverResult properly as it is used as firstArgumentResult: argument in the next iteration of the loop. That way, when we get to visit arguments in reinferMethodAndVisitArguments (and ultimately in VisitArgument) we skip visiting that argument, so that we don't end up visiting it twice.
The added tests verify this.

On the flip side, I couldn't come up with any test case where extensionReceiverResult = _visitResult; wouldn't be good enough. The call to VisitArgumentEvaluateEpilogue is right as far as I can tell, but I was not able to observe it (and neither do existing nullable tests).

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed one more commit for a test that covers this properly. It took me some time to come up with a test but it's possible :-)


if (argumentRefKindsOpt.IsDefault)
{
if (extensionParameter.RefKind == RefKind.None)
Copy link
Member

@jjonescz jjonescz Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shouldn't this check receiverRefKind? Otherwise if extensionParameter.RefKind is In for example, we construct the array unnecessarily. #Resolved


ParameterSymbol? extensionParameter = method.ContainingType.ExtensionParameter;
Debug.Assert(extensionParameter is not null);
var receiverRefKind = extensionParameter.RefKind == RefKind.Ref ? RefKind.Ref : RefKind.None;
Copy link
Member

@jjonescz jjonescz Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this came up previously, can you remind me why we check only Ref and not other ref kinds? #Resolved

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This follows what we do for classic extension methods. See OverloadReolution.IsApplicable:

                    if (forExtensionMethodThisArg)
                    {
                        Debug.Assert(argumentRefKind == RefKind.None);
                        if (parameterRefKind == RefKind.Ref)
                        {
                            // For ref extension methods, we omit the "ref" modifier on the receiver arguments
                            // Passing the parameter RefKind for finding the correct conversion.
                            // For ref-readonly extension methods, argumentRefKind is always None.
                            argumentRefKind = parameterRefKind;
                        }
                    }

So when we need to do inference or argument conversions, we add a ref if the parameter is ref, and we leave none for ref readonly/in. out is a declaration error.

Debug.Assert(receiverSlot >= 0);
if (method.GetIsNewExtensionMember())
{
return;
Copy link
Member

@jjonescz jjonescz Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are post conditions skipped for new extension members? #Resolved

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The are not just "post-conditions", they are "member post-conditions". The attributes that are processed below are MemberNotNull and MemberNotNullWhen.
Normally those apply to members of the containing type, which holds states. My thinking was that extensions don't hold state, so those attributes don't apply.
But extension members could be stateful, so it may make sense after all.
I'll add a follow-up comment to confirm whether we want this to work. If so, we'll need to spec it first. Thanks!

#nullable enable

object? oNull = null;
oNull.M(); // 1
Copy link
Member

@jjonescz jjonescz Apr 17, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider testing the suppression operator (!) as well for at least a few cases. #Resolved

@jcouv jcouv requested a review from jjonescz April 18, 2025 20:01
@jcouv jcouv merged commit 19e545b into dotnet:main Apr 23, 2025
24 checks passed
@jcouv jcouv deleted the extensions-nullable branch April 23, 2025 13:24
@dotnet-policy-service dotnet-policy-service bot added this to the Next milestone Apr 23, 2025
@RikkiGibson RikkiGibson modified the milestones: Next, 18.0 P1 Aug 19, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area-Compilers Feature - Extension Everything The extension everything feature untriaged Issues and PRs which have not yet been triaged by a lead

Projects

None yet

Development

Successfully merging this pull request may close these issues.

False positive CS8602 when using Extension Member Feature

4 participants