[ty] Support "legacy" typing.Self in combination with PEP 695 generic contexts#20304
[ty] Support "legacy" typing.Self in combination with PEP 695 generic contexts#20304
typing.Self in combination with PEP 695 generic contexts#20304Conversation
Diagnostic diff on typing conformance testsNo changes detected when running ty on typing conformance tests ✅ |
d1d136e to
18318a6
Compare
|
18318a6 to
23a3882
Compare
| GenericContext::from_function_params(db, definition, ¶meters, return_ty); | ||
|
|
||
| if generic_context.is_some() && legacy_generic_context.is_some() { | ||
| // TODO: Raise a diagnostic! |
There was a problem hiding this comment.
The previous comment here indicates that this is probably not the right fix, but I don't see an immediate problem with merging legacy typevars and PEP 695 typevars in a single method, and it solves the problem with self: Self. So I'm opening this for discussion.
There was a problem hiding this comment.
yes... I think it's invalid to combine legacy and PEP-695 type variables in general... but I think the sole exception to this rule is probably Self, which I guess counts as a "legacy type variable", but is obviously different in several ways to most legacy type variables
There was a problem hiding this comment.
Yes, https://peps.python.org/pep-0695/#compatibility-with-traditional-typevars states that this is not legal in all situations. Maybe this indicates that implicit self should work differently for functions with an explicit [S, T, U, …] PEP 695 generic context. Maybe those functions shouldn't use typing.Self, but rather explicitly add a new synthetic type variable to this context: [_Self: C, S, T, U, …].
There was a problem hiding this comment.
The section about typing.Self doesn't seem to explicitly say one way or the other whether it's allowed in a PEP 695 generic function. I think it's worth supporting, and I think a slight modification to this approach is the right way to handle it.
Instead of merging the entirety of the legacy generic context, I would suggest looking to see if it contains (only) typing.Self. If so, add it to the PEP 695 generic context. If not, continue to "TODO: raise a diagnostic".
(Note that we do have a distinct TypeVarKind::TypingSelf for typing.Self, so it should be easy to distinguish from "real" legacy typevars.)
There was a problem hiding this comment.
Maybe those functions shouldn't use
typing.Self, but rather explicitly add a new synthetic type variable to this context:[_Self: C, S, T, U, …].
Hmm, I don't think the intention was to deprecate typing.Self as part of PEP 695. I think the PEP would have stated this explicitly if that was the intention.
Adding Self to the type system didn't make it possible to express anything in type annotations that couldn't be expressed before (except maybe in attribute annotations): it just made a common case much less verbose and much more ergonomic. Before you would have done this:
from typing import TypeVar
_SelfT = TypeVar("_SelfT", bound="MyClass")
class MyClass:
def method(self: _SelfT) -> SelfT:
return selfand after the introduction of Self, you would do this:
from typing import Self
class MyClass:
def method(self) -> Self:
return selfAnd the two mean exactly the same thing. The ergonomics argument for Self maybe isn't quite as strong for methods that use PEP-695 parameters, since there's no need to declare the type variable outside the body of the method anymore -- but I still think I'd find it a pain to have to explicitly include it in the type parameter list every time.
There was a problem hiding this comment.
(Though my point about them not being equivalent is a stronger argument in favor of what you're suggesting, @AlexWaygood, which is that we should support Self in PEP 695 methods!)
There was a problem hiding this comment.
it makes sense to me that this would be syntactic sugar for
def pep695_method[_Self: MyClass, T](self: _Self, x: T)instead ofdef pep605_method[T](self: typing.Self, x: T), because the former wouldn't have to mix legacy and PEP 695 type variables.
Yeah, that makes sense to me too
There was a problem hiding this comment.
I don't think they're equivalent, since
Selfdoes something more sophisticated with the bound:
(We discussed this example on Discord -- other type checkers have the same behaviour on this snippet if you use a legacy TypeVar with an upper bound, and we should probably aim to have that behaviour too in the long run!)
There was a problem hiding this comment.
we should probably aim to have that behaviour too in the long run
I don't think that behavior is right, either for typing.Self or for an explicit typevar -- discussed more on Discord.
it makes sense to me that this would be syntactic sugar for
def pep695_method[_Self: MyClass, T](self: _Self, x: T)instead ofdef pep605_method[T](self: typing.Self, x: T), because the former wouldn't have to mix legacy and PEP 695 type variables.
Is this a distinction that matters in any practical way? Ultimately legacy and PEP 695 typevars are the same thing, the only difference is that they are declared differently -- and in the case of un-annotated self, there is no explicit declaration, so that sole difference disappears, too. And if explicit typing.Self annotations are allowed in a PEP 695 generic method signature, then the need to mix typing.Self with PEP 695 generic contexts exists regardless.
There was a problem hiding this comment.
Is this a distinction that matters in any practical way?
Not if typing.Self, TypeVar("_SelfT", bound="MyClass"), and [_Self: MyClass] all behave the same in all situations. From what I understand now, this is the case (which is a relief). I think I was maybe confused by the fact that TypeVarKind::TypingSelf exists, implying that it needs special treatment somehow (and @dcreager's example seemed to indicate that, but apparently that is just wrong behavior of other type checkers).
23a3882 to
c645a73
Compare
c645a73 to
83cc559
Compare
|
This change removes ~800 false positives across the ecosystem when based on #20303 |
typing.Self in combination with PEP 695 generic contexts
There was a problem hiding this comment.
This looks reasonable to me now that it only applies to Self, but I'd appreciate it if @dcreager could take another look!
| if legacy_ctx | ||
| .variables(db) | ||
| .iter() | ||
| .exactly_one() |
There was a problem hiding this comment.
😭 TIL exactly_one, I've been looking for an adapter like that forever!
…ic contexts (astral-sh#20304) ## Summary Support cases like the following, where we need the generic context to include both `Self` and `T` (not just `T`): ```py from typing import Self class C: def method[T](self: Self, arg: T): ... C().method(1) ``` closes astral-sh/ty#1131 ## Test Plan Added regression test
Summary
Support cases like the following, where we need the generic context to include both
SelfandT(not justT):closes astral-sh/ty#1131
Test Plan
Added regression test