Skip to content

[ty] Support "legacy" typing.Self in combination with PEP 695 generic contexts#20304

Merged
sharkdp merged 1 commit intomainfrom
david/fix-1131
Sep 8, 2025
Merged

[ty] Support "legacy" typing.Self in combination with PEP 695 generic contexts#20304
sharkdp merged 1 commit intomainfrom
david/fix-1131

Conversation

@sharkdp
Copy link
Contributor

@sharkdp sharkdp commented Sep 8, 2025

Summary

Support cases like the following, where we need the generic context to include both Self and T (not just T):

from typing import Self

class C:
    def method[T](self: Self, arg: T): ...

C().method(1)

closes astral-sh/ty#1131

Test Plan

Added regression test

@sharkdp sharkdp added the ty Multi-file analysis & type inference label Sep 8, 2025
@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

Diagnostic diff on typing conformance tests

No changes detected when running ty on typing conformance tests ✅

@github-actions
Copy link
Contributor

github-actions bot commented Sep 8, 2025

mypy_primer results

No ecosystem changes detected ✅
No memory usage changes detected ✅

@sharkdp sharkdp marked this pull request as ready for review September 8, 2025 12:16
GenericContext::from_function_params(db, definition, &parameters, return_ty);

if generic_context.is_some() && legacy_generic_context.is_some() {
// TODO: Raise a diagnostic!
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The previous comment here indicates that this is probably not the right fix, but I don't see an immediate problem with merging legacy typevars and PEP 695 typevars in a single method, and it solves the problem with self: Self. So I'm opening this for discussion.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes... I think it's invalid to combine legacy and PEP-695 type variables in general... but I think the sole exception to this rule is probably Self, which I guess counts as a "legacy type variable", but is obviously different in several ways to most legacy type variables

Copy link
Contributor Author

@sharkdp sharkdp Sep 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, https://peps.python.org/pep-0695/#compatibility-with-traditional-typevars states that this is not legal in all situations. Maybe this indicates that implicit self should work differently for functions with an explicit [S, T, U, …] PEP 695 generic context. Maybe those functions shouldn't use typing.Self, but rather explicitly add a new synthetic type variable to this context: [_Self: C, S, T, U, …].

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The section about typing.Self doesn't seem to explicitly say one way or the other whether it's allowed in a PEP 695 generic function. I think it's worth supporting, and I think a slight modification to this approach is the right way to handle it.

Instead of merging the entirety of the legacy generic context, I would suggest looking to see if it contains (only) typing.Self. If so, add it to the PEP 695 generic context. If not, continue to "TODO: raise a diagnostic".

(Note that we do have a distinct TypeVarKind::TypingSelf for typing.Self, so it should be easy to distinguish from "real" legacy typevars.)

Copy link
Member

@AlexWaygood AlexWaygood Sep 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe those functions shouldn't use typing.Self, but rather explicitly add a new synthetic type variable to this context: [_Self: C, S, T, U, …].

Hmm, I don't think the intention was to deprecate typing.Self as part of PEP 695. I think the PEP would have stated this explicitly if that was the intention.

Adding Self to the type system didn't make it possible to express anything in type annotations that couldn't be expressed before (except maybe in attribute annotations): it just made a common case much less verbose and much more ergonomic. Before you would have done this:

from typing import TypeVar

_SelfT = TypeVar("_SelfT", bound="MyClass")

class MyClass:
    def method(self: _SelfT) -> SelfT:
        return self

and after the introduction of Self, you would do this:

from typing import Self

class MyClass:
    def method(self) -> Self:
        return self

And the two mean exactly the same thing. The ergonomics argument for Self maybe isn't quite as strong for methods that use PEP-695 parameters, since there's no need to declare the type variable outside the body of the method anymore -- but I still think I'd find it a pain to have to explicitly include it in the type parameter list every time.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Though my point about them not being equivalent is a stronger argument in favor of what you're suggesting, @AlexWaygood, which is that we should support Self in PEP 695 methods!)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

it makes sense to me that this would be syntactic sugar for def pep695_method[_Self: MyClass, T](self: _Self, x: T) instead of def pep605_method[T](self: typing.Self, x: T), because the former wouldn't have to mix legacy and PEP 695 type variables.

Yeah, that makes sense to me too

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think they're equivalent, since Self does something more sophisticated with the bound:

(We discussed this example on Discord -- other type checkers have the same behaviour on this snippet if you use a legacy TypeVar with an upper bound, and we should probably aim to have that behaviour too in the long run!)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should probably aim to have that behaviour too in the long run

I don't think that behavior is right, either for typing.Self or for an explicit typevar -- discussed more on Discord.

it makes sense to me that this would be syntactic sugar for def pep695_method[_Self: MyClass, T](self: _Self, x: T) instead of def pep605_method[T](self: typing.Self, x: T), because the former wouldn't have to mix legacy and PEP 695 type variables.

Is this a distinction that matters in any practical way? Ultimately legacy and PEP 695 typevars are the same thing, the only difference is that they are declared differently -- and in the case of un-annotated self, there is no explicit declaration, so that sole difference disappears, too. And if explicit typing.Self annotations are allowed in a PEP 695 generic method signature, then the need to mix typing.Self with PEP 695 generic contexts exists regardless.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this a distinction that matters in any practical way?

Not if typing.Self, TypeVar("_SelfT", bound="MyClass"), and [_Self: MyClass] all behave the same in all situations. From what I understand now, this is the case (which is a relief). I think I was maybe confused by the fact that TypeVarKind::TypingSelf exists, implying that it needs special treatment somehow (and @dcreager's example seemed to indicate that, but apparently that is just wrong behavior of other type checkers).

@sharkdp
Copy link
Contributor Author

sharkdp commented Sep 8, 2025

This change removes ~800 false positives across the ecosystem when based on #20303

@sharkdp sharkdp changed the title [ty] Merge legacy and PEP 695 generic contexts [ty] Support "legacy" typing.Self in combination with PEP 695 generic contexts Sep 8, 2025
Copy link
Member

@AlexWaygood AlexWaygood left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This looks reasonable to me now that it only applies to Self, but I'd appreciate it if @dcreager could take another look!

Copy link
Member

@dcreager dcreager left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

if legacy_ctx
.variables(db)
.iter()
.exactly_one()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😭 TIL exactly_one, I've been looking for an adapter like that forever!

@sharkdp sharkdp merged commit d55edb3 into main Sep 8, 2025
38 checks passed
@sharkdp sharkdp deleted the david/fix-1131 branch September 8, 2025 14:57
second-ed pushed a commit to second-ed/ruff that referenced this pull request Sep 9, 2025
…ic contexts (astral-sh#20304)

## Summary

Support cases like the following, where we need the generic context to
include both `Self` and `T` (not just `T`):

```py
from typing import Self

class C:
    def method[T](self: Self, arg: T): ...

C().method(1)
```

closes astral-sh/ty#1131

## Test Plan

Added regression test
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ty Multi-file analysis & type inference

Projects

None yet

Development

Successfully merging this pull request may close these issues.

invalid-argument-type for annotated self argument when a generic context exists

4 participants