Documentation preview for this PR (built with commit 4ee4a99; changes) is ready! 🎉
This preview will update shortly after each push to this PR.

The linter fails now. What's the plan to follow the new rule? Deprecate the shadowing methods and provide new aliases for them?

I've turned off the rule in ruff. I thought the rule would only match type annotations, which are actually wrong, but it seems to also match names that are just stylistically not ideal.

Enforcement for the type annotations should be something that a type checker can catch once we have more use of type annotations. I think the main issue with our type annotations right now is missing type stubs for Cython files.

The linter fails now. What's the plan to follow the new rule? Deprecate the shadowing methods and provide new aliases for them?

I've turned off the rule in ruff. I thought the rule would only match type annotations, which are actually wrong, but it seems to also match names that are just stylistically not ideal.

I think it makes sense to activate that rule (and fix the issues), as shadowing builtins is indeed not ideal (regardless of the circumstances). But that of course doesn't have to happen in this PR.

I think the main issue with our type annotations right now is missing type stubs for Cython files.

I agree fully! AI could help, but is hallucinating quite a bit (as the cython source files are usually too long). My idea was to generate the empty stubs (without types) using #41377, and then adding typing info using AI. What do you think?

The current version of the PR looks good to me. Feel free to set it to positive review.

I think it makes sense to activate that rule (and fix the issues), as shadowing builtins is indeed not ideal (regardless of the circumstances). But that of course doesn't have to happen in this PR.

As long as the shadowed names aren't in the global namespace I don't think it's a huge issue. So while I probably wouldn't have a method named list on a class, there are higher priority fixes than removing list methods on old code. (I guess we could just enable it and noqa the existing violations, there aren't many.)

I think the main issue with our type annotations right now is missing type stubs for Cython files.

I agree fully! AI could help, but is hallucinating quite a bit (as the cython source files are usually too long). My idea was to generate the empty stubs (without types) using #41377, and then adding typing info using AI. What do you think?

I think our first goal needs to be getting the whole codebase to pass a type checker with minimal settings. The main issue with that right now I think is the lack of .pyi files causing import issues for type checkers. So automatically generating the stub files (like with #41377) would be a good start. From there we can work on adding more detailed annotations. My preference would be to improve the typestub generator to add those types (and ideally get it upstreamed into Cython), but I won't object to correct type annotations from other sources (by hand, AI, one-off ad-hoc CLI commands, etc.).

I think that having types for the .pyi files is probably less value than for Python files. Cython will loudly crash immediately if something is the wrong type (at least for a typed method/function), whereas wrong types in Python can introduce more subtle bugs. Having basic type stubs that can be picked up by a type checker so it doesn't give an error because it can't resolve an import is probably most of the value for stub files. Having explicit types in the stub files is still valuable of course though. I guess once #40634 is done that's another valuable addition from Cython type annotations (maybe, not sure how stub files and Cython files interact for documentation purposes).

I can look at #41377 once it is ready. One thing I will say for a long complicated custom script like that: it needs to be documented well enough that others can work on it without having to ask the original author questions.

I think our first goal needs to be getting the whole codebase to pass a type checker with minimal settings. The main issue with that right now I think is the lack of .pyi files causing import issues for type checkers. So automatically generating the stub files (like with #41377) would be a good start. From there we can work on adding more detailed annotations.

My preference would be to improve the typestub generator to add those types (and ideally get it upstreamed into Cython), but I won't object to correct type annotations from other sources (by hand, AI, one-off ad-hoc CLI commands, etc.).

Actually, many of our Cython methods don't have type infos for parameters. So it's very hard to autogen the pyi files with meaningful type infos. One approach could of course be to add typing first to the Cython files, which would likely have also other advantages in regard to performance and compile time errors.

I think that having types for the .pyi files is probably less value than for Python files.

Partly yes. Adding typing info to a method has two advantages:

• Type checkers can verify issues in the implementation of the method. As you say, this is not the case for pyi files.
• Type checkers can verify that callers of the method are passing the correct parameters / handle the return value correctly. For this, it doesn't really matter if the method is implemented in Cython or Python.

I can look at #41377 once it is ready. One thing I will say for a long complicated custom script like that: it needs to be documented well enough that others can work on it without having to ask the original author questions.

I marked this PR now as ready for review. Incrementally improving the script as we add more typing info/run it on more cython files is probably a good strategy.