-
-
Notifications
You must be signed in to change notification settings - Fork 12k
ENH,BLD: Dynamically generate a .pyi file with platform-specific type hints #17514
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thanks @BvB93, this sounds like a good idea.
I'm actually failing to spot the platform-specific bit. E.g. a common type of dynamic check like |
They keyword here is "dynamic". The example you provided works just fine during runtime, but for static As a side note, the CI failures suggest that there are still some problems with the current implementation; |
|
Converting this into a draft for now, as there are still some issues getting the tests running properly. |
eric-wieser
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think it would be worth a detailed explanation somewhat about exactly what is going on with the aliases:
charthroughlonglongare the native types used inside numpy corresponding to C typesint8throughint64are both:- aliases for the above types
- the default name of all but one of these types (
intcorlonglongdepending on platform)
- the typing stubs are written as if the sized types are the native ones
- these files generate aliases in the reverse direction
- the reverse aliases are not quite accurate - they suggest that two of
intc, int_, longlongare identical, when in fact there are distinct scalar types
Good point, I'll update the PR description in a bit.
To be fair, this is consist with how the In [2]: np.byte
Out[2]: numpy.int8
I'm not quite sure how you arrived at this conclusion, as can be seen below the only two integer types Running the script locally yields the following output (MacOS): byte = int8
short = int16
intc = int32
intp = int0 = int64
int_ = int64
longlong = Any |
Ah, I wasn't aware that |
Yes, |
|
So I'm having some trouble to get this working with the likes of The part I'm struggling with is accessing the content of Could anyone provide some more insight here? |
|
Xref #16548 as this issue describes more or less what this PR is doing. |
|
Say, aren't the numerical scalars in Any thoughts? |
numpy/typing/_platform_aliases.py
Outdated
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If this file is replaced by the generated one at installation, what is the point in this version?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Its purpose is to provide runtime-accessible values so we can just export them to numpy/typing/__init__.pyi.
This way we don't have to worry about whether typing.TYPE_CHECKING is True or False.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'd also like to note that the script run during the numpy installation generates
_platform_aliases.pyi, not _platform_aliases.py.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It would be great if you could draw attention to the fact that the generated file has a different name in a comment.
`ctypes` scalars are now used for inferring the size of their `np.number`-based counterpart
mypy has issues recognizing an object as valid type aliases when using the likes of `clongfloat = longcomplex = clongdouble = Any`
Should be more consist with the `half, single, double, ...` series
…f `uint`; set the return type `signedinteger[Any]`
Use the return precision of either `self` or `int_`, depending on which one is larger
…cript * Clarified docstrings * Clarified comments * Use a `NamedTuple` as values in the `TO_BE_ANNOTATED` dictionary
|
Rebased to get rid of a merge conflict. |
|
Just out of curiosity - wouldn't it be easier to implement a small |
In principle we could do this with the help of from typing import Any, Annotated, Union, Literal
import numpy as np
import numpy.typing as npt
_Int64CodesBase = Literal['i8', ...]
_LongLongCodes = Literal['q', ...]
_IntPCodes = Literal['p', ...]
# A plugin can append the union with charcodes from `np.longlong`, `np.intp`, etc
_Int64Codes = Annotated[Union[_Int64CodesBase], "_Int64Codes"]
# A plugin can transform this into `intp = signedinteger[npt._64Bit]`
class intp(np.singedinteger[Any]): ... Personally I'd okay with either option, though going the plugin route does mean we'll have expose it to public API. |
|
After a bit of trial and error I managed to get the plugin-based approach working. |
|
Closed in favor of #17843. |
An issue which has plagued the typing numpy for some time are objects whose value or, even worse,
whose type is platform-specific. The most notorious example here is
np.int_, the latter being analias for either
np.int32(on windows for example) ornp.int64.It is desirable to have some sort of mechanism for annotating these platform-specifics; in order to
accomplish this the PR does two things:
.pyifile. This script is run at the very end ofsetup_package(), ensuring that a fresh.pyifile is generated after every numpy installation(similar to
np.version).np.typingwith valid character codes for eachnp.genericsubclass, the latter also containing a number of platform specifics in the case of
np.numbersubclasses.
Pinging @person142 as this would help with a few issues encountered in #16759.
EDIT: See #17843 for an alternative approach using a mypy plugin.