Skip to content

TYP: fix stubtest errors in numpy.lib._index_tricks_impl#28529

Merged
charris merged 2 commits intonumpy:mainfrom
jorenham:numtype/235
Mar 15, 2025
Merged

TYP: fix stubtest errors in numpy.lib._index_tricks_impl#28529
charris merged 2 commits intonumpy:mainfrom
jorenham:numtype/235

Conversation

@jorenham
Copy link
Copy Markdown
Member

@jorenham jorenham commented Mar 15, 2025

Ported from numpy/numtype#235


  • move ndenumerate and ndindex definitions to lib._index_tricks_impl
  • add deprecated ndindex.ndincr property
  • removed non-existent ndenumerate.iter property
  • remove incorrect "pass" and "reveal" type-tests for ndenumerate.iter
  • fix incorrect ndenumerate constructor fallback return type
  • fix AxisConcatenator.makemat signature

@jorenham jorenham added 09 - Backport-Candidate PRs tagged should be backported 41 - Static typing numtype Isssue/PR related to numpy/numtype labels Mar 15, 2025
@jorenham jorenham force-pushed the numtype/235 branch 2 times, most recently from c4c58b9 to f6d6a92 Compare March 15, 2025 00:12
@github-actions

This comment has been minimized.

Ported from numpy/numtype#235

---

- move `ndenumerate` and `ndindex` definitions to `lib._index_tricks_impl`
- add deprecated `ndenumerate.ndincr` property
- removed non-existent `ndenumerate.iter` property
- remove incorrect "pass" and "reveal" type-tests for `ndenumerate.iter`
- fix incorrect `ndenumerate` constructor fallback return type
- fix `AxisConcatenator.makemat` signature
@github-actions

This comment has been minimized.

@jorenham jorenham removed the 09 - Backport-Candidate PRs tagged should be backported label Mar 15, 2025
@jorenham
Copy link
Copy Markdown
Member Author

jorenham commented Mar 15, 2025

@jorenham
Copy link
Copy Markdown
Member Author

jorenham commented Mar 15, 2025

Somehow, these changes caused mypy to infer a different return type for numpy.nonzero if Any is passed, even though nonzero lives in the completely untouched lib.fromnumeric module.

I tried to remove the first overload of nonzero. But this somehow caused the type-test mean to fail. Even though it's clear as day that mean and nonzero are completely independent function definitions.

Maybe this is what Einstein meant when he used the phrase "spooky action at a distance" 🤔.

@github-actions
Copy link
Copy Markdown

Diff from mypy_primer, showing the effect of this PR on type check results on a corpus of open source code:

pandas (https://github.com/pandas-dev/pandas)
- pandas/io/formats/format.py:1234: error: No overload variant matches argument type "ExtensionArray"  [operator]
- pandas/io/formats/format.py:1234: note: Possible overload variants:
- pandas/io/formats/format.py:1234: note:     def __and__(self, Literal[False] | numpy.bool[Literal[False]], /) -> numpy.bool[Literal[False]]
- pandas/io/formats/format.py:1234: note:     def __and__(self, Literal[True] | numpy.bool[Literal[True]], /) -> numpy.bool[builtins.bool]
- pandas/io/formats/format.py:1234: note:     def __and__(self, builtins.bool | numpy.bool[builtins.bool], /) -> numpy.bool[builtins.bool]
- pandas/io/formats/format.py:1234: note:     def [_IntegerT: integer[Any]] __and__(self, _IntegerT, /) -> _IntegerT
- pandas/io/formats/format.py:1234: note:     def __and__(self, int, /) -> numpy.bool[builtins.bool] | signedinteger[_32Bit | _64Bit]
- pandas/io/formats/format.py:1234: error: No overload variant of "__rand__" of "ndarray" matches argument type "ExtensionArray"  [operator]
- pandas/io/formats/format.py:1234: note:     def __rand__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]]]] | builtins.bool | _NestedSequence[builtins.bool], /) -> ndarray[tuple[int, ...], dtype[numpy.bool[builtins.bool]]]
- pandas/io/formats/format.py:1234: note:     def __rand__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[unsignedinteger[Any]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[unsignedinteger[Any]]]] | builtins.bool | _NestedSequence[builtins.bool], /) -> ndarray[tuple[int, ...], dtype[unsignedinteger[Any]]]
- pandas/io/formats/format.py:1234: note:     def __rand__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[integer[Any]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[integer[Any]]]] | builtins.bool | int | _NestedSequence[builtins.bool | int], /) -> ndarray[tuple[int, ...], dtype[signedinteger[Any]]]
- pandas/io/formats/format.py:1234: note:     def __rand__(self, _SupportsArray[dtype[object_]] | _NestedSequence[_SupportsArray[dtype[object_]]], /) -> Any
- pandas/io/formats/format.py:1234: note: Both left and right operands are unions
- pandas/io/formats/format.py:1240: error: Incompatible types in assignment (expression has type "Any | numpy.bool[builtins.bool]", variable has type "builtins.bool | None")  [assignment]

static-frame (https://github.com/static-frame/static-frame)
- static_frame/core/index_hierarchy.py:2193: error: Argument "condition" to "dropna" of "IndexHierarchy" has incompatible type overloaded function; expected "Callable[[ndarray[Any, Any]], ndarray[Any, Any]]"  [arg-type]
- static_frame/core/index_hierarchy.py:2512: error: Incompatible default for argument "condition" (default has type overloaded function, argument has type "Callable[[ndarray[Any, Any]], ndarray[Any, Any]]")  [assignment]
- static_frame/core/index_hierarchy.py:2524: error: Incompatible default for argument "condition" (default has type overloaded function, argument has type "Callable[[ndarray[Any, Any]], ndarray[Any, Any]]")  [assignment]
- static_frame/core/frame.py:4672: error: Incompatible default for argument "condition" (default has type overloaded function, argument has type "Callable[[ndarray[Any, Any]], ndarray[Any, Any]]")  [assignment]
- static_frame/core/frame.py:4727: error: Incompatible default for argument "condition" (default has type overloaded function, argument has type "Callable[[ndarray[Any, Any]], ndarray[Any, Any]]")  [assignment]
- static_frame/core/batch.py:1167: error: Incompatible default for argument "condition" (default has type overloaded function, argument has type "Callable[[ndarray[Any, Any]], builtins.bool]")  [assignment]
- static_frame/core/batch.py:1197: error: Incompatible default for argument "condition" (default has type overloaded function, argument has type "Callable[[ndarray[Any, Any]], builtins.bool]")  [assignment]

@jorenham
Copy link
Copy Markdown
Member Author

jorenham commented Mar 15, 2025

Diff from mypy_primer, showing the effect of this PR on type check results on a corpus of open source code:

pandas (https://github.com/pandas-dev/pandas)
- pandas/io/formats/format.py:1234: error: No overload variant matches argument type "ExtensionArray"  [operator]
- pandas/io/formats/format.py:1234: note: Possible overload variants:
- pandas/io/formats/format.py:1234: note:     def __and__(self, Literal[False] | numpy.bool[Literal[False]], /) -> numpy.bool[Literal[False]]
- pandas/io/formats/format.py:1234: note:     def __and__(self, Literal[True] | numpy.bool[Literal[True]], /) -> numpy.bool[builtins.bool]
- pandas/io/formats/format.py:1234: note:     def __and__(self, builtins.bool | numpy.bool[builtins.bool], /) -> numpy.bool[builtins.bool]
- pandas/io/formats/format.py:1234: note:     def [_IntegerT: integer[Any]] __and__(self, _IntegerT, /) -> _IntegerT
- pandas/io/formats/format.py:1234: note:     def __and__(self, int, /) -> numpy.bool[builtins.bool] | signedinteger[_32Bit | _64Bit]
- pandas/io/formats/format.py:1234: error: No overload variant of "__rand__" of "ndarray" matches argument type "ExtensionArray"  [operator]
- pandas/io/formats/format.py:1234: note:     def __rand__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]]]] | builtins.bool | _NestedSequence[builtins.bool], /) -> ndarray[tuple[int, ...], dtype[numpy.bool[builtins.bool]]]
- pandas/io/formats/format.py:1234: note:     def __rand__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[unsignedinteger[Any]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[unsignedinteger[Any]]]] | builtins.bool | _NestedSequence[builtins.bool], /) -> ndarray[tuple[int, ...], dtype[unsignedinteger[Any]]]
- pandas/io/formats/format.py:1234: note:     def __rand__(self, _SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[integer[Any]]] | _NestedSequence[_SupportsArray[dtype[numpy.bool[builtins.bool]] | dtype[integer[Any]]]] | builtins.bool | int | _NestedSequence[builtins.bool | int], /) -> ndarray[tuple[int, ...], dtype[signedinteger[Any]]]
- pandas/io/formats/format.py:1234: note:     def __rand__(self, _SupportsArray[dtype[object_]] | _NestedSequence[_SupportsArray[dtype[object_]]], /) -> Any
- pandas/io/formats/format.py:1234: note: Both left and right operands are unions
- pandas/io/formats/format.py:1240: error: Incompatible types in assignment (expression has type "Any | numpy.bool[builtins.bool]", variable has type "builtins.bool | None")  [assignment]

static-frame (https://github.com/static-frame/static-frame)
- static_frame/core/index_hierarchy.py:2193: error: Argument "condition" to "dropna" of "IndexHierarchy" has incompatible type overloaded function; expected "Callable[[ndarray[Any, Any]], ndarray[Any, Any]]"  [arg-type]
- static_frame/core/index_hierarchy.py:2512: error: Incompatible default for argument "condition" (default has type overloaded function, argument has type "Callable[[ndarray[Any, Any]], ndarray[Any, Any]]")  [assignment]
- static_frame/core/index_hierarchy.py:2524: error: Incompatible default for argument "condition" (default has type overloaded function, argument has type "Callable[[ndarray[Any, Any]], ndarray[Any, Any]]")  [assignment]
- static_frame/core/frame.py:4672: error: Incompatible default for argument "condition" (default has type overloaded function, argument has type "Callable[[ndarray[Any, Any]], ndarray[Any, Any]]")  [assignment]
- static_frame/core/frame.py:4727: error: Incompatible default for argument "condition" (default has type overloaded function, argument has type "Callable[[ndarray[Any, Any]], ndarray[Any, Any]]")  [assignment]
- static_frame/core/batch.py:1167: error: Incompatible default for argument "condition" (default has type overloaded function, argument has type "Callable[[ndarray[Any, Any]], builtins.bool]")  [assignment]
- static_frame/core/batch.py:1197: error: Incompatible default for argument "condition" (default has type overloaded function, argument has type "Callable[[ndarray[Any, Any]], builtins.bool]")  [assignment]

and again, the diff is completely unrelated to the changes 🤷🏻

https://github.com/pandas-dev/pandas/blob/8943c97c597677ae989d47e612705f088aab4cd0/pandas/io/formats/format.py#L1234

@jorenham jorenham added the 09 - Backport-Candidate PRs tagged should be backported label Mar 15, 2025
@charris charris merged commit 415a30f into numpy:main Mar 15, 2025
74 checks passed
@charris
Copy link
Copy Markdown
Member

charris commented Mar 15, 2025

Hmm, well let's give it a shot. Thanks Joren.

@jorenham jorenham deleted the numtype/235 branch March 15, 2025 19:14
@charris charris removed the 09 - Backport-Candidate PRs tagged should be backported label Mar 15, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

41 - Static typing numtype Isssue/PR related to numpy/numtype

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants