Skip to content

TYP: Overlapping overloads with incompatible return types #27032

@jorenham

Description

@jorenham

Describe the issue:

The commonly used @overload pattern with _ArrayLike{}_co parameter types can lead to unexpected type inferences if the return types are incompatible with one another other.

An example of this is the numpy.dtype constructor:
It correctly infers dtype(bool) and dtype(int) as dtype[np.bool] and dtype[signedinteger[Any]], respectively.
But in the case that either bool or int can be passed, e.g as dtype(int_co) with int_co: type[bool] | tuple[int], the resulting type is inferred as dtype[signedinteger[Any]], which is incompatible with dtype[bool].
The correct type in this case would be dtype[bool | signedinteger[Any]].

Reproduce the code example:

# overlapping_overload_issue.pyi

import numpy as np

x: type[bool]
y: type[int]
z: type[bool] | type[int]

reveal_type(np.dtype(x))  # OK: dtype[bool]
reveal_type(np.dtype(y))  # OK: dtype[signedinteger[Any]]
reveal_type(np.dtype(z))  # WRONG: dtype[signedinteger[Any]]

But the last revealed type doesn't include bool, i.e. it should have been dtype[bool | signedinteger[Any]].

This isn't limited to only bool and int; all possible non-trivial unions of bool, int, float or complex have this problem, see https://typing.readthedocs.io/en/latest/spec/special-types.html#special-cases-for-float-and-complex

And this also isn't limited to dtype.__new__:
The same issue can be observed with the arithmetic operator methods (e.g. __add__) of numpy.ndarray, numpy.bool, and all subclasses of numpy.number, just to name a few.

Error message:

With Pyright / Pylance, enabling the reportOverlappingOverload setting will in such cases result in an error message (unless there's a # type: ignore comment), e.g. for numpy.dtype.__new__ Pylance reports:

Overload 1 for "__new__" overlaps overload 51 and returns an incompatible type
Overload 2 for "__new__" overlaps overload 3 and returns an incompatible type
Overload 2 for "__new__" overlaps overload 4 and returns an incompatible type
Overload 2 for "__new__" overlaps overload 5 and returns an incompatible type

Python and NumPy Versions:

2.1.0.dev0+git20240724.3df4283
3.10.12 (main, Mar 22 2024, 16:50:05) [GCC 11.4.0]

Type-checker version and settings:

basedpyright 1.15.0 (pyright 1.1.373)

Additional typing packages.

typing-extensions 4.12.2

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions