close/reopen

@BvB93 Thoughts?

Not a typing expert, but I would think the solution is to add another overload for ArrayLike, ArrayLike -> NDarray[Any] indicating that an unknown dtype should generally pass. (That is probably not true, but the best guess when the dtype is not typed.)

After a bit more thought I realize this is a terrible solution for many reasons, but still may be a good starting point for dealing with the problem.

Not a typing expert, but I would think the solution is to add another overload for ArrayLike, ArrayLike -> NDarray[Any] indicating that an unknown dtype should generally pass. (That is probably not true, but the best guess when the dtype is not typed.)

I think the issue here is not that there is no matching overload for NDArray[Any], but rather that there are too many. Here's a very straightforward example of what I mean:

@overload
def fun(x: int) -> int:
    ...


@overload
def fun(x: Any) -> float:
    ...


x: Any = ...
y = fun(x)

reveal_type(y)

The problem with this code is that there is no "right" answer for what the type of y should be.

Some people (like me) would intuitively assume that since there is an explicitly defined overload for arguments of type Any, that's the one that would be selected and y would be a float. However, since Any matches any type, not just itself, that's not the only option.

Mypy takes the more conservative approach and assumes y is also Any, since there's no way of knowing what overload would actually be called until runtime.

Pyright takes the opposite approach and picks the first matching overload, meaning y ends up as an int. After all, x could be an int, so this is not wrong and it allows the pylance IDE autosuggestions to at least give you some options.

You might notice the issue with this code snippet is not how the type checkers deal with it, but rather a bug in the code itself. The code in the second overload is telling type checkers "if the input is anything other than an int, we return a float". That's what Any means, after all. Indeed, the code works correctly according to that definition -- changing it to b = "" makes both type checkers agree that y is a float. What we really want to say, however, is "if the input type is anything other than int, or if we don't know what the input type is, assume the return is a float". Unfortunately, there's no construct in python that I know of that is able to express this.

One option I've thought of is defining a new type that is unused anywhere else, like class _Unknown: ... and checking against that before anything else. This should match the "if we don't know what the input type is" part of our goal, at least for pyright, since it matches the Any input type (and nothing alse) and is the first defined overload. It wouldn't work with mypy, but the issues with mypy are less severe anyway since they assume the return to be Any already. This kind of feels like a hack, but it's the best idea I've got so far.

Some people (see here) have suggested adding the option to delete specific matches from a type to pyhton itself. This would be a neat solution, being able to say "match anything except the types we will define later", but unfortunately it seems to not have gained much traction.

Sorry for the wall of text and the non-conclusion, I'll keep thinking about this and see if I can come up with a better solution. In the meantime, feel free to comment with thoughts/suggestions/criticisms.

TL;DR: I messed up, the issue is way more complicated than originally assumed, I did come up with something that sort-of-works but I'm not yet sure if it's good.

Thanks for your time and comments @BvB93, I've gone ahead and fixed everything you mentioned (I think).

I do also have my concerns about longevity, especially if pyright decides to change how they handle this at some point, but it's the best I could come up with for now.

Thanks @iantra, LGTM.