Just curious about something since I've been following the work to add typing support for dtypes: won't making a mutable container type covariant cause issues? For the usual reasons that, e.g. if my_func accepts ndarray[Any, np.dtype[np.float64]) it could assign values of float64 to the array, but calling my_func with ndarray[Any, np.dtype[np.float32]) would no longer raise typecheck errors despite being wrong.

@a-reich this is arguably more of a problem with the variancy of number, one that I'm not quite about what would be the best solution. Namely, making number[~NBit] invariant (instead of covariant) w.r.t. its precision would resolve the issue. On the other hand, it would also make precision-based casting even difficult than it already is, to the point where I'm not convinced it'd be worthwhile. The latter means ditching the limited cases where we can currently deal with precisions, i.e. changing the return types to the likes of number[Any].

In any case, besides abovementioned issue the problem with leaving ndarray (and dtype) invariant is that it leads to some strange situations with the abstract(-ish) part of the generic hierarchy. For example, it would mean that a float64 array would not be acceptable if an inexact array is expected.

@BvB93 I am trying to get a full picture for this. For your last example, it seems clear that inexact must behave covariant (as do all abstract dtypes). The concrete dtypes/types must be invariant (in most ways at least), but because of that, my preferred solution is to not allow subclassing at all (and if we ever allow it, we would be very careful to note that it must be extremely limited – whatever that means exactly).

The number[Any], I am a bit confused about how/where this is used (with a bit-width)?
My first thought would be that number[Any] would be covariant as well (it is an abstract dtype/type), but it would be "invariant" with respect to precision since a higher precision is not a subtype to begin with?

@BvB93 I see what you mean about the abstract parts of the dtype hierarchy making it difficult. However, your point about invariance of number w.r.t. to precision is interesting.
I guess ultimately, there might not be a way to have the whole type system work "correctly" here, so it comes down to which way generates more false positives/false negatives and is more annoying for downstream users to patch over? For example, how common actually are operations with different precisions, and if lack of automatic inference by mypy means users have to annotate their casting explicitly how bad is that? Compared to the false negatives for assigning with wrong precision (which admittedly does not raise at runtime, but silently stores truncated results).

The number[Any], I am a bit confused about how/where this is used (with a bit-width)?

To give a bit of background: number is currently parametrized w.r.t. a set of npt.NBitBase subclasses,
the latter used for representing their precision via an object-based approach.

_NBit_co = TypeVar(..., covariant=True)

class floating(inexact[_NBit_co]):
    ...

# Note that the types below are not necasrelly treated as `floating` subclasses,
# it's more along the line of how `Sequence[int]` is a subtype of  plain `Sequence`.
#
# This greatly simplifies the rather complicated typing of precisions, as you can always 
# just hit the panic button when things get too complicated: _e.g._ `return floating[Any]`
float16 = floating[_16Bit]
float32 = floating[_32Bit]
float64 = floating[_64Bit]

The key point is that because NBitBase subclasses inherit from each other, whenever one of them is
used as a covariant parameter it would allow mypy to simplify the likes of Union[_64Bit, _32Bit] to
_64Bit, allowing for some basic precision-based casting without adding loads of overloads.

I recall from #17540 that something like this was actually possible for invariant parameters, but I
believe there were more immediate issues with it, at least compared to its covariant counterpart.
I'd have to a bit of digging, as I don't quite remember the details.

As a side note:
Let's wait with merging until #18155 is merged, as it would allow for the removal of an overload.

Never mind #18128 (comment), turns out there were, unfortunately, detrimental problems with the therein referenced PR

Needs rebase

The branch has been rebased and the tests seem to be passing.

Thanks Bas.