MAINT,API: Introduce __numpy_dtype__ and fix dtype attribute recursion #30179

seberg · 2025-11-08T12:14:23Z

This is a lot more complex then I expected. We, correctly, deprecated .dtype attribute lookup recursion, however...
The code still had a try/except that still recursed and that try except carried meaning at least in a weird path of:

class mydtype(np.void):
    dtype = ...

Now does that make sense? Maybe not... we also fall back to object in some paths which should have been broken when a dtype attribute existed on a type but it is a property/descriptor.

So, this removes the recursion, but adds a check for __get__ to filter out those cases, this is something we successfully did for other protocols __array__, __array_interface__, etc.

This is a lot more complex then I expected. We, correctly, deprecated `.dtype` attribute lookup recursion, however... The code still had a `try/except` that still recursed and that try except carried meaning at least in a weird path of: ``` class mydtype(np.void): dtype = ... ``` Now does that make sense? Maybe not... we also fall back to object in some paths which should have been broken when a dtype attribute existed on a type but it is a property/descriptor. So, this removes the recursion, but adds a check for `__get__` to filter out those cases, this is something we successfully did for other protocols `__array__`, `__array_interface__`, etc. Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>

mattip · 2025-11-08T17:29:27Z

Would adding a __numpy_dtype__ attribute to np.float64 make the detection logic simpler? The documentation mentions one must use np.dtype(np.float64), maybe we could have np.float64.__numpy_dytpe__ instead

seberg · 2025-11-09T08:22:42Z

Yeah, that is a nice thought, we should maybe add it to deprecate the use of the dtype on the instance.
I think there are two things that make me not jump at it. First, we need type lookup for some things anyway (i.e. I would prefer not to require this for user DTypes; that said such user dtypes don'tsupport the conversion right nowI suspect, that should be fixed).
Second, it does need a careful property dance, since the class attribute needs to give a dtype, while the instance attribute should give an error. That said... while I sm not sure that the Python API makes that simple, it alsodoesn't make it hard.

I guess that makesme lean to: worth an issue/followup, but maybe not needed here?

timhoffm · 2025-11-09T17:05:28Z

doc/release/upcoming_changes/30179.new_feature.rst

+For array-like objects we encourage you to implement ``__numpy_dtype__``
+with a warning or error to _prevent_ using e.g. ``dtype=dataframe`` in
+NumPy functions (it may be good to go via a Deprecation).


Do we really need other array libraries to take action? If they don't do anything, the status-quo for these types of array-likes is maintained (works or doesn't work depending on the case).

A warning / deprecation / error can be handled centrally in numpy. You could even make this behavior dependent on whether the passed object is array-like and/or whether they return a numpy dtype.

Let me rephrase it to say that you could do this. NumPy would indeed deprecate it for you as soon as we actually deprecate .dtype.
I agree with that deprecation, but think we may want to wait a little bit, implementing it in the downstream library would be an early opt-in effectively.

inakleinbottle

I had a look through the changes to the C code. I can't see any obvious issues in the C code.

ngoldbaum · 2025-11-26T20:54:11Z

numpy/_core/tests/test_dtype.py

        dt_instance = dt()
        dt_instance.dtype = dt
-        with pytest.raises(RecursionError):
+        with pytest.raises(ValueError):


Does this behavior change deserve a release note?

🤷 I don't think recursion errors are a type of error that anyonen should ever run into really.

ngoldbaum

LGTM, just one small comment about whether the changing error type deserves a release note

charris · 2025-11-27T16:26:36Z

Thanks Sebastian.

jorenham · 2025-11-28T04:30:58Z

I guess I'll do the typing stuff for this tomorrow then (in 12 hours or so). I assume that branching 2.4 is still planned for his weekend?

jorenham · 2025-11-28T14:07:28Z

Maybe we could add a __numpy_dtype__ to dtype itself, i.e.e.g. dtype(int8).__numpy_dtype__ == dtype(int8)?

And how about also adding it to the scalar-type classes, e.g.i.e. int8.__numpy_dtype__ == dtype(int8)?

That way we'd be able to simplify numpy._typing._SupportsDType and numpy.typing.DTypeLike, from

# numpy._typing
class _SupportsDType[DTypeT: np.dtype = np.dtype](Protocol):
    @property
    def dtype(self) -> DTypeT: ...

type _VoidDTypeLike = # <omitted spaghetti>
type _DTypeLike[ScalarT: np.generic] = type[ScalarT] | np.dtype[ScalarT] | _SupportsDType[np.dtype[ScalarT]]

# numpy.typing
type DTypeLike =  np.dtype | type | str | _SupportsDType | _VoidDTypeLike

to just

# numpy._typing
class _SupportsDType[DTypeT: np.dtype = np.dtype](Protocol):
    @property
    def __numpy_dtype__(self) -> DTypeT: ...

type _VoidDTypeLike = # <omitted spaghetti>
type _DTypeLike[ScalarT: np.generic] = _SupportsDType[np.dtype[ScalarT]]

# numpy.typing
type DTypeLike = type | str | _DTypeLike | _VoidDTypeLike

(this uses Python 3.13+ PEP 696 syntax for aesthetics)

Typing aside, I suppose it'd also be nice if we'd "practice what we preach", analogous to e.g. ndarray.__array__, int.__int__, str.__str__, etc.

seberg · 2025-11-28T14:19:28Z

And how about also adding it to the scalar-type classes

Possible, but is it worth it? You need to make a funky dance with a custom descriptor stuffed into the __dict__ again, because you must error when accessing it from the class...

I do also do like to (in principle!) allow tying scalar types to dtypes that do not allow adding a custom attribute like this (becuase they are defined by a 3rd party). I.e. that still needs something like the sctypeDict "registration".

EDIT: Although, I guess typing wise the last thing might not matter too much, in the sense that I am not sure user dtypes are supported typing wise to begin with...

charris · 2025-11-28T14:23:23Z

I assume that branching 2.4 is still planned for his weekend?

Only if we are ready.

jorenham · 2025-11-28T14:38:19Z

I.e. that still needs something like the sctypeDict "registration".

EDIT: Although, I guess typing wise the last thing might not matter too much, in the sense that I am not sure user dtypes are supported typing wise to begin with...

Yea, that's right. It'd be similar to that ABCMeta.register stuff, that type-checkers also aren't able to deal with. I'm not even sure if there exists a type theory (yes; there's not just one "type-theory") that would allow for such a thing, considering the (arguably) most important static typing axiom, "types don't change" (at runtime), that all type theories share (as far as I'm aware of).

The approach that we currently use to statically determine a scalar type's associated dtype, via its .dtype attribute (with a typing.Protocol). But since it's an instance attribute, and the dtype constructor only accepts scalar classes, that can be pretty awkward to deal with.

jorenham · 2025-11-28T14:39:22Z

Possible, but is it worth it? You need to make a funky dance with a custom descriptor stuffed into the __dict__ again, because you must error when accessing it from the class...

Can't we use _DTypeMeta for that?

seberg · 2025-11-28T15:07:56Z

You are confusing scalar types and dtypes.

jorenham · 2025-11-28T15:10:07Z

You are confusing scalar types and dtypes.

Ah yea, you're right; classic mistake :P

numpy#30179) * MAINT,API: Introduce __numpy_dtype__ and fix dtype attribute recursion This is a lot more complex then I expected. We, correctly, deprecated `.dtype` attribute lookup recursion, however... The code still had a `try/except` that still recursed and that try except carried meaning at least in a weird path of: ``` class mydtype(np.void): dtype = ... ``` Now does that make sense? Maybe not... we also fall back to object in some paths which should have been broken when a dtype attribute existed on a type but it is a property/descriptor. So, this removes the recursion, but adds a check for `__get__` to filter out those cases, this is something we successfully did for other protocols `__array__`, `__array_interface__`, etc. Signed-off-by: Sebastian Berg <sebastianb@nvidia.com> * DOC: Add release note snippet * DOC: Make release note more precise, array-likes don't need the new attr --------- Signed-off-by: Sebastian Berg <sebastianb@nvidia.com>

github-actions bot added the 03 - Maintenance label Nov 8, 2025

seberg added 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes and removed 03 - Maintenance labels Nov 8, 2025

seberg mentioned this pull request Nov 8, 2025

DEP: Deprecate constructing dtypes from any object having a .dtype attribute #25306

Open

DOC: Add release note snippet

fa61765

seberg added 03 - Maintenance 30 - API and removed 56 - Needs Release Note. Needs an entry in doc/release/upcoming_changes labels Nov 8, 2025

timhoffm reviewed Nov 9, 2025

View reviewed changes

DOC: Make release note more precise, array-likes don't need the new attr

78bb3c2

seberg added this to the 2.4.0 release milestone Nov 19, 2025

inakleinbottle reviewed Nov 26, 2025

View reviewed changes

ngoldbaum reviewed Nov 26, 2025

View reviewed changes

ngoldbaum approved these changes Nov 26, 2025

View reviewed changes

charris merged commit e16fc01 into numpy:main Nov 27, 2025
91 of 96 checks passed

seberg deleted the __numpy_dtype__ branch November 28, 2025 07:57

jorenham mentioned this pull request Nov 28, 2025

TYP: __numpy_dtype__ #30321

Merged

jakevdp mentioned this pull request Nov 29, 2025

Add support for NumPy 2.4 __numpy_dtype__ protocol jax-ml/jax#33601

Merged

jorenham mentioned this pull request Dec 20, 2025

NumPy 2.4 __numpy_dtype__ Protocol jorenham/numpy-typing-compat#76

Open

fangchenli mentioned this pull request Dec 31, 2025

BUG: pd.api.types.pandas_dtype raise different exception with numpy=2.4.0 pandas-dev/pandas#63484

Closed

3 tasks

Uh oh!

MAINT,API: Introduce __numpy_dtype__ and fix dtype attribute recursion #30179

MAINT,API: Introduce __numpy_dtype__ and fix dtype attribute recursion #30179

Uh oh!

Conversation

seberg commented Nov 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mattip commented Nov 8, 2025

Uh oh!

seberg commented Nov 9, 2025

Uh oh!

timhoffm Nov 9, 2025

Choose a reason for hiding this comment

Uh oh!

seberg Nov 10, 2025

Choose a reason for hiding this comment

Uh oh!

inakleinbottle left a comment

Choose a reason for hiding this comment

Uh oh!

ngoldbaum Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

seberg Nov 26, 2025

Choose a reason for hiding this comment

Uh oh!

ngoldbaum left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

charris commented Nov 27, 2025

Uh oh!

jorenham commented Nov 28, 2025

Uh oh!

jorenham commented Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seberg commented Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

charris commented Nov 28, 2025

Uh oh!

jorenham commented Nov 28, 2025

Uh oh!

jorenham commented Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

seberg commented Nov 28, 2025

Uh oh!

jorenham commented Nov 28, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

seberg commented Nov 8, 2025 •

edited

Loading

jorenham commented Nov 28, 2025 •

edited

Loading

seberg commented Nov 28, 2025 •

edited

Loading

jorenham commented Nov 28, 2025 •

edited

Loading