Perfect, LGTM and thanks for the nice new test @jbrockmendel.

booyah, thanks for the quick turnaround. im looking forward to un-xfailing the relevant pandas tests!

Should this be backported?

I would have leaned to not backport, since it is a very long-standing issue and maybe a bit subtle. But if it helps pandas, it should also be OK to do.

im on a hotstreak, want to take a shot at the various .astpye(object) and .item() issues for dt64.

My best guess (figuring out where things are defined in numpy is hard) for a culprit is DATETIME_getitem in arraytypes.c.src, which calls convert_datetime_to_pyobject, which can return an int or datetime or date depending on values.

So I want to create a np.datetime64 object and return it from within convert_datetime_to_pyobject. My best guess here looks something like

NPY_NO_EXPORT PyObject *
convert_datetime_to_pyobject(npy_datetime dt, PyArray_DatetimeMetaData *meta)
{
    PyDatetimeScalarObject* dt64_obj;

    dt64_obj->obval = dt;
    dt64_obj->obmeta = meta;

    return (PyObject*)dt64_obj;
}

but im not seeing any way to actually initialize dt64_obj here. Is there a constructor somewhere?

but im not seeing any way to actually initialize dt64_obj here. Is there a constructor somewhere?

It is not a great function, but you can just use PyArray_Scalar() in getitem (the other function can be deleted). That seems to be the closest we got to a constructor.

The main issue is really all the fallout that might happen afterwards (like things magically working because NumPy converts the datetime to a python one).

Thanks. I successfully got .item() and .astype(object) to work as expected! ... and got a segfault in test_array_coercion. But we're getting somewhere!

minimal reproducer for the segfault:

td = np.timedelta64(123, "ns")
arr = np.array(0)
arr[()] = td  # <- segfault

troubleshooting...