Skip to content

DeprecationWarnings when C extension was compiled with older version of NumPy #17318

@octachrome

Description

@octachrome

We have a C extension that defines some custom data types for NumPy arrays. Versions of our extension that were compiled with NumPy 1.16 have started to print warnings when used with NumPy 1.19.0 and later:

<frozen importlib._bootstrap>:219: DeprecationWarning: The ->f->fastputmask member of custom dtypes is ignored; setting it may be an error in the future.
The custom dtype you are using must be revised, but results will not be affected.
<frozen importlib._bootstrap>:219: DeprecationWarning: The ->f->fastclip member of custom dtypes is deprecated; setting it will be an error in the future.
The custom dtype you are using must be changed to use PyUFunc_RegisterLoopForDescr to attach a custom loop to np.core.umath.clip, np.minimum, and np.maximum

But our extension does not use these deprecated features. I believe the warnings are printed due to a bug in PyArray_InitArrFuncs.

We register our data types like this (some lines are omitted):

PyArray_InitArrFuncs (array_funcs);

array_funcs->getitem   = arrfunctions->getitem;
array_funcs->setitem   = arrfunctions->setitem;
array_funcs->copyswapn = arrfunctions->copyswapn;
array_funcs->copyswap  = arrfunctions->copyswap;
array_funcs->nonzero   = arrfunctions->nonzero;

*p_npy_descr = PyArray_DescrNew (object_v);
(*p_npy_descr)->f       = array_funcs;
// etc

*p_dt_index = PyArray_RegisterDataType (*p_npy_descr);

The warnings go away if we explicitly null out the deprecated fields in array_funcs:

array_funcs->fastputmask  = NULL;
array_funcs->fastclip  = NULL;
array_funcs->fasttake  = NULL;

I think what is happening is that PyArray_InitArrFuncs does not null out the deprecated fields, so they are left with whatever random bytes were returned when array_funcs was allocated. Later, NumPy believes that these random values indicate that we have specified a deprecated fastputmask function, etc., and it prints a warning. This in fact means the warnings are intermittent, because malloc sometimes returns zeroed memory by chance.

I believe that PyArray_InitArrFuncs should null out the deprecated fields, so that if a user sets these fields after calling the function, NumPy can correctly determine that the user has intentionally set the deprecated fields and so print a warning, rather than the warning being triggered at random.

Numpy/Python version information:

Extension compiled with:

1.16.6 3.8.5 (default, Sep  4 2020, 02:22:02) 
[Clang 10.0.0 ]

Warnings occur when running with:

1.19.0 3.8.5 (default, Sep  4 2020, 02:22:02) 
[Clang 10.0.0 ]

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions