Skip to content

BUG: protect against accessing base attribute of a NULL subarray#19326

Merged
charris merged 4 commits intonumpy:mainfrom
grlee77:dtype-segfault-fix
Jun 25, 2021
Merged

BUG: protect against accessing base attribute of a NULL subarray#19326
charris merged 4 commits intonumpy:mainfrom
grlee77:dtype-segfault-fix

Conversation

@grlee77
Copy link
Contributor

@grlee77 grlee77 commented Jun 24, 2021

closes #19325

This PR is to protect against potential segfaults in void_to_void_resolve_descriptors. This is the first time I have looked at this code, so there may be a better solution, but this seems to work for the example given in #19325

Have PyArray_GetCastSafety return -1 if from is NULL
@grlee77
Copy link
Contributor Author

grlee77 commented Jun 24, 2021

If this seems like a reasonable fix, we can add a new test case for it

@grlee77
Copy link
Contributor Author

grlee77 commented Jun 24, 2021

#19322 makes other changes in the same file, but does not resolve the issue addressed here


PyArray_Descr *from_base = (from_sub == NULL) ? NULL : from_sub->base;
PyArray_Descr *to_base = (to_sub == NULL) ? NULL : to_sub->base;
NPY_CASTING field_casting = PyArray_GetCastSafety(from_base, to_base, NULL);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the right idea, but should be given_descrs[0] or given_descrs[1] rather than NULL. I guess the unstructured void case is missing (i.e. there must be a V4 or similar involved here somehow), I don't know yet how or why view even triggers it...

For tests, it may be good to use also check with np.can_cast (casting="unsafe").

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, I had not realized that in your example x.dtype is "V40", than it all adds up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sure, that works without requiring any change to PyArray_GetCastSafety itself. Let me look into a test case and I will update

@grlee77
Copy link
Contributor Author

grlee77 commented Jun 24, 2021

The following is a minimal can_cast call that segfaults on 1.21.0 and works for this PR (It returns True if I set casting="unsafe", otherwise False)

np.can_cast('V40', np.dtype([('foo', '<f4', (3, 2))]), casting="unsafe")

Cases not involving the shape argument in the dtype seem fine on 1.21.0, e.g.

np.can_cast(
    'V40', np.dtype([('foo', '<i8'), ('bar', [('foo', '<i4'), ('baz', 'u1')])]),  casting="unsafe"
)

does not segfault

@seberg
Copy link
Member

seberg commented Jun 24, 2021

Thanks, this looks good to me. If you are up to it, adding a test for the reverse direction would be good (I know its even stranger).

I am still considering if there may be some incorrect corner cases for subarray -> void casts. The strangest thing that works is probably:

arr = np.ones(3, dtype="i4,(1,2)i4")
arr.astype("i4,V4")

(The crazy part is that we the first element of the subarray, ignoring the rest! I don't want to backport such a change, but maybe we should consider that a bug... If you change it to V8 there, you do not get the second i4 at all)

Copy link
Member

@seberg seberg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, this works for me. There are probably more subtleties we should test, but there is no pressing reason to do it here.

@seberg seberg added this to the 1.21.1 release milestone Jun 24, 2021
@seberg seberg added the 09 - Backport-Candidate PRs tagged should be backported label Jun 24, 2021
@charris charris merged commit 6e9b746 into numpy:main Jun 25, 2021
@charris
Copy link
Member

charris commented Jun 25, 2021

Thanks @grlee77 .

@charris charris removed the 09 - Backport-Candidate PRs tagged should be backported label Jun 25, 2021
@charris charris removed this from the 1.21.1 release milestone Jun 25, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dtype-related segfault when taking a view in NumPy 1.21

3 participants