Skip to content

API: Use ResultType in PyArray_ConvertToCommonType#14933

Merged
mattip merged 1 commit intonumpy:masterfrom
seberg:cleanup-converttocommontype
Dec 18, 2019
Merged

API: Use ResultType in PyArray_ConvertToCommonType#14933
mattip merged 1 commit intonumpy:masterfrom
seberg:cleanup-converttocommontype

Conversation

@seberg
Copy link
Member

@seberg seberg commented Nov 19, 2019

This slightly modifies the behaviour of np.choose (practically a
bug fix) and the public function itself. The function is not used within
e.g. SciPy, so the small performance hit of this implementation
is probably insignificant.

The change should help clean up dtypes a bit, since the whole "scalar
cast" logic is brittle and should be deprecated/removed, and this is
probably one of the few places actually using it.

The choose change means that:

np.choose([0], (1000, np.array([1], dtype=np.uint8)))

will actually return a value of 1000 (the dtype not being uint8).


EDIT: To be clear, the "performance hit" is since there can be multiple conversions to array, or an unecessary copy/cast in principle.

@seberg seberg force-pushed the cleanup-converttocommontype branch 2 times, most recently from 60cabfc to 154a6c3 Compare November 19, 2019 00:24
@seberg
Copy link
Member Author

seberg commented Nov 19, 2019

This is a slight API change... SciPy does not use the function at all, we use it in a single place (choose), where I think this can be considered a bug fix. So I tend to think we can just pull this off?

Removing this may allow to remove some "scalar casting" logic, which I am not sure is much code, but is just confusing in any case (since it is a complete second system next to the typical "value based scalar casting/promotion" of result type; even if in principle it is exposed to custom DTypes and as slots, on the dtypes, I would be extremely surprised if anyone uses it).

We do seem to have a few other remnants of this in power and in type promotion (I have to look at those in more detail).

EDIT: OK, the thing in power is just using some defines, not the actual logic. The promotion is a bit strange for user dtypes, but would be phased out automatically when deprecating old user dtype creation functionality (it is also slightly less strange/limited probably).

@seberg seberg force-pushed the cleanup-converttocommontype branch from 154a6c3 to a2bce42 Compare November 19, 2019 00:53
@seberg
Copy link
Member Author

seberg commented Nov 19, 2019

OK, I think I am leaning towards just doing this, but to be honest. Keeping things as they are is probably just a mild annoyance. It is simple logic for builtin dtypes, and for user dtypes we will just phase out the logic in any case.

@charris
Copy link
Member

charris commented Nov 19, 2019

Needs a release note. Might have backwards compatibility implications.

@seberg seberg force-pushed the cleanup-converttocommontype branch from a2bce42 to b56691d Compare November 19, 2019 17:46
@seberg seberg added the 63 - C API Changes or additions to the C API. Mailing list should usually be notified. label Nov 21, 2019
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"in the same way as PyArray_ResultType."

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A mix of scalars and zero-dimensional arrays now produces a type capable of holding the scalar value. Previously priority was given to the dtype of the arrays.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

adpoted these.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe "correct arrays" -> PyArrayObjects

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

clarified the comment.

@mattip
Copy link
Member

mattip commented Nov 22, 2019

+1 for simplifying the code and being more consistent. Does this affect pandas or astropy?

This slightly modifies the behaviour of `np.choose` (practically a
bug fix) and the public function itself. The function is not used within
e.g. SciPy, so the small performance hit of this implementation
is probably insignificant.

The change should help clean up dtypes a bit, since the whole "scalar
cast" logic is brittle and should be deprecated/removed, and this is
probably one of the few places actually using it.

The choose change means that:

```
np.choose([0], (1000, np.array([1], dtype=np.uint8)))
```
will actually return a value of 1000 (the dtype not being uint8).
@seberg seberg force-pushed the cleanup-converttocommontype branch from b56691d to c5da77d Compare November 22, 2019 01:06
@seberg
Copy link
Member Author

seberg commented Nov 22, 2019

Well, they would be affected by the side effect of changing np.choose. But I will assume they will not notice that.

@mattip
Copy link
Member

mattip commented Nov 23, 2019

From what I can tell, pandas tests are unaffected by this change.

@mattip
Copy link
Member

mattip commented Nov 25, 2019

@mhvk do you have an idea if this will affect astropy?

@mattip mattip added the triage review Issue/PR to be discussed at the next triage meeting label Dec 3, 2019
@mattip mattip merged commit 0cea652 into numpy:master Dec 18, 2019
@mattip
Copy link
Member

mattip commented Dec 18, 2019

Thanks @seberg.

@seberg seberg deleted the cleanup-converttocommontype branch December 18, 2019 19:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

03 - Maintenance 63 - C API Changes or additions to the C API. Mailing list should usually be notified. component: numpy._core triage review Issue/PR to be discussed at the next triage meeting

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants