Skip to content

ENH: Use new argument parsing for array creation functions#18642

Merged
mattip merged 5 commits intonumpy:mainfrom
seberg:splitup-faster-argparsing-optimize-asarray
Mar 21, 2021
Merged

ENH: Use new argument parsing for array creation functions#18642
mattip merged 5 commits intonumpy:mainfrom
seberg:splitup-faster-argparsing-optimize-asarray

Conversation

@seberg
Copy link
Copy Markdown
Member

@seberg seberg commented Mar 18, 2021

The array creation functions have the most to gain:

  1. np.asarray is 4 times faster and commonly used.
  2. Other functions are wrapped using array_function in Python
    making it more difficult

This commit (unfortunatly) has to do a few things:

  • Modify array_function C-side dispatching to accomodate
    the fastcall argument convention.
  • Move asarray, etc. to C after removing all "fast paths" from
    np.array (simplifying the code)
  • Fixup imports, since asarray was imported directly in a few places
  • Replace some places where np.array was probably used for speed
    instead of np.asarray or similar. (or by accident in 1 or 2 places)

Sorry, this should have been what was in gh-15270 :(. I split out those PRs from the "full" branch, and apparently picked the wrong commit the last time around.

The array creation functions have the most to gain:

1. np.asarray is 4 times faster and commonly used.
2. Other functions are wrapped using __array_function__ in Python
   making it more difficult

This commit (unfortunatly) has to do a few things:

* Modify __array_function__ C-side dispatching to accomodate
  the fastcall argument convention.
* Move asarray, etc. to C after removing all "fast paths" from
  np.array (simplifying the code)
* Fixup imports, since asarray was imported directly in a few places
* Replace some places where `np.array` was probably used for speed
  instead of np.asarray or similar. (or by accident in 1 or 2 places)
@seberg seberg force-pushed the splitup-faster-argparsing-optimize-asarray branch from 5d01d28 to f531110 Compare March 18, 2021 20:37
@mattip
Copy link
Copy Markdown
Member

mattip commented Mar 18, 2021

Looks good. Do the benchmarks confirm that this is more performant or are they about the same? I am a little concerned about possible unintentional behavior changes in the move from array to asanyarray in numpy/testing/_private/utils.py, are you sure this will not affect downstream users? Their classes may leak through to the actual equal/not-equal comparisons now, no?

@seberg
Copy link
Copy Markdown
Member Author

seberg commented Mar 18, 2021

I could undo those, I did go through fairly carefully, my thought was just that it is more idiomatic and previously was likely a micro-optimization. Those calls had copy=False, subok=True which is the identical meaning to np.asanyarray.

@BvB93
Copy link
Copy Markdown
Member

BvB93 commented Mar 18, 2021

As a request: could you add a # TODO note to _asarray.pyi to move the respective function-annotations to the (to-be created) mutliarray.pyi stub file?

seberg added 3 commits March 18, 2021 19:08
The argument name for the python side defined function was always
`a` and not `object` so fix it and add a test. Also add a test to
ensure the error paths are taken for all of the function.

The missing NULL returns cannot be tested easily (but thanks to
code coverage its nice to look at the uncovered stuff closer...)
I had forgotten that the docs for these functions where updated
when rebasing.
@mattip mattip merged commit 9e47444 into numpy:main Mar 21, 2021
@mattip
Copy link
Copy Markdown
Member

mattip commented Mar 21, 2021

Thanks @seberg

@seberg seberg deleted the splitup-faster-argparsing-optimize-asarray branch June 30, 2021 20:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants