Skip to content

API: Remove The MapIter API from public#24274

Closed
seberg wants to merge 3 commits intonumpy:mainfrom
seberg:mapiter-removal
Closed

API: Remove The MapIter API from public#24274
seberg wants to merge 3 commits intonumpy:mainfrom
seberg:mapiter-removal

Conversation

@seberg
Copy link
Member

@seberg seberg commented Jul 27, 2023

This API is very slow to use compared to normal advanced indexing
(which reaches deep into the much richer internals).
Making this fast is difficult and the only known users is Theano:

  • While probably still in use Theano is end-of-life for a long time.
  • The use-case in Theano would be better served with ufunc.at or
    direct advanced indexing use probably (even if that requires
    wrapping into NumPy arrays). Because it will be vastly faster
    normally.

So remove it (make it private) to allow making the ABI of it not be
a strange mix for backwards compatibility.


It could also make sense to only make the struct almost fully opaque (with some macros to fetch the pointers needed in a version agnostic way). But considering that the only user is Theano, and that this is rather complicated...

Looking it partially, because I think we we need to improve ufunc.at. At that point we won't use the "truly" public portion of the API anymore, ourselves.

seberg added 2 commits July 27, 2023 15:01
This API is very slow to use compared to normal advanced indexing
(which reaches deep into the much richer internals).
Making this fast is difficult and the only known users is Theano:
* While probably still in use Theano is end-of-life for a long time.
* The use-case in Theano would be better served with `ufunc.at` or
  direct advanced indexing use probably (even if that requires
  wrapping into NumPy arrays).  Because it will be vastly faster
  normally.

So remove it (make it private) to allow making the ABI of it not be
a strange mix for backwards compatibility.
@seberg seberg force-pushed the mapiter-removal branch from 8846dda to 7b6ad5b Compare July 27, 2023 13:11
@rkern
Copy link
Member

rkern commented Jul 27, 2023

While Theano per se is EOL, the codebase continues on as Aesara, which is the foundation for PyMC. Worth checking there.

@seberg
Copy link
Member Author

seberg commented Jul 27, 2023

Ouch! Pinged them there. While I think that using np.add.at is good, it does require a newer NumPy version to get good performance (and some paths will still have terrible performance).

Will close it for, I still want to stop using the slow API internally, but probably little point in breaking them just yet.

@seberg seberg closed this Jul 27, 2023
@seberg
Copy link
Member Author

seberg commented Jul 27, 2023

Seems like Aesara is deprecating that code, so I might revive it after all: aesara-devs/aesara#1506

@seberg
Copy link
Member Author

seberg commented Jul 27, 2023

A major reason would be part of lifting the 32 maxargs/maxdims limit in the iterator, given the above, I think we should not let this hold the maxdims/maxarg lifting back.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants