Skip to content

feat: adapt __array__ according to numpy guidelines#3592

Merged
ianna merged 11 commits intomainfrom
ikrommyd/fix-dunder-array
Aug 26, 2025
Merged

feat: adapt __array__ according to numpy guidelines#3592
ianna merged 11 commits intomainfrom
ikrommyd/fix-dunder-array

Conversation

@ikrommyd
Copy link
Copy Markdown
Collaborator

@ikrommyd ikrommyd commented Jul 28, 2025

@ikrommyd ikrommyd marked this pull request as ready for review July 30, 2025 01:56
@ikrommyd ikrommyd requested a review from ianna July 30, 2025 01:57
Copy link
Copy Markdown
Member

@ianna ianna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ikrommyd - thanks for the PR—appreciate the enthusiasm! Just a heads-up: I’ve marked it with requested changes so we don’t accidentally merge it before discussing. Going forward, let’s focus on the assigned issues first and loop in a quick chat before tackling anything extra. That way we’re all aligned and can make sure your efforts land exactly where they’re needed.

@pfackeldey
Copy link
Copy Markdown
Collaborator

Thanks @ikrommyd,

this looks good to me!
__array__ will be triggered if anything tries to convert the awkward array to a numpy array implicitly. At that point you can't always use ak.to_numpy (because that may happen in 3rd party libraries, e.g. matplotlib maybe?), thus we have to make sure that __array__ works correctly.

@ianna
Copy link
Copy Markdown
Member

ianna commented Aug 2, 2025

Thanks @ikrommyd,

this looks good to me! __array__ will be triggered if anything tries to convert the awkward array to a numpy array implicitly. At that point you can't always use ak.to_numpy (because that may happen in 3rd party libraries, e.g. matplotlib maybe?), thus we have to make sure that __array__ works correctly.

@pfackeldey - thanks for checking this. Let’s discuss it at our next meeting. Implicit copy is something we want to watch out for or actively prevent.

@ikrommyd
Copy link
Copy Markdown
Collaborator Author

ikrommyd commented Aug 11, 2025

I would like to also add that before this PR, we were copying more than we needed because of this:

    if dtype is None:
        return out
    else:
        return numpy.array(out, dtype=dtype)

That meant that np.asarray(some awkward array with int64 dtype already, dtype=np.int64)would be copied because numpy.array always copies and that's very unnecessary. This statement "either performs a zero-copy conversion or raises an error." in the __array__ docstring was also wrong because it would actually copy more than it should even. Numpy clearly recommends to error only if copy=False and a copy is required. copy=None should only copy if required and never error and copy=True should always copy. This is the expected __array__ behavior from numpy.

@ikrommyd
Copy link
Copy Markdown
Collaborator Author

Needs tests

@ikrommyd
Copy link
Copy Markdown
Collaborator Author

Hi @ianna, I have added tests for both numpy2 and numpy1 because of slight differences regarding where the copy kwarg is available and its possible values.

Copy link
Copy Markdown
Collaborator

@pfackeldey pfackeldey left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Thanks @ikrommyd

@ikrommyd
Copy link
Copy Markdown
Collaborator Author

Anyone know what to do about the docs failure? It looks like it's timing out when reading the chicago taxi parquet dataset.

Copy link
Copy Markdown
Member

@ianna ianna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ikrommyd - thanks for addressing the comments! Looks good to me.

@ikrommyd
Copy link
Copy Markdown
Collaborator Author

Seems like the docs building ci finally didn't crash the last time. Are we good to merge this? 😃

@ianna
Copy link
Copy Markdown
Member

ianna commented Aug 20, 2025

Seems like the docs building ci finally didn't crash the last time. Are we good to merge this? 😃

Please hold on. We’ll do it later

@ianna ianna added the pr-next-release Required for the next release label Aug 21, 2025
@ianna ianna merged commit 99239a8 into main Aug 26, 2025
58 of 60 checks passed
@ianna ianna deleted the ikrommyd/fix-dunder-array branch August 26, 2025 12:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

pr-next-release Required for the next release

Projects

None yet

Development

Successfully merging this pull request may close these issues.

__array__ needs to be updated to accept copy kwarg

3 participants