Skip to content

refactor: VirtualArrays#3620

Merged
ianna merged 17 commits intoscikit-hep:mainfrom
pfackeldey:refactor_virtual_arrays
Aug 26, 2025
Merged

refactor: VirtualArrays#3620
ianna merged 17 commits intoscikit-hep:mainfrom
pfackeldey:refactor_virtual_arrays

Conversation

@pfackeldey
Copy link
Copy Markdown
Collaborator

This PR does some refactoring for VirtualArrays:

  • VirtualArray -> VirtualNDArray (closes Proposal: Rename VirtualArray in _nplikes.virtual to avoid confusion with ak.Array #3604)
  • add new base class: MaterializeableArray
  • PlaceholderArray implements its own materialize function that raises with a much better error than before; also we get rid of many assertions extra for PlaceholderArrays
  • materialize_if_virtual -> maybe_materialize (because now any MaterializableArray can materialize, not just VirtualNDArrays
  • add strides to the ArrayLike protocol, this solves also some previous typing issues that we ignored with type: ignore

This PR does not touch the TypetracerArray on purpose. If someone wants to add def materialize(self) with touching-behavior to them at some point it could be possible, but I'd prefer to not touch this code path as it is carefully optimized.

@codecov
Copy link
Copy Markdown

codecov bot commented Aug 14, 2025

Codecov Report

❌ Patch coverage is 72.68293% with 56 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.53%. Comparing base (b749e49) to head (e6e8d6f).
⚠️ Report is 407 commits behind head on main.

Files with missing lines Patch % Lines
src/awkward/_nplikes/placeholder.py 25.00% 27 Missing ⚠️
src/awkward/_nplikes/cupy.py 15.38% 11 Missing ⚠️
src/awkward/_nplikes/jax.py 54.54% 5 Missing ⚠️
src/awkward/_nplikes/virtual.py 76.92% 3 Missing ⚠️
src/awkward/_nplikes/array_module.py 96.36% 2 Missing ⚠️
src/awkward/_nplikes/typetracer.py 50.00% 2 Missing ⚠️
src/awkward/forms/form.py 0.00% 2 Missing ⚠️
src/awkward/_kernels.py 75.00% 1 Missing ⚠️
src/awkward/contents/bitmaskedarray.py 66.66% 1 Missing ⚠️
src/awkward/contents/listoffsetarray.py 83.33% 1 Missing ⚠️
... and 1 more
Additional details and impacted files
Files with missing lines Coverage Δ
src/awkward/_backends/dispatch.py 91.66% <100.00%> (-1.20%) ⬇️
src/awkward/_connect/jax/reducers.py 93.87% <100.00%> (+7.67%) ⬆️
src/awkward/_nplikes/__init__.py 68.96% <100.00%> (-12.86%) ⬇️
src/awkward/_nplikes/array_like.py 100.00% <100.00%> (+30.61%) ⬆️
src/awkward/_nplikes/dispatch.py 96.66% <100.00%> (+0.37%) ⬆️
src/awkward/_nplikes/numpy.py 97.61% <100.00%> (+5.51%) ⬆️
src/awkward/_nplikes/numpy_like.py 100.00% <ø> (+24.70%) ⬆️
src/awkward/_util.py 95.55% <100.00%> (+5.55%) ⬆️
src/awkward/contents/bytemaskedarray.py 86.95% <100.00%> (-2.70%) ⬇️
src/awkward/contents/indexedarray.py 84.37% <100.00%> (+3.33%) ⬆️
... and 18 more

... and 168 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Copy Markdown
Member

@ianna ianna left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pfackeldey - Looks great! Please check the review comments. To summarize:

  • strides property: use a proper exception (RuntimeError/NotImplementedError) and clarify why it’s unsupported.
  • materialize method: fix grammar and make the error message more user-friendly, explaining what happened and how to report it.
  • maybe_materialize helper: update docstring, clarify behavior for non-MaterializableArray args, and optionally make it more type-safe.
  • VirtualNDArray conversion: improve readability, separate reason from solution, and consider user- vs. developer-friendly versions.

Overall: focus on clear, helpful error messages and consistent exception types, and ensure that all other documentation is written for end users. Thanks!

@pfackeldey pfackeldey requested a review from ianna August 15, 2025 13:17
@ikrommyd
Copy link
Copy Markdown
Collaborator

Hi @pfackeldey, the recent merges required a couple more VirtualArray -> VirtualNDArray renames + renames in the tests. Took care of that and ci should be passing again now.

Comment on lines +393 to +395
warnings.warn(
"The `VirtualArray` class is deprecated and will be removed in a future release of Awkward Array. "
"Please plan to migrate your code to use the `VirtualNDArray` class instead.",
Copy link
Copy Markdown
Collaborator

@ikrommyd ikrommyd Aug 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not say deprecated here. This message points to a potential difference between the classes and a migration effort required. This should make it clear that it's just a rename and no effort is required apart from a find-replace.

@github-actions
Copy link
Copy Markdown

The documentation preview is ready to be viewed at http://preview.awkward-array.org.s3-website.us-east-1.amazonaws.com/PR3620

@ianna ianna merged commit c7a8c03 into scikit-hep:main Aug 26, 2025
46 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Proposal: Rename VirtualArray in _nplikes.virtual to avoid confusion with ak.Array

3 participants