Skip to content

Prevent builtins.any from being shadowed in dask.array.reductions#11988

Merged
jrbourbeau merged 5 commits intodask:mainfrom
m-albert:fix_quantile_error_axis_none
Jun 12, 2025
Merged

Prevent builtins.any from being shadowed in dask.array.reductions#11988
jrbourbeau merged 5 commits intodask:mainfrom
m-albert:fix_quantile_error_axis_none

Conversation

@m-albert
Copy link
Copy Markdown
Contributor

Fixes #11986.

As @TomAugspurger commented, the shadowing of builtins.any in dask.array.reductions leads to an error when calling

  • da.quantile
  • da.percentile
  • da.nanquantile
  • da.nanpercentile

together with ..., axis=None.

This PR explicitly calls builtins.any where required and adds a minimal test checking that the functions above yield a NotImplementedError in the case of axis=None.

Copy link
Copy Markdown
Member

@TomAugspurger TomAugspurger left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @m-albert. A request for one more test and then I think we're good.

@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 12, 2025

Unit Test Results

See test report for an extended history of previous test failures. This is useful for diagnosing flaky tests.

      9 files  ± 0        9 suites  ±0   3h 19m 28s ⏱️ + 3m 28s
 18 037 tests + 3   16 821 ✅ + 3   1 216 💤 ±0  0 ❌ ±0 
161 407 runs  +27  149 300 ✅ +27  12 107 💤 ±0  0 ❌ ±0 

Results for commit 44a467c. ± Comparison against base commit c37e613.

♻️ This comment has been updated with latest results.

@m-albert
Copy link
Copy Markdown
Contributor Author

m-albert commented Jun 12, 2025

Thanks for reviewing @TomAugspurger and good catch!

I added the test you suggested and needed to take out da.percentile from the test function, as it doesn't take axis as an argument (it had been catching a different NotImplementedError before). Found #2824.

While doing this I found the following:

import dask.array as da
import numpy as np

x_np = np.ones(3)
x_da = da.from_array(x_np)

np_percentile_result = np.percentile(x_np, 50) # np.array(1.)
np_nanpercentile_result = np.nanpercentile(x_np, 50) # np.array(1.)
da_percentile_result = da.percentile(x_da, 50).compute() # np.array(1.)
da_nanpercentile_result = da.nanpercentile(x_da, 50).compute() # np.array([1.])

# ok
assert np_percentile_result.shape == np_nanpercentile_result.shape

# ok
assert np_percentile_result.shape == da_nanpercentile_result.shape

# fails
assert np_percentile_result.shape == da.percentile_result.shape

I.e. da.percentile and da.nanpercentile have different behaviours. Will open a separate issue about this.

Copy link
Copy Markdown
Member

@jrbourbeau jrbourbeau left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the fix @m-albert and for reviewing @TomAugspurger

Also @m-albert I noticed this is your first code contribution to this repository. Welcome!

@jrbourbeau jrbourbeau changed the title Prevent builtins.any from being shadowed in dask.array.reductions Prevent builtins.any from being shadowed in dask.array.reductions Jun 12, 2025
@jrbourbeau jrbourbeau merged commit 20d7d4d into dask:main Jun 12, 2025
24 checks passed
@m-albert
Copy link
Copy Markdown
Contributor Author

Thanks @jrbourbeau !

@m-albert m-albert deleted the fix_quantile_error_axis_none branch June 13, 2025 07:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

da.quantile: AttributeError: 'generator' object has no attribute 'ndim'

3 participants