Skip to content

MAINT: Shrink the size of memory allocations to 1 bytes for arrays with size == 0.#15788

Closed
eric-wieser wants to merge 1 commit intonumpy:mainfrom
eric-wieser:reduce-empty-allocation
Closed

MAINT: Shrink the size of memory allocations to 1 bytes for arrays with size == 0.#15788
eric-wieser wants to merge 1 commit intonumpy:mainfrom
eric-wieser:reduce-empty-allocation

Conversation

@eric-wieser
Copy link
Member

Alternative to #15780.

In order to avoid undefined behavior, we need to ensure the generated strides are all zero.

@eric-wieser eric-wieser requested a review from mattip March 20, 2020 17:11
…th `size == 0`.

In order to avoid undefined behavior, we need to ensure the generated strides are all zero.
@eric-wieser eric-wieser force-pushed the reduce-empty-allocation branch from c003daf to c858e66 Compare March 20, 2020 17:28
@eric-wieser
Copy link
Member Author

Test failure is the same as #15789

@mattip
Copy link
Member

mattip commented Mar 21, 2020

I think this also needs to disallow changing the strides when ndarray.size == 0 in PyArray_CheckStrides.

Could maybe add the test from 912aedd with a @pytest.mark.skipif(not NPY_RELAXED_STRIDES_CHECKING ...)

int i;

#if NPY_RELAXED_STRIDES_CHECKING
npy_bool empty = 0;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we use NP_TRUE and NPY_FALSE with npy_bool?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if we can just replace everything with bool now that we require C99... Obviously not in this patch.

@eric-wieser
Copy link
Member Author

Regarding PyArray_CheckStrides, I think you're suggesting

--- numpy\core\src\common\mem_overlap.c	Tue Aug 20 01:12:25 2019
+++ numpy\core\src\common\mem_overlap.c	Sat Mar 21 18:46:06 2020
@@ -666,10 +666,7 @@
 
     for (i = 0; i < nd; i++) {
         if (dims[i] == 0) {
-            /* If the array size is zero, return an empty range */
-            *lower_offset = 0;
-            *upper_offset = 0;
-            return;
+            continue;
         }
         /* Expand either upwards or downwards depending on stride */
         max_axis_offset = strides[i] * (dims[i] - 1);

@mattip
Copy link
Member

mattip commented Mar 21, 2020

I was thinking more of disallowing assignment of anything but 0 to strides. Currently this succeeds:

a = np.zeros((10, 9, 0))
assert a.size == 0
a.strides = (13, 17, 19)

@mattip
Copy link
Member

mattip commented Dec 15, 2021

@eric-wieser any thoughts on moving this forward? The "uhoh" messages are annoying.

@charris charris added the 52 - Inactive Pending author response label Apr 6, 2022
@charris charris closed this Apr 6, 2022
@mattip
Copy link
Member

mattip commented May 9, 2022

Why did we close this instead of merging it? Was something important missing or was it a matter of this fix exposing yet another already-problematic API?

@seberg
Copy link
Member

seberg commented May 9, 2022

I think it was just closed as part of trying to close a few stalled PRs, just reopen if you like. We removed NPY_RELAXED_STRIDES_CHECKING=0 now, so this should be even simpler.

Maybe this can now be simplified to just memsetting strides to 0? I suppose there might be some (weird) ways to abuse run into the UB still, OTOH, I still doubt that the UB matters in practice usually.

@eric-wieser
Copy link
Member Author

Yeah, I think @charris made the reasonable decision to close out all my stalled PRs, as I haven't found much time in the last year or so to pick them back up again

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

03 - Maintenance 52 - Inactive Pending author response

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants