Skip to content

Issue 1003: Multidimensional reshape()#1062

Merged
bbayles merged 15 commits intomore-itertools:masterfrom
rhettinger:reshape
Aug 21, 2025
Merged

Issue 1003: Multidimensional reshape()#1062
bbayles merged 15 commits intomore-itertools:masterfrom
rhettinger:reshape

Conversation

@rhettinger
Copy link
Copy Markdown
Contributor

@rhettinger rhettinger commented Aug 19, 2025

The core use case is to transform one multidimensional array into another multidimensioal array of the same size but with a different shape. The remaining design decisions pertain to handling size mismatches and non-uniform input:

  • The iterator stops when the requested shape has been built or the input has been exhausted, whichever comes first.
  • Accordingly, an empty input gives an empty output.
  • Like collapse, subarrays are not required to have equal sizes. This isn't typical but is easily supported.
  • Unlike collapse, each dimension is presumed to uniformly consist of all arrays or all scalars.
    • If an initial scalar is followed by an iterable, that iterable is treated as a scalar.
    • If an initial iterable is followed by a non-iterable, a TypeError is raised.

The lazily evaluated transformation stack is constructed by a while-loop and a reduce call. The only slow part is the value lookahead and the is_scalar check which are done only once for each dimension in the input.

Once constructed, the transformation stack runs at C speed. It consists of nested calls to islice, batched, and chain. Here is the transformation stack for converting a 3-d structure to a 4-d structure:

# Input shape: 2 x 3 x 2
matrix = (((0, 1), (2, 3), (4, 5)), ((6, 7), (8, 9), (10, 11)))

# Transformation stack
iterator = iter(matrix)  
iterator = chain.from_iterable(chain((next(iterator),), iterator))
iterator = chain.from_iterable(chain((next(iterator),), iterator))
scalar_stream = chain((next(iterator),), iterator)
reshaped = islice(batched(batched(batched(scalar_stream, 2), 1), 1), 6)

# Output shape: 6 x 1 x 1 x 2
print(list(reshaped))

@bbayles bbayles linked an issue Aug 20, 2025 that may be closed by this pull request
@rhettinger
Copy link
Copy Markdown
Contributor Author

@bbayles I think this is ready to go. Can the 15 commits be squashed together to make a single clean commit in the history?

@bbayles bbayles merged commit 21d3d88 into more-itertools:master Aug 21, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Teach reshape() to accept and emit multidimensional arrays

3 participants