Skip to content

Update 5057, ENH: Add a block function for creating stacked block arrays.#7768

Closed
charris wants to merge 1 commit intonumpy:masterfrom
charris:update-5057
Closed

Update 5057, ENH: Add a block function for creating stacked block arrays.#7768
charris wants to merge 1 commit intonumpy:masterfrom
charris:update-5057

Conversation

@charris
Copy link
Member

@charris charris commented Jun 21, 2016

Updates #5057.

Add a block function to the current stacking functions vstack, hstack,
stack. It is similar to Matlab's square bracket stacking functionality
for block matrices.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

matrix or array?

@madphysicist
Copy link
Contributor

This function is very nice. However, it only appears to work in 2D. I realize that making it work in ND will require a lot more work, but it does seem like a good thing to have. Perhaps a recursive solution would work?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably be result.append(block(row)) to support ND. If it is that simple, the docs, examples and tests need to be updated.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just wanted to bump this implementation suggetsion.

@madphysicist
Copy link
Contributor

One test that I think is missing is for mismatched dtypes. Looping over all the possible combinations is perhaps overkill, but at least a check that block(intArray, floatArray) returns a float_ is probably a good idea for regression.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Missing a (TM) ? :-)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've decided to omit it ;)

@madphysicist
Copy link
Contributor

Besides the lack of support for >2D, I like this PR.

@homu
Copy link
Contributor

homu commented Jun 22, 2016

☔ The latest upstream changes (presumably #7268) made this pull request unmergeable. Please resolve the merge conflicts.

Add a block function to the current stacking functions vstack, hstack,
stack. It is similar to Matlab's square bracket stacking functionality
for block matrices.
@charris
Copy link
Member Author

charris commented Jun 22, 2016

I've made a couple of changes. Thinking about it, I don't think we should allow block(A, B, C) notation. We do that in a couple of places in Numpy but I think the recent tendency is to simply use lists or tuples. Allowing it also makes it more difficult if we ever want to add arguments. @njsmith Thoughts?

@ahaldane
Copy link
Member

ahaldane commented Jun 23, 2016

Keeping the outer brackets also makes it look more like np.array, right? You are proposing to add back the outer brackets like

>>> np.block([a,b,c])          # 2d array  (6, 2), "column" stack
>>> np.block([[a,b,c]])        # 2d array  (2, 6), "row" stack
>>> np.block([[a],[b],[c]])    # 2d array  (6, 2), "column" stack

where o = np.ones((2,2)); a,b,c = 1*o, 2*o, 3*o. (remove outer bracket to get current behavior).

Compare that to np.array:

>>> np.array([1,2,3])        # 1d array (3,)
>>> np.array([[1,2,3]])      # 2d "row" array     (1, 3)
>>> np.array([[1],[2],[3]])  # 2d "column" array  (3, 1)

So the last two lines of each case look more alike. I'll also admit that block(a,b,c) stacking in columns was unexpected to me initially because I think of 1d arrays as rows, but I see how it might be more convenient this way because [a],[b],[c] is annoying to write.

@mhvk
Copy link
Contributor

mhvk commented Jun 23, 2016

Sorry to be late to this: my sense would be to, initially at least, limit the possible ways to do the same thing. I think visually it is much clearer if any times arrays are placed next to each other, they are in fact put next to each other in a row in the final array, never below each other in a column. So, I would
np.block([a, b, c]) be equivalent to np.hstack (with the number of braces just indicating the (minimum) number of dimensions one gets out).

Here, I like @ahaldane's analogy with the np.array -- the real difference is that each item can have its own shape. It also seems that, as @madphysicist suggested, one might as well take this to its logical conclusion, and allow any level of lists for as high n-dimensional arrays as one has the patience typing [[[[[[... for. It seems this should not be overly hard to implement by calling np.block recursively (this especially if one does not special-case [a, b, c]).

@ahaldane
Copy link
Member

By the way, the previous mailing list discussion is at

http://thread.gmane.org/gmane.comp.python.numeric.general/58748/

There it still looks up in the air whether to include the outer brackets, but if we keep them it would mean block does the same thing as np.bmat, except in a simpler way and for ndarrays instead of matrices. Personally I probably prefer with, for the reasons above.

@homu
Copy link
Contributor

homu commented Aug 28, 2016

☔ The latest upstream changes (presumably #7985) made this pull request unmergeable. Please resolve the merge conflicts.

@madphysicist
Copy link
Contributor

One more request I would like to propose (possibly for another PR), is to add this to masked arrays as well.

@charris charris added this to the 1.13.0 release milestone Nov 4, 2016
@charris charris removed this from the 1.12.0 release milestone Nov 4, 2016
@charris
Copy link
Member Author

charris commented Nov 4, 2016

Will branch 1.12.x soon, so pushing this off to 1.13.

dstack : Stack arrays in sequence depth wise (along third dimension).
concatenate : Join a sequence of arrays along an existing axis.
vsplit : Split array into a list of multiple sub-arrays vertically.
block : Assemble arrays from blocks.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that this phrasing of the description is by far the best one. "Create block arrays" does not really make much sense without going into a lot more detail.

dstack : Stack arrays in sequence depth wise (along third axis).
concatenate : Join a sequence of arrays along an existing axis.
hsplit : Split array along second axis.
block : Create block arrays.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to Assemble arrays from blocks.

--------
concatenate : Join a sequence of arrays along an existing axis.
split : Split array into a list of multiple sub-arrays of equal size.
block : Create block arrays.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to Assemble arrays from blocks.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in #8886


def block(*arrays):
"""
Create a block array consisting of other arrays.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change to Assemble arrays from blocks consisting of other arrays. Without further explanation, the term "block array" does not have an unambiguous meaning.

Returns
-------
blocked : ndarray
The 2-D array assembled from the given blocks.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hopefully this can be extended to N-D.

@eric-wieser
Copy link
Member

Seems like this should be done in two passes, so as to pre-allocate the destination array, rather than allocating again for each dimension

@eric-wieser
Copy link
Member

@mhvk

one might as well take this to its logical conclusion, and allow any level of lists for as high n-dimensional arrays as one has the patience typing [[[[[[... for

@charris

I don't think we should allow block(A, B, C) notation.

@madphysicist

Besides the lack of support for >2D, I like this PR.

All these points are addressed in #8886

@jakirkham
Copy link
Contributor

jakirkham commented Apr 4, 2017

Sorry to interrupt the discussion here, but am curious to see if this might handle a use case I have. A snippet of an example of where I might like to use block is below. Thanks for your hard work on this.

Details
In [1]: a
Out[1]: 
array([[-10,  -2,   1],
       [  4,  -7,   6]])

In [2]: b
Out[2]: 
array([[ 5, -2,  3],
       [-3, -8,  2],
       [-7,  1, -2],
       [-9, -2, -3]])

In [3]: c
Out[3]: 
array([[-10,   8,   4,  -6,  -1],
       [  1,  -1,  -3,  -2,   6]])

In [4]: d
Out[4]: 
array([[ 1, -2,  0,  6,  1],
       [ 9, -1,  9,  5, -7],
       [-7, -7,  8, -1,  0],
       [-8,  0,  7,  3,  9]])

In [5]: np.concatenate([np.concatenate([a, c], axis=1), np.concatenate([b, d], axis=1)], axis=0)
Out[5]: 
array([[-10,  -2,   1, -10,   8,   4,  -6,  -1],
       [  4,  -7,   6,   1,  -1,  -3,  -2,   6],
       [  5,  -2,   3,   1,  -2,   0,   6,   1],
       [ -3,  -8,   2,   9,  -1,   9,   5,  -7],
       [ -7,   1,  -2,  -7,  -7,   8,  -1,   0],
       [ -9,  -2,  -3,  -8,   0,   7,   3,   9]])

In[6]: np.block([[a, c], [b, d]])
Out[6]: 
array([[-10,  -2,   1, -10,   8,   4,  -6,  -1],
       [  4,  -7,   6,   1,  -1,  -3,  -2,   6],
       [  5,  -2,   3,   1,  -2,   0,   6,   1],
       [ -3,  -8,   2,   9,  -1,   9,   5,  -7],
       [ -7,   1,  -2,  -7,  -7,   8,  -1,   0],
       [ -9,  -2,  -3,  -8,   0,   7,   3,   9]])

@eric-wieser
Copy link
Member

eric-wieser commented Apr 4, 2017

@jakirkham: I'm afraid you're out of luck there - dimensions are concatenated from right to left, so [a, b] will not concatenate.

Of course, if you want to concatenate from left to right, you can just transpose: np.block([[a.T, b.T], [c.T, d.T]]).T there.

You could also write it as, without much benefit over what you already have:

np.block([  # basically hstack
    np.block([[a], [b]]), np.block([[c], [d]])  # basically vstack
])

@jakirkham
Copy link
Contributor

Oops, sorry, I think I had the order flipped in my example. b and c were in the wrong places. Have edited it to fix this issue. Does this now accurately reflect the behavior of block?

@eric-wieser
Copy link
Member

Yes, what you now have would work just fine

@jakirkham
Copy link
Contributor

Awesome, thanks. Sorry for the confusion.

@charris
Copy link
Member Author

charris commented Apr 4, 2017

@eric-wieser I should close this, right?

@charris charris closed this Apr 4, 2017
@charris charris deleted the update-5057 branch April 4, 2017 18:46
eric-wieser added a commit to eric-wieser/numpy that referenced this pull request Apr 20, 2017
eric-wieser added a commit to eric-wieser/numpy that referenced this pull request Apr 20, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

9 participants