Align auto chunks to provided chunks, rather than shape#3679
Merged
mrocklin merged 1 commit intodask:masterfrom Jun 29, 2018
Merged
Align auto chunks to provided chunks, rather than shape#3679mrocklin merged 1 commit intodask:masterfrom
mrocklin merged 1 commit intodask:masterfrom
Conversation
Previously we used to align "auto" chunks to the shape, even if the old chunks
are not well aligned to the shape. Now we align to the previous chunking if it
was consistent.
Here is an example that would change:
```python
import dask
import dask.array as da
x = da.ones(3000, chunks=512).astype('u1')
with dask.config.set({'array.chunk-size': '1kiB'}):
y = x.rechunk('auto')
>>> y.chunks # Previously
((1000, 1000, 1000),)
>>> y.chunks # New
((1024, 1024, 952),)
```
4 tasks
Member
Author
|
Merging this tomorrow if there are no comments |
convexset
added a commit
to convexset/dask
that referenced
this pull request
Jul 1, 2018
….com/convexset/dask into fix-tsqr-case-chunk-with-zero-height * 'fix-tsqr-case-chunk-with-zero-height' of https://github.com/convexset/dask: fixed typo in documentation and improved clarity Implement .blocks accessor (dask#3689) Fix wrong names (dask#3695) Adds endpoint and retstep support for linspace (dask#3675) Add the @ operator to the delayed objects (dask#3691) Align auto chunks to provided chunks, rather than shape (dask#3679) Adds quotes to source pip install (dask#3678) Prefer end-tasks with low numbers of dependencies when ordering (dask#3588) Reimplement argtopk to release the GIL (dask#3610) Note `da.pad` can be used with `map_overlap` (dask#3672) Allow tasks back onto ordering stack if they have one dependency (dask#3652) Fix extra progressbar (dask#3669) Break apart uneven array-of-int slicing to separate chunks (dask#3648) fix for `dask.array.linalg.tsqr` fails tests (intermittently) with arrays of uncertain dimensions (dask#3662)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Previously we used to align "auto" chunks to the shape, even if the old chunks
are not well aligned to the shape. Now we align to the previous chunking if it
was consistent.
Here is an example that would change:
flake8 dask