-
Notifications
You must be signed in to change notification settings - Fork 23
Closed
Labels
bugSomething isn't workingSomething isn't workingregriddingRelating to regridding operationsRelating to regridding operations
Milestone
Description
Foe version 3.14.0 to 3.15.1, regrid operations with Field.regrids or Field.regridc whose regrid axes are chunked with more than one chunk will fail as follows:
>>> obs.data.chunks # axes 1 and 2 are the regrid axes
((322, 322, 322, 322, 164), (322, 38), (322, 322, 76))
>>> obs_regrid = obs.regrids(model, method="linear")
File ~/miniconda3/lib/python3.10/site-packages/dask/array/rechunk.py:307, in rechunk(x, chunks, threshold, block_size_limit, balance, method)
305 if isinstance(chunks, (tuple, list)):
306 chunks = tuple(lc if lc is not None else rc for lc, rc in zip(chunks, x.chunks))
--> 307 chunks = normalize_chunks(
308 chunks, x.shape, limit=block_size_limit, dtype=x.dtype, previous_chunks=x.chunks
309 )
311 # Now chunks are tuple of tuples
312 ndim = x.ndim
File ~/miniconda3/lib/python3.10/site-packages/dask/array/core.py:3128, in normalize_chunks(chunks, shape, limit, dtype, previous_chunks)
3120 raise ValueError(
3121 "Input array has %d dimensions but the supplied "
3122 "chunks has only %d dimensions" % (len(shape), len(chunks))
3123 )
3124 if not all(
3125 c == s or (math.isnan(c) or math.isnan(s))
3126 for c, s in zip(map(sum, chunks), shape)
3127 ):
-> 3128 raise ValueError(
3129 "Chunks do not add up to shape. "
3130 "Got chunks=%s, shape=%s" % (chunks, shape)
3131 )
3133 return tuple(
3134 tuple(int(x) if not math.isnan(x) else np.nan for x in c) for c in chunks
3135 )
ValueError: Chunks do not add up to shape. Got chunks=((322, 322, 322, 322, 164), (-1,), (-1,)), shape=(1452, 360, 720)The problem lies in cf.Data._regrid rechunking the regrid axes to, (-1,) as opposed to the correct -1.
PR to follow.
bewithankit
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't workingregriddingRelating to regridding operationsRelating to regridding operations