-
-
Notifications
You must be signed in to change notification settings - Fork 1.9k
Description
We would like to enable other people to write high level layers. For example some of the imaging crowd is trying to accomplish this for Map Overlap.
Currently it's not clear what contract needs to be filled to do this well, particularly around efficient culling (it is common in imaging to construct very large dask arrays and then slide very small pieces out of them, so there are great potentials for benefit here). The current cull method has a brief docstring on what is needed, but it's not super clear. In particular, I think that it would be helpful to explain that the result Layer should probably have changed state so that it only generates the correct keys. Also, we should define and give examples for the resulting deps mapping. The current doctest example doesn't include this, and it's not clear what the expected format is.
cc @madsbk @rjzamora if either of you have the time.
@GenevieveBuckley is working on map-overlap and is somewhat blocked on trying to follow the high level graph layer flow