Skip to content

map_overlap graph has floating point keys #5824

@chrisroat

Description

@chrisroat

I am using map_overlap and noticed the graph contains floating point keys. I saw this while troubleshooting performance, and I'm not sure if it's related. I thought perhaps the floating point might induce some sort lookup or cache miss. (I think I'm seeing a lot more tasks being computed in my scheduler status window than are reported being done.)

It's simple to see in the following with a small array. And if I make the array dimensionality larger, things get big very fast. But it's the floating points that worry me more than graph size (I understand overlaps are expensive).

# Using dask==2.9.2
import dask.array as da
da.zeros((10,)).map_overlap(lambda x: x, depth=1).visualize()

With shape 10:
download

With shape 10,10
download (1)

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions