Skip to content

Unable to provide workers as dict for client.persist() #4551

@Iroy30

Description

@Iroy30

client.persist() API provided an option to provide workers as 'dict' in order to specify which keys/computation needs to be run on which worker. However in PR #4406 this was removed and I now get TypeError: unhashable type "dict'.

To provide more information on what I'm trying to do:
I have each worker mapped to a GPU in my system. When I run dask shuffle, I need partition 0 to got to GPU 0, partition 1 to GPU 1 and so on. Prior to the change I accomplished this by passing a dict mapping keys to intended workers/GPUs in client.persist.

  1. Is this feature permanently removed?
  2. Is there some other way in dask to accomplish what I'm trying to do?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions