Skip to content

Add Computation model to Scheduler #4613

@mrocklin

Description

@mrocklin

Currently the Scheduler state is aware of a hierarchy of computation elements

  • TaskState
  • TaskGroup (often equivalent to a layer in a client-side graph)
  • TaskPrefix (a collection of layers that we think have similar computational characteristics

All of these elements are useful in scheduling heuristics and visual diagnostics.

We might consider adding a fourth class to this system, the Computation. This would be the result of a single compute/persist/map call, and would include a collection of the groups involved, as well as the context of the call (perhaps using the same frame-grabbing tricks that we use in the performance reports to show the code in the first tab).

This would give us additional context that we could use in constructing diagnostics to give back to the user. For example, we might be able to generate a performance_report on the fly for every compute/persist call that they've run during their session.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions