-
-
Notifications
You must be signed in to change notification settings - Fork 757
Open
Description
Currently we have two mechanisms to extend the scheduler and workers:
- Extensions: primarily used by Dask developers to modularize the scheduler and worker systems.
- Plugins: primarily used by Dask users to add new features to Dask from the user-side
Plugins contract
Plugins have an explicit contract and methods like transition, remove_worker, and so on. They are well defined.
Extension conventions
Extensions don't have such a contract but often do a few things by convention:
- Define some new handlers and inject those into the server's handlers
- Add themselves to server.extensions (although this is moved to a contract in Allow worker extensions to piggy-back on heartbeat #5957)
Well, that's pretty much it actually. In #5957 it's proposed to add a heartbeat mechanism .
Nothing here is set or required. Extensions are just classes that hang around. We don't really have any rules around them.
Open questions
- Should we try to merge these two so that there is only one extension mechanism?
- If not, should we try to formalize Extensions a little more?
This is low priority (I think). I'm raising this mostly to capture a conversation that was in a PR but didn't get resolved.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
No labels