To be able to automatically restart P2P shuffles, we need some notion of a transaction in the plugin hooks (#7970). This can be implemented by optionally passing an existing stimulus_id to the SchedulerPlugin.transition and SchedulerPlugin.remove_worker hooks.
This would also generally be helpful to track lineage in the plugin hooks. As this is a breaking change to the public API of the SchedulerPlugin, I'm torn between adding the stimulus_id to all hooks once and just making the necessary changes to unblock #7970.
To be able to automatically restart P2P shuffles, we need some notion of a transaction in the plugin hooks (#7970). This can be implemented by optionally passing an existing
stimulus_idto theSchedulerPlugin.transitionandSchedulerPlugin.remove_workerhooks.This would also generally be helpful to track lineage in the plugin hooks. As this is a breaking change to the public API of the
SchedulerPlugin, I'm torn between adding thestimulus_idto all hooks once and just making the necessary changes to unblock #7970.