Skip to content

Remove Scheduler.reschedule from public API #7209

@gjoseph92

Description

@gjoseph92

I don't know if Scheduler.reschedule is even considered public API. There's no client method for it. It does show up in Scheduler API docs.

But Scheduler.reschedule presents misleading functionality. It looks like it can be used in general, in any circumstances, to reschedule any task:

"""Reschedule a task
Things may have shifted and this task may now be better suited to run
elsewhere

Reading the implementation, this function should actually only be used in one extremely specific circumstance: when the task has already been released in some way on the worker it's assigned to—either via cancellation or a Reschedule exception—and you are certain the worker will not send any further updates about the task to the scheduler.

Calling reschedule outside of that case can cause a task to be executing on multiple workers at once, and the scheduler only knows about it on one.

This inconsistency creates the need for "unexpected worker completed task" logic, and strange and un-tested transitions like transition_waiting_memory, transition_no_worker_memory, transition_queued_memory.

reschedule is really just a handler for the worker informing the scheduler of a Reschedule exception being raised. I'm not even 100% sure that work stealing's use of it is appropriate.

We should note in the docstring that this shouldn't be called directly.

cc @crusaderky @fjetter

Metadata

Metadata

Assignees

Labels

bugSomething is brokendiscussionDiscussing a topic with no specific actions yetscheduler

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions