Skip to content

Design and Discussion for One-off Tasks/Jobs/Whatever #2852

@dperny

Description

@dperny

A highly requested feature in for Swarm Mode is the ability to run one-off operations of some kind. However, the scope of exactly what users need from these one-off jobs is too broad to make satisfactory progress on at the moment. Therefore, this issue is for design, discussion, and sharing of use cases, in order to pin down exactly what features users need.

The goal is to converge on a simple but powerful service mode that accomplishes one-off operations for the large majority of users while not compromising on Swarm's promises of simplicity.

In technical terms, currently, swarm Tasks can enter several different terminal states. Relevant among these is the COMPLETED state, which a Task enters if it exits with exit code zero. Therefore, it should be possible to create a new service mode (in addition to replicated or global) which has special handling for this terminal state in order to run one-off tasks. This would require a new Orchestrator component to handle this new service mode.

Some examples of open questions are below. This is a non-exhaustive list, and you should feel free to bring up anything else in this space.

  • Should one-off tasks be only one task per service, or should you be able to schedule more than one concurrent task as part of the same job?
  • What happens if a one-off task fails? Should it be rescheduled and retried? If there are multiple tasks in a job, what happens if some fail and some succeed? What happens if a task persistently fails? Should there be a failure threshold for when we stop trying?
  • Is the existing docker service CLI command adequate to express the desired behaviors, or is a new CLI command needed? What should it look like?
  • What kind of workloads do you want to run? What do you currently do to solve those use cases?
  • Is cron-style periodic scheduling support a necessary part of this for you?

The discussion in this issue will lead to a full design document for community review before we start building anything.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions