Skip to content

Submitted count is incorrect when aborting a run #6280

@bentsherman

Description

@bentsherman

When a run is aborted (e.g. cancelling a run from Seqera Platform), all submitted/running tasks are killed, marked as "aborted", and the workflow stats are updated accordingly:

void markCompleted(TaskRun task, TraceRecord trace) {
ProgressRecord state = getOrCreateRecord(task.processor)
state.taskName = task.name
state.hash = task.hashLog
state.running --
state.loadCpus -= task.getConfig().getCpus()
state.loadMemory -= (task.getConfig().getMemory()?.toBytes() ?: 0)
this.runningCount --
this.loadCpus -= task.getConfig().getCpus()
this.loadMemory -= (task.getConfig().getMemory()?.toBytes() ?: 0)

However, markCompleted always decrements the running count regardless of whether the completed task was running or submitted. If there are submitted tasks that were aborted, the submitted count will be too high and the running count will be too low (or even negative).

I see two solutions:

  1. Include the previous state in the call to WorkflowStats::markCompleted() so that it knows which count to decrement
  2. Add a function markAborted() and call it before updating the task state in TaskPollingMonitor, so that it can see the previous state.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions