Skip to content

Only one job is submitted locally, even though resources are available #3042

@fgvieira

Description

@fgvieira

Snakemake version

v8.18.2

Describe the bug

When running snakemake locally, only one job is submitted even though there are enough resources for more.

Logs

Initially there are 500Gb, 80 cores and 42 jobs ready:

Resources before job selection: {'mem_mb': 500000, '_cores': 80, '_nodes': 9223372036854775807, '_job_count': 9223372036854775807}                                                                                                                       Ready jobs (42)

and each jobs requires 1Gb and 1 core:

    resources: tmpdir=/scratch/production/temp/, mem=1 GiB, mem_mb=1024, runtime=60, shell_exec=sh

However, only one job is selected to run:

Selected jobs (1)

even though there are still available resources:

Resources after job selection: {'mem_mb': 498976, '_cores': 79, '_nodes': 9223372036854775806, '_job_count': 1}
Execute 1 jobs...

Additional context

Looking into the results from the ILP solver, only one has math.isclose(variable.value(), 1.0) == True.
It happens the same with the greedy scheduler, only one job is selected.

EDIT: the problem is with the new option --max-jobs-per-second introduced on v8.17.0. Since it has no default, it seems it just submits one job at a time.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions