Snakemake version

9.14.5

Describe the bug

The greedy scheduler sometimes schedules jobs in parallel such that the total number of threads exceeds the provided number of cores. In my case, I ran snakemake with -c 8, and two jobs using 8 cores each were scheduled, using 16 cores in total.

Minimal example

Create a Conda environment with conda create -n tmp snakemake-minimal=9.14.5.

Create this Snakefile:

rule final:
  input: "file1", "file2"

rule prepare:
  output: "common"
  threads: 8
  shell:
    "echo Using {threads} threads; touch {output}"

rule do_something:
  output: "file{number}"
  input: "common",
  threads: 8
  shell:
    "echo Using {threads} threads; sleep 5; touch {output}"

Run

snakemake -c 8 --scheduler=greedy

The log output shows that both do_something jobs are started in parallel, consuming 16 threads.

Additional context

• If the common input file already exists (that is, if the prepare job does not need to run), then the two do_something jobs are run serially as expected.
• I have not tried with the ILP solver. I use the greedy scheduler because I get Failed to solve scheduling problem with ILP solver, falling back to greedy scheduler.
• The first Snakemake version with the problem is 9.10.0. The preceding release 9.9.0 behaves as expected. Looking at the changelog for 9.10.0, this likely has to do with feat: migrate to scheduler plugin interface and scheduler plugins #3676.
• The machine I am running this on has 16 cores including hyperthreading. If I write threads: 2 in the do_something rule and add "file3" up to "file8" to the input of the "all" rule, the scheduler runs 8 jobs at the same time.