-
Notifications
You must be signed in to change notification settings - Fork 634
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Snakemake version
v8.5.3
Describe the bug
Temp files are being deleted when they will still be needed. In the example below, the problem seems to be that temp file A.txt is removed after rule b1, but it is still needed by rule a1.
Logs
Assuming unrestricted shared filesystem usage for local execution.
Building DAG of jobs...
Using shell: /usr/bin/bash
Provided cores: 1 (use --cores to define parallelism)
Rules claiming more threads will be scaled down.
Job stats:
job count
----- -------
a 1
all 1
b1 1
b2 1
total 4
Select jobs to execute...
Execute 1 jobs...
[Wed Mar 6 10:04:25 2024]
localrule a:
output: A.txt
jobid: 3
reason: Forced execution
wildcards: name=A
resources: tmpdir=/tmp
[Wed Mar 6 10:04:25 2024]
Finished job 3.
1 of 4 steps (25%) done
Select jobs to execute...
Execute 1 jobs...
[Wed Mar 6 10:04:25 2024]
localrule b1:
input: A.txt
output: A.b1.out
jobid: 2
reason: Forced execution
resources: tmpdir=/tmp
[Wed Mar 6 10:04:25 2024]
Finished job 2.
2 of 4 steps (50%) done
Removing temporary output A.txt.
Select jobs to execute...
Execute 1 jobs...
[Wed Mar 6 10:04:25 2024]
localcheckpoint b2:
input: A.b1.out
output: A.b2.out
jobid: 1
reason: Forced execution
resources: tmpdir=/tmp
DAG of jobs will be updated after completion.
[Wed Mar 6 10:04:25 2024]
Finished job 1.
3 of 4 steps (75%) done
Select jobs to execute...
Execute 1 jobs...
[Wed Mar 6 10:04:25 2024]
localrule a1:
input: A.txt
output: A.a1.out
jobid: 6
reason: Forced execution
wildcards: name=A
resources: tmpdir=/tmp
Waiting at most 5 seconds for missing files.
[...]
raise IOError(
OSError: Missing files after 5 seconds. This might be due to filesystem latency. If that is the case, consider to increase the wait time with --latency-wait:
A.txt
Minimal example
rule a:
output:
temp("{name}.txt"),
shell:
"touch {output}"
rule a1:
input:
"{name}.txt",
output:
temp("{name}.a1.out"),
shell:
"touch {output}"
rule b1:
input:
"A.txt",
output:
temp("A.b1.out"),
shell:
"touch {output}"
checkpoint b2:
input:
"A.b1.out"
output:
"A.b2.out"
shell:
"echo A > {output}"
def in_func1(w):
file = checkpoints.b2.get().output[0]
return open(file, "r").read().strip()
rule all:
input:
lambda w: expand(rules.a1.output, name=in_func1(w)),
default_target: True
Additional context
Maybe related to #2464
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working