-
Notifications
You must be signed in to change notification settings - Fork 633
cannot set tmpdir default resource on SLURM #2423
Copy link
Copy link
Closed
Labels
bugSomething isn't workingSomething isn't working
Description
Snakemake version
v7.32.3
Describe the bug
I want to set up different temp folders depending on where the workflow is being run and the amount of data. For example:
- when being run locally with small files, I can use the default
/tmp - If I am running it locally with a lot of data, then I need to use a bigger temp folder (e.g.
/scratch) - When running it through SLURM, it should use the nodes local temp (e.g.
/scratch/$SLURM_JOBID).
The 1. and 2. are ok, since I can use --default-resources tmpdir="\"/scratch\"", but for 3. I cannot get it so use the $SLURM_JOBID.
Minimal example
rule all:
output:
"temp.txt"
shell:
"echo {resources.tmpdir} > {output}"
Running snakemake --jobs 1 --default-resources tmpdir="\"/tmp/scratch\"" works fine (even though \tmp\scratch is not deleted at the end); the same with snakemake --jobs 1 --default-resources tmpdir="\"/tmp/scratch\"" --slurm.
But when using an env variable:
snakemake --jobs 1 --default-resources tmpdir="\"/scratch/$SLURM_JOBID\"" --slurm->/scratchsnakemake --jobs 1 --default-resources tmpdir="\"/scratch/\$SLURM_JOBID\"" --slurm->/scratch/$SLURM_JOBID(literal)snakemake --jobs 1 --default-resources tmpdir='\"/scratch/$SLURM_JOBID\"' --slurm->Failed to evaluate default resources value '\\\\"/scratch/$SLURM_JOBID\\\\"'.snakemake --jobs 1 --default-resources tmpdir='/scratch/$SLURM_JOBID' -p --slurm->Failed to evaluate default resources value '/scratch/$SLURM_JOBID'
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
bugSomething isn't workingSomething isn't working