Snakemake version
8.21.0
Describe the bug
Short summary: When using an existing conda environment together with a checkpoint I get
AttributeError: 'CondaEnvDirSpec' object has no attribute 'file'
Longer description:
I'm using pixi to install environments then point Snakemake to the environment subdirectories stored under .pixi/envs/, e.g. conda: ".pixi/envs/vsearch". This works fine except when the output from a checkpoint is used.
Logs
Below is the snakemake log from running on real Snakemake workflow:
Assuming unrestricted shared filesystem usage.
host: johnsumbp
Building DAG of jobs...
Traceback (most recent call last):
File ".pixi/envs/default/lib/python3.11/site-packages/snakemake/cli.py", line 2095, in args_to_api
dag_api.execute_workflow(
File ".pixi/envs/default/lib/python3.11/site-packages/snakemake/api.py", line 595, in execute_workflow
workflow.execute(
File ".pixi/envs/default/lib/python3.11/site-packages/snakemake/workflow.py", line 1123, in execute
self._build_dag()
File ".pixi/envs/default/lib/python3.11/site-packages/snakemake/workflow.py", line 1072, in _build_dag
async_run(self.dag.update_checkpoint_dependencies())
File ".pixi/envs/default/lib/python3.11/site-packages/snakemake/common/__init__.py", line 89, in async_run
return asyncio.run(coroutine)
^^^^^^^^^^^^^^^^^^^^^^
File ".pixi/envs/default/lib/python3.11/asyncio/runners.py", line 190, in run
return runner.run(main)
^^^^^^^^^^^^^^^^
File ".pixi/envs/default/lib/python3.11/asyncio/runners.py", line 118, in run
return self._loop.run_until_complete(task)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".pixi/envs/default/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
return future.result()
^^^^^^^^^^^^^^^
File ".pixi/envs/default/lib/python3.11/site-packages/snakemake/dag.py", line 1851, in update_checkpoint_dependencies
await self.postprocess_after_update()
File ".pixi/envs/default/lib/python3.11/site-packages/snakemake/dag.py", line 1855, in postprocess_after_update
await self.postprocess()
File ".pixi/envs/default/lib/python3.11/site-packages/snakemake/dag.py", line 1653, in postprocess
self.update_conda_envs()
File ".pixi/envs/default/lib/python3.11/site-packages/snakemake/dag.py", line 316, in update_conda_envs
env_set = {
^
File ".pixi/envs/default/lib/python3.11/site-packages/snakemake/dag.py", line 326, in <setcomp>
and SharedFSUsage.SOFTWARE_DEPLOYMENT
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File ".pixi/envs/default/lib/python3.11/site-packages/snakemake/deployment/conda.py", line 952, in __eq__
return self.path == other.file
^^^^^^^^^^
AttributeError: 'CondaEnvDirSpec' object has no attribute 'file'If I manually change
|
return self.path == other.file |
to
return self.path == other.path
this appears to fix the issue but I don't know if this breaks something else.
Minimal example
- Install a dummy conda environment under
envs/test:
mkdir envs && conda create -y -p envs/test python
- Run Snakemake with attached Snakefile.txt
Command
snakemake -s Snakefile.txt
Snakefile
import os
rule all:
input:
"test_agg.txt"
rule testfile:
output:
"testfile.txt"
conda: "envs/test"
shell:
"""
echo "1. Hello, world!" > {output}
echo "2. Hello, world!" >> {output}
"""
checkpoint test_checkpoint:
input:
"testfile.txt"
output:
directory("test_checkpoint")
shell:
"""
mkdir -p {output}
head -1 {input} > {output}/test.01.txt
tail -n 1 {input} > {output}/test.02.txt
"""
def agg_func(wildcards):
checkpoint_output = checkpoints.test_checkpoint.get(**wildcards).output[0]
return expand(os.path.join(checkpoint_output, "test.{i}.txt"),
i = glob_wildcards(os.path.join(checkpoint_output, "test.{i}.txt")).i)
rule agg:
input:
agg_func
output:
"test_agg.txt"
conda: "envs/test"
shell:
"""
cat {input} > {output}
"""Additional context
In the example above, the error only happens if both the testfile and agg rules have the same conda environment directory specified in the conda directive.
Snakemake version
8.21.0Describe the bug
Short summary: When using an existing conda environment together with a checkpoint I get
Longer description:
I'm using pixi to install environments then point Snakemake to the environment subdirectories stored under
.pixi/envs/, e.g.conda: ".pixi/envs/vsearch". This works fine except when the output from a checkpoint is used.Logs
Below is the snakemake log from running on real Snakemake workflow:
If I manually change
snakemake/snakemake/deployment/conda.py
Line 952 in c1cc605
to
this appears to fix the issue but I don't know if this breaks something else.
Minimal example
envs/test:mkdir envs && conda create -y -p envs/test pythonCommand
Snakefile
Additional context
In the example above, the error only happens if both the
testfileandaggrules have the same conda environment directory specified in the conda directive.