fix: reduce number of tests leaving temporary files behind#4033
fix: reduce number of tests leaving temporary files behind#4033johanneskoester merged 9 commits intosnakemake:mainfrom
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
📝 WalkthroughWalkthroughAdds a public helper prepare_tmpdir to tests, refactors run/tmpdir/no_tmpdir handling and Snakefile/path resolution in tests/common.py, and updates multiple tests to use prepare_tmpdir or capture tmpdir and perform explicit, platform-aware temporary-directory cleanup. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 2 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (2)
tests/common.py (2)
516-520:⚠️ Potential issue | 🔴 CriticalBug:
tmpdiris undefined whenno_tmpdir=Trueand execution fails.When
no_tmpdir=True, thetmpdirvariable is never assigned. If the workflow execution fails (success=False), line 517 callsprint_tree(tmpdir, ...)which will raise aNameErroror passNonecausing unexpected behavior.🐛 Proposed fix
else: if not success: if snakemake_api is not None and exception is not None: snakemake_api.print_exception(exception) - print("Workdir:") - print_tree(tmpdir, exclude=".snakemake/conda") + if not no_tmpdir: + print("Workdir:") + print_tree(tmpdir, exclude=".snakemake/conda") if exception is not None: raise exception🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/common.py` around lines 516 - 520, The code calls print_tree(tmpdir, ...) unconditionally but tmpdir is never assigned when no_tmpdir=True, causing a NameError on failure; update the function to initialize tmpdir = None at the top (or ensure it's always assigned), and guard the print_tree call with a check like "if tmpdir is not None" or "if not no_tmpdir and tmpdir is not None" before printing, keeping the existing exception handling and assert logic intact; reference variables/functions: tmpdir, no_tmpdir, print_tree, exception, success.
529-530:⚠️ Potential issue | 🔴 CriticalBug:
tmpdirundefined in result checking whenno_tmpdir=True.When
no_tmpdir=True, thetmpdirvariable is not assigned, but lines 529-530 use it to constructtargetfile. Ifcheck_results=True(which is the default whenshouldfail=False), this will fail with aNameError.The result checking should use the original
pathwhenno_tmpdir=True.🐛 Proposed fix
if check_results: + workdir = path if no_tmpdir else tmpdir for resultfile in get_expected_files(results_dir): if resultfile in [".gitignore", ".gitkeep"] or not os.path.isfile( os.path.join(results_dir, resultfile) ): # this means tests cannot use directories as output files continue - targetfile = join(tmpdir, resultfile) + targetfile = join(workdir, resultfile) expectedfile = join(results_dir, resultfile)And similarly update line 533 and 536 to use
workdirinstead oftmpdir.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/common.py` around lines 529 - 530, The code builds targetfile and expectedfile using tmpdir even when no_tmpdir=True (so tmpdir is undefined); change the logic in the result-checking block that computes targetfile/expectedfile (and the other uses of tmpdir around the same area) to use workdir (or the original path variable) when no_tmpdir is true — i.e., compute targetfile = join(workdir, resultfile) (or join(path, resultfile) depending on how workdir/path are used earlier) and leave expectedfile = join(results_dir, resultfile) as-is; update the other occurrences mentioned (the lines referenced near the targetfile/expectedfile computation and the later checks that currently reference tmpdir) to use workdir/path so no NameError occurs when no_tmpdir=True.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@tests/common.py`:
- Around line 516-520: The code calls print_tree(tmpdir, ...) unconditionally
but tmpdir is never assigned when no_tmpdir=True, causing a NameError on
failure; update the function to initialize tmpdir = None at the top (or ensure
it's always assigned), and guard the print_tree call with a check like "if
tmpdir is not None" or "if not no_tmpdir and tmpdir is not None" before
printing, keeping the existing exception handling and assert logic intact;
reference variables/functions: tmpdir, no_tmpdir, print_tree, exception,
success.
- Around line 529-530: The code builds targetfile and expectedfile using tmpdir
even when no_tmpdir=True (so tmpdir is undefined); change the logic in the
result-checking block that computes targetfile/expectedfile (and the other uses
of tmpdir around the same area) to use workdir (or the original path variable)
when no_tmpdir is true — i.e., compute targetfile = join(workdir, resultfile)
(or join(path, resultfile) depending on how workdir/path are used earlier) and
leave expectedfile = join(results_dir, resultfile) as-is; update the other
occurrences mentioned (the lines referenced near the targetfile/expectedfile
computation and the later checks that currently reference tmpdir) to use
workdir/path so no NameError occurs when no_tmpdir=True.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: ffee7b94-c34a-4846-8f66-0fe2969025e5
📒 Files selected for processing (2)
tests/common.pytests/tests.py
I think best is to create the tempdir outside of run in these cases, using the tempfile.TemporaryDirectory context manager, and pass it to run via |
johanneskoester
left a comment
There was a problem hiding this comment.
Great! Just see my comment above about using the TemporaryDirectory context manager in case of expected exceptions.
…r by default - factor out test directory setup into helper function - add ability to pass in an existing empty directory - use TemporaryDirectory in cases that xfail due to a decorator
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/common.py`:
- Around line 145-154: Replace the use of the private
tempfile._get_candidate_names() + os.mkdir split with a single atomic
tempfile.mkdtemp() call: when dest_path is falsy, ensure
snakefile_parent_dirname is set (as currently done), then call tempfile.mkdtemp
with a prefix like f"snakemake-{snakefile_parent_dirname}-" and
dir=tempfile.gettempdir() to create and return the directory atomically (assign
result to dest_path) and remove the separate os.mkdir call; update references to
tmpdir accordingly.
In `@tests/tests.py`:
- Around line 2890-2897: Replace the try/except that only removes tmpdir on
AmbiguousRuleException with a try/finally around the call to run(path,
tmpdir=tmpdir) so that shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) is always
executed; keep the except AmbiguousRuleException branch only if you need to
return early (or convert it into a conditional after run), and apply the same
change to the analogous block around the other test at the region covering lines
2931-2935 (i.e., ensure the run(path, tmpdir=tmpdir) invocation is wrapped in
try/finally and tmpdir is removed in the finally block).
- Around line 1785-1792: The test creates a temporary dir (tmpdir = run(...))
then does shell(...) and shutil.rmtree(tmpdir) but if shell(...) raises the
rmtree is skipped; wrap the post-run operations in a try/finally: after tmpdir
is assigned call shell(...) in the try block and ensure shutil.rmtree(tmpdir) is
executed in the finally block (guarding existence/permission errors as needed).
Update the test surrounding symbols tmpdir, run(...), shell(...), and
shutil.rmtree to perform cleanup reliably even on failures.
- Around line 2037-2040: The TemporaryDirectory usage is wrong: change the
prefix to use the resolved Path variable (use path.name, not dpath.name), call
prepare_tmpdir with the temporary directory returned (prepare_tmpdir(tmpdir)
instead of prepare_tmpdir(path) so you prepare the same folder you pass to run),
and pass tmpdir (a str) directly to run as tmpdir=tmpdir (do not use
tmpdir.name). Update the lines referencing dpath, prepare_tmpdir, and run
accordingly.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 1ca609b0-93e0-42db-b2fb-132aec8a0658
📒 Files selected for processing (2)
tests/common.pytests/tests.py
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/tests.py (1)
470-491:⚠️ Potential issue | 🟠 MajorMove these new manual cleanups into
finally.The added
rmtree(...)calls still only run on the happy path. If any intermediaterun(...),os.utime(...), or assertion fails, the tmpdir is left behind anyway, which is exactly the litter this PR is trying to remove.Also applies to: 2563-2568, 2600-2607
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/tests.py` around lines 470 - 491, The test test_script_python currently calls shutil.rmtree(tmpdir, ...) only on the happy path; wrap the setup and all intermediate operations (the run(...) calls, os.utime(...), and assertions) in a try/finally and move the final cleanup call shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) into the finally block so the temporary directory is removed even if a run, os.utime, or assertion raises; apply the same change pattern to the other test blocks noted (the similar rmtree calls around lines 2563-2568 and 2600-2607) and ensure tmpdir is defined before the try so finally can always reference it.
♻️ Duplicate comments (3)
tests/tests.py (3)
2037-2040:⚠️ Potential issue | 🔴 CriticalUse
path.namefor theTemporaryDirectoryprefix.
dpathis the imported helper function, sodpath.namewill raiseAttributeErrorbefore this test reachesrun(). This should use the resolvedPathfrom Line 2037 instead.🔧 Proposed fix
- with tempfile.TemporaryDirectory(prefix=f"snakemake-{dpath.name}") as tmpdir: + with tempfile.TemporaryDirectory(prefix=f"snakemake-{path.name}-") as tmpdir:#!/bin/bash python - <<'PY' from pathlib import Path def dpath(arg): return Path(arg) path = dpath("test_github_issue1158") print("path.name:", path.name) print("hasattr(dpath, 'name'):", hasattr(dpath, "name")) PY🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/tests.py` around lines 2037 - 2040, The TemporaryDirectory prefix is incorrectly using the helper function name dpath (which has no .name) causing AttributeError; change the prefix to use the resolved Path variable path (i.e., use path.name) in the TemporaryDirectory call so the prefix becomes f"snakemake-{path.name}"; update the line that constructs TemporaryDirectory (where TemporaryDirectory(prefix=f"snakemake-{dpath.name}") is used) and leave prepare_tmpdir(path, dest_path=tmpdir) and run(path, ...) unchanged.
1785-1792:⚠️ Potential issue | 🟠 MajorPut the post-run cleanup in
finally.If Line 1791 fails, Line 1792 never runs and this test still leaves litter behind.
🔧 Proposed fix
tmpdir = run( dpath("test_github_issue640"), targets=["Output/FileWithRights"], executor="dryrun", cleanup=False, ) - shell(f"chmod -R u+rwX {tmpdir}") - shutil.rmtree(tmpdir) + try: + shell(f"chmod -R u+rwX {tmpdir}") + finally: + shutil.rmtree(tmpdir)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/tests.py` around lines 1785 - 1792, The test currently assigns tmpdir from run(...), runs shell("chmod ...") and then calls shutil.rmtree(tmpdir) but if the test fails before the rmtree call the temp directory is left behind; wrap the run and shell calls in a try/finally and move the shutil.rmtree(tmpdir) into the finally block so removal always runs; update the block surrounding the tmpdir variable (the run(...) call and the subsequent shell(...) chmod) to use try: ... finally: shutil.rmtree(tmpdir) and guard removal for a non-None tmpdir where necessary.
2890-2898:⚠️ Potential issue | 🟠 MajorGuarantee tmpdir cleanup with
finallyin both exception-path tests.These tests still remove
tmpdironly when the expected control flow succeeds. An unexpected exception, or a failingpytest.raises(...), will skip cleanup and leave the temporary directory behind.🔧 Proposed fix
def test_ambiguousruleexception(): path = dpath("test_ambiguousruleexception") tmpdir = prepare_tmpdir(path) try: - run(path, tmpdir=tmpdir) - except AmbiguousRuleException: - shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) - return - shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) - raise AssertionError("This is an ambiguous case! Should have raised an error...") + with pytest.raises(AmbiguousRuleException): + run(path, tmpdir=tmpdir) + finally: + shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) def test_wildcard_annotatedstrings(): path = dpath("test_wildcard_annotatedstrings") tmpdir = prepare_tmpdir(path, path.name) - with pytest.raises(WorkflowError, match=r"unpack\(\) is not allowed with params"): - run(path, targets=["test.out"], tmpdir=tmpdir) - shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) + try: + with pytest.raises( + WorkflowError, match=r"unpack\(\) is not allowed with params" + ): + run(path, targets=["test.out"], tmpdir=tmpdir) + finally: + shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS)Also applies to: 2931-2935
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/tests.py` around lines 2890 - 2898, The tmpdir cleanup currently only happens inside the try/except and is skipped on unexpected failures; wrap the run(...) call and its except branch in a try/finally so shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) is executed unconditionally—update the test block that uses dpath("test_ambiguousruleexception"), prepare_tmpdir, run, and AmbiguousRuleException to perform cleanup in a finally, and apply the same change to the duplicate test block (the one around lines 2931-2935) to guarantee tmpdir is always removed.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@tests/tests.py`:
- Around line 470-491: The test test_script_python currently calls
shutil.rmtree(tmpdir, ...) only on the happy path; wrap the setup and all
intermediate operations (the run(...) calls, os.utime(...), and assertions) in a
try/finally and move the final cleanup call shutil.rmtree(tmpdir,
ignore_errors=ON_WINDOWS) into the finally block so the temporary directory is
removed even if a run, os.utime, or assertion raises; apply the same change
pattern to the other test blocks noted (the similar rmtree calls around lines
2563-2568 and 2600-2607) and ensure tmpdir is defined before the try so finally
can always reference it.
---
Duplicate comments:
In `@tests/tests.py`:
- Around line 2037-2040: The TemporaryDirectory prefix is incorrectly using the
helper function name dpath (which has no .name) causing AttributeError; change
the prefix to use the resolved Path variable path (i.e., use path.name) in the
TemporaryDirectory call so the prefix becomes f"snakemake-{path.name}"; update
the line that constructs TemporaryDirectory (where
TemporaryDirectory(prefix=f"snakemake-{dpath.name}") is used) and leave
prepare_tmpdir(path, dest_path=tmpdir) and run(path, ...) unchanged.
- Around line 1785-1792: The test currently assigns tmpdir from run(...), runs
shell("chmod ...") and then calls shutil.rmtree(tmpdir) but if the test fails
before the rmtree call the temp directory is left behind; wrap the run and shell
calls in a try/finally and move the shutil.rmtree(tmpdir) into the finally block
so removal always runs; update the block surrounding the tmpdir variable (the
run(...) call and the subsequent shell(...) chmod) to use try: ... finally:
shutil.rmtree(tmpdir) and guard removal for a non-None tmpdir where necessary.
- Around line 2890-2898: The tmpdir cleanup currently only happens inside the
try/except and is skipped on unexpected failures; wrap the run(...) call and its
except branch in a try/finally so shutil.rmtree(tmpdir,
ignore_errors=ON_WINDOWS) is executed unconditionally—update the test block that
uses dpath("test_ambiguousruleexception"), prepare_tmpdir, run, and
AmbiguousRuleException to perform cleanup in a finally, and apply the same
change to the duplicate test block (the one around lines 2931-2935) to guarantee
tmpdir is always removed.
There was a problem hiding this comment.
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
tests/tests.py (1)
471-491:⚠️ Potential issue | 🟠 MajorMove these tmpdir cleanups into
finally.These tests still only call
rmtree()on the success path. If any intermediaterun()or assertion fails, the tmpdir survives, which undercuts the goal of this PR.Also applies to: 2564-2568, 2603-2607
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/tests.py` around lines 471 - 491, The tmpdir teardown is only performed on the success path; wrap the test body that creates and uses tmpdir (the calls to run(..., tmpdir=tmpdir) and assertions that reference outfile_path and outfile_timestamp_orig) in a try/finally and move the shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) call into the finally block so the temporary directory is always removed even if a run() or assertion fails; apply the same change to the other occurrences noted around the blocks that use tmpdir (lines referenced by the runs at the top and the ones at 2564-2568 and 2603-2607).
♻️ Duplicate comments (4)
tests/tests.py (4)
2931-2936:⚠️ Potential issue | 🟠 MajorCleanup still needs a
finallyblock here.If
pytest.raises(...)fails orrun()raises a different exception,shutil.rmtree()is skipped and this test still leaves litter behind.Suggested fix
def test_wildcard_annotatedstrings(): path = dpath("test_wildcard_annotatedstrings") tmpdir = prepare_tmpdir(path, path.name) - with pytest.raises(WorkflowError, match=r"unpack\(\) is not allowed with params"): - run(path, targets=["test.out"], tmpdir=tmpdir) - shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) + try: + with pytest.raises( + WorkflowError, match=r"unpack\(\) is not allowed with params" + ): + run(path, targets=["test.out"], tmpdir=tmpdir) + finally: + shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/tests.py` around lines 2931 - 2936, The test test_wildcard_annotatedstrings leaves the temporary directory behind if pytest.raises or run() fails; wrap the run/assertion and the rmtree call in a try/finally so shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) always executes. Locate the test_wildcard_annotatedstrings function (uses prepare_tmpdir, pytest.raises and run) and move the run(...) call into a try block and perform the cleanup in the finally block to guarantee removal of tmpdir.
1785-1792:⚠️ Potential issue | 🟠 MajorKeep the
chmodcleanup path infinally.If
shell(...)raises here, thermtree()is skipped and this test still leaves litter behind. Letfinallyown cleanup, and tolerate deletion failures there becausechmodis exactly what makes the tree removable.Suggested fix
tmpdir = run( dpath("test_github_issue640"), targets=["Output/FileWithRights"], executor="dryrun", cleanup=False, ) - shell(f"chmod -R u+rwX {tmpdir}") - shutil.rmtree(tmpdir) + try: + shell(f"chmod -R u+rwX {tmpdir}") + finally: + shutil.rmtree(tmpdir, ignore_errors=True)🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/tests.py` around lines 1785 - 1792, The temporary-directory cleanup currently calls shell("chmod -R u+rwX {tmpdir}") and then shutil.rmtree(tmpdir) inline, which can skip removal if shell() raises; move both the chmod and shutil.rmtree call into a finally block so cleanup always runs, applying chmod before attempting rmtree; wrap shutil.rmtree(tmpdir) in a tolerant try/except (catch Exception) to ignore removal failures (since chmod is what enables deletion) and reference the existing tmpdir variable and the calls to run(), shell(), and shutil.rmtree() when making the change.
2890-2899:⚠️ Potential issue | 🔴 CriticalDon't delete the same tmpdir in both
exceptandfinally.On the expected
AmbiguousRuleExceptionpath, the firstrmtree()removes the directory and thefinallyblock immediately tries to remove it again. On non-Windows that second call raisesFileNotFoundError, so the test fails for the wrong reason.Suggested fix
def test_ambiguousruleexception(): path = dpath("test_ambiguousruleexception") tmpdir = prepare_tmpdir(path) try: - run(path, tmpdir=tmpdir) - except AmbiguousRuleException: - shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) - return + with pytest.raises(AmbiguousRuleException): + run(path, tmpdir=tmpdir) finally: shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) - raise AssertionError("This is an ambiguous case! Should have raised an error...")🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/tests.py` around lines 2890 - 2899, The test currently calls shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) in both the except AmbiguousRuleException handler and the finally block, which can cause a FileNotFoundError on non-Windows; remove the redundant removal from the except block so cleanup is performed only in the finally block (leave the finally's shutil.rmtree as the single cleanup), keeping the try/run(path, tmpdir=tmpdir) call and the except AmbiguousRuleException return behavior intact (references: tmpdir, run(), AmbiguousRuleException, finally, shutil.rmtree, ON_WINDOWS).
2037-2040:⚠️ Potential issue | 🔴 CriticalUse
path.namehere, notdpath.name.
dpathis the helper function, sodpath.nameraises before the test body runs. The prefix needs to come from the resolvedpathobject instead.#!/bin/bash sed -n '2037,2040p' tests/tests.py rg -n -A4 '^def dpath' tests/common.py python - <<'PY' from pathlib import Path def dpath(_): return Path("test_github_issue1158") print("function_has_name_attr:", hasattr(dpath, "name")) print("resolved_path_name:", dpath("x").name) PY🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/tests.py` around lines 2037 - 2040, The test constructs a TemporaryDirectory prefix using the helper function name (dpath.name) which is incorrect because dpath is a function; change the prefix to use the resolved Path object's name (path.name) instead — update the TemporaryDirectory call in the test around run(path, cluster="./qsub.py", tmpdir=tmpdir) so it uses tempfile.TemporaryDirectory(prefix=f"snakemake-{path.name}") (referencing the dpath() helper and the local variable path to locate the change).
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Outside diff comments:
In `@tests/tests.py`:
- Around line 471-491: The tmpdir teardown is only performed on the success
path; wrap the test body that creates and uses tmpdir (the calls to run(...,
tmpdir=tmpdir) and assertions that reference outfile_path and
outfile_timestamp_orig) in a try/finally and move the shutil.rmtree(tmpdir,
ignore_errors=ON_WINDOWS) call into the finally block so the temporary directory
is always removed even if a run() or assertion fails; apply the same change to
the other occurrences noted around the blocks that use tmpdir (lines referenced
by the runs at the top and the ones at 2564-2568 and 2603-2607).
---
Duplicate comments:
In `@tests/tests.py`:
- Around line 2931-2936: The test test_wildcard_annotatedstrings leaves the
temporary directory behind if pytest.raises or run() fails; wrap the
run/assertion and the rmtree call in a try/finally so shutil.rmtree(tmpdir,
ignore_errors=ON_WINDOWS) always executes. Locate the
test_wildcard_annotatedstrings function (uses prepare_tmpdir, pytest.raises and
run) and move the run(...) call into a try block and perform the cleanup in the
finally block to guarantee removal of tmpdir.
- Around line 1785-1792: The temporary-directory cleanup currently calls
shell("chmod -R u+rwX {tmpdir}") and then shutil.rmtree(tmpdir) inline, which
can skip removal if shell() raises; move both the chmod and shutil.rmtree call
into a finally block so cleanup always runs, applying chmod before attempting
rmtree; wrap shutil.rmtree(tmpdir) in a tolerant try/except (catch Exception) to
ignore removal failures (since chmod is what enables deletion) and reference the
existing tmpdir variable and the calls to run(), shell(), and shutil.rmtree()
when making the change.
- Around line 2890-2899: The test currently calls shutil.rmtree(tmpdir,
ignore_errors=ON_WINDOWS) in both the except AmbiguousRuleException handler and
the finally block, which can cause a FileNotFoundError on non-Windows; remove
the redundant removal from the except block so cleanup is performed only in the
finally block (leave the finally's shutil.rmtree as the single cleanup), keeping
the try/run(path, tmpdir=tmpdir) call and the except AmbiguousRuleException
return behavior intact (references: tmpdir, run(), AmbiguousRuleException,
finally, shutil.rmtree, ON_WINDOWS).
- Around line 2037-2040: The test constructs a TemporaryDirectory prefix using
the helper function name (dpath.name) which is incorrect because dpath is a
function; change the prefix to use the resolved Path object's name (path.name)
instead — update the TemporaryDirectory call in the test around run(path,
cluster="./qsub.py", tmpdir=tmpdir) so it uses
tempfile.TemporaryDirectory(prefix=f"snakemake-{path.name}") (referencing the
dpath() helper and the local variable path to locate the change).
…#4033) Currently, many tests in the default test suite fail to clean up after themselves. On CI this doesn't really matter as the VM will be destroyed once the run completes, but it's mildly annoying for local development. This PR currently cleans up the low-hanging fruit: either removing unneeded `cleanup=False` options presumably left over from debugging, or adding explicit cleanup. A few cases are harder to deal with: the following tests cause the call to `run()` to raise an exception: - `test_github_issue1158` - `test_ambiguousruleexception` - `test_wildcard_annotatedstrings` In this case, `run()` never reaches the cleanup code, and the calling function doesn't (and can't) know the temporary path used to be able to clean up. I can see a few options on how to work around this: 1. Ugly hack: make an extra call to `run()` but requesting a no-op target to generate the temporary directory, then pass this in to the current call to `run()` to perform the test; after generating the error, then remove the temporary directory. 2. Move the logic in `run()` that creates the temporary directory to a separate helper function, and call this explicitly from the error-producing tests, then proceed as in (1) above. 3. Refactor the temporary directory functionality to be a class that manages the temporary directory, and deletes it when the instance is destroyed—this could be just directly using `tempfile.TemporaryDirectory`, or reimplementing similar functionality. @johanneskoester Do you have a preference (one of these three, or something else entirely)? ### QC <!-- Make sure that you can tick the boxes below. --> * [ ] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [X] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). In principle one could add a test case that tests do not leave litter, but having tests for your tests seems a little excessive. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **Tests** * Added a reusable temp-directory helper to support both in-place and isolated test modes. * Standardized temp-dir resolution and result-path handling across tests, with clearer behavior when running without a temp dir. * Restored and unified cleanup behavior (including platform-aware handling), ensuring temporary artifacts are removed on success and failure. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
🤖 I have created a release *beep* *boop* --- ## [9.17.0](v9.16.3...v9.17.0) (2026-03-13) ### Features * Allow storing snakemake metadata in files or databases ([#4012](#4012)) ([dd75f31](dd75f31)) * Allow to specify comparison command per-unit test ([#3956](#3956)) ([b88171c](b88171c)) * job table orderd topological when run is started ([#4018](#4018)) ([75cf506](75cf506)) * lambda functions for priority in rules ([#3253](#3253)) ([d2aa226](d2aa226)) * Make on... directive of modules accessible ([#4050](#4050)) ([e9f2e1c](e9f2e1c)) ### Bug Fixes * adjust conda tests to not fail on apple silicon; fix [#4040](#4040) ([#4049](#4049)) ([f5b0142](f5b0142)) * allow "--containerize apptainer" to output apptainer format instead of dockerfile ([#4030](#4030)) ([f5cac30](f5cac30)) * apptainer command not recognized when singularity is absent ([#4010](#4010)) ([b8162e2](b8162e2)) * capture stderr when tests fail ([#3995](#3995)) ([97d74ba](97d74ba)) * **docs:** make Data-dependent conditional execution a complete example ([#4043](#4043)) ([3a1d7f2](3a1d7f2)) * don't build the DAG when running unlock. Fixes [#4000](#4000) and [#198](#198) ([#4007](#4007)) ([acf79fd](acf79fd)) * Ensure pixi tasks may be run as advertised ([#4046](#4046)) ([88253c2](88253c2)) * fix checkpoint handling corner cases ([#3870](#3870) and [#3559](#3559)) ([#4015](#4015)) ([63f4257](63f4257)) * issue 3642 ([#4054](#4054)) ([76e6fc2](76e6fc2)) * issue 3815 ([#4026](#4026)) ([b0eec96](b0eec96)) * logging None in shellcmd context causes error ([#4064](#4064)) ([d0652cd](d0652cd)) * lookup function returns default value for empty DataFrame queries ([#4056](#4056)) ([f71de97](f71de97)) * make `cache: omit-software` a rule specific property ([#4085](#4085)) ([034a9e7](034a9e7)) * reduce number of tests leaving temporary files behind ([#4033](#4033)) ([a3a1c97](a3a1c97)) * regression in dynamic resource handling ([#4038](#4038)) ([f2c554a](f2c554a)) * somewhat shorter announce message ([#4080](#4080)) ([57efc71](57efc71)) ### Performance Improvements * switch reretry with tenacity; decouple container classes (with Python 3.7 compat for old scripts) from rest of the codebase (enabling moving to newer python versions) ([#4032](#4032)) ([ffb19e7](ffb19e7)) ### Documentation * Add AI-assisted contributions policy to contributing guidelines ([#4051](#4051)) ([dd70526](dd70526)) * **codebase:** Update & simplify plugin architecture section ([#4052](#4052)) ([176cf63](176cf63)) * Correct workflow.source_path() description in documentation ([#4036](#4036)) ([45883c5](45883c5)) * fixed wrong code example for collect() function ([#4037](#4037)) ([5c85ed8](5c85ed8)) * Minor docs improvements ([#4089](#4089)) ([29ea226](29ea226)) * switch to sphinx_design for tabs ([#3976](#3976)) ([9674614](9674614)) * typo in the migration table breaking a pip install command ([#4024](#4024)) ([66f9dda](66f9dda)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
Currently, many tests in the default test suite fail to clean up after themselves. On CI this doesn't really matter as the VM will be destroyed once the run completes, but it's mildly annoying for local development.
This PR currently cleans up the low-hanging fruit: either removing unneeded
cleanup=Falseoptions presumably left over from debugging, or adding explicit cleanup.A few cases are harder to deal with: the following tests cause the call to
run()to raise an exception:test_github_issue1158test_ambiguousruleexceptiontest_wildcard_annotatedstringsIn this case,
run()never reaches the cleanup code, and the calling function doesn't (and can't) know the temporary path used to be able to clean up.I can see a few options on how to work around this:
run()but requesting a no-op target to generate the temporary directory, then pass this in to the current call torun()to perform the test; after generating the error, then remove the temporary directory.run()that creates the temporary directory to a separate helper function, and call this explicitly from the error-producing tests, then proceed as in (1) above.tempfile.TemporaryDirectory, or reimplementing similar functionality.@johanneskoester Do you have a preference (one of these three, or something else entirely)?
QC
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).In principle one could add a test case that tests do not leave litter, but having tests for your tests seems a little excessive.
Summary by CodeRabbit