fix: don't build the DAG when running unlock. Fixes #4000 and #198#4007
fix: don't build the DAG when running unlock. Fixes #4000 and #198#4007dkoppstein merged 1 commit intosnakemake:mainfrom
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Run ID: 📒 Files selected for processing (4)
💤 Files with no reviewable changes (1)
🚧 Files skipped from review as they are similar to previous changes (1)
📝 WalkthroughWalkthroughWorkflow.unlock no longer calls _build_dag(); it now runs _prepare_dag(...) (if applicable), then directly performs persistence.cleanup_locks() and logging, skipping DAG reconstruction during unlock. Changes
Sequence Diagram(s)mermaid mermaid Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Added a test which is failing, need to think about how to restructure. WIP, do not merge! |
Sounds good! thanks for checking thoroughly |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In `@tests/tests.py`:
- Around line 73-92: Update test_unlock_cli so it proves the unlock path does
not initialize the workflow/DAG: after the initial run that creates
.snakemake/locks, overwrite the Snakefile in the test workspace with a file that
will raise during DAG expansion (e.g., top-level "raise RuntimeError('DAG build
should not run')") so that any code path that initializes the workflow (see
src/snakemake/workflow.py:969-984) would fail; then run the CLI unlock command
and assert it succeeds and cleans the locks. Keep references to the test
function name test_unlock_cli and the problematic workflow initialization in
READMEed code path so reviewers can locate the change.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: 8b58196f-3915-4009-a822-ea459a6bb94e
📒 Files selected for processing (4)
src/snakemake/workflow.pytests/test_unlock/Snakefiletests/test_unlock/expected-results/output.txttests/tests.py
💤 Files with no reviewable changes (1)
- src/snakemake/workflow.py
|
While changing this, I ran into the issue that |
There was a problem hiding this comment.
🧹 Nitpick comments (1)
tests/tests.py (1)
73-92: Use try/finally for cleanup to ensure tmpdir removal on test failure.If the assertion on line 91 fails, the
shutil.rmtreeon line 92 won't execute, leaving the temporary directory behind. Other tests in this file (e.g.,test_github_issue1384,test_github_issue1882) use try/finally for this pattern.♻️ Suggested refactor
def test_unlock_cli(): # run the workflow once to create the .snakemake/locks folder tmpdir = run(dpath("test_unlock"), cleanup=False) - locks = os.path.join(tmpdir, ".snakemake", "locks") - # the lock directory may be cleaned up automatically by Snakemake, - # so ensure there's at least one file inside to simulate a stale lock. - os.makedirs(locks, exist_ok=True) - lockfile = os.path.join(locks, "dummy.lock") - open(lockfile, "w").close() - assert os.listdir(locks), "expected at least one lock file" - - # run the CLI unlock command and verify it succeeds - sp.run( - [sys.executable, "-m", "snakemake", "--unlock"], - cwd=tmpdir, - check=True, - ) - # after unlocking the locks directory should be removed or empty - assert not os.path.exists(locks) or not os.listdir(locks) - shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS) + try: + locks = os.path.join(tmpdir, ".snakemake", "locks") + # the lock directory may be cleaned up automatically by Snakemake, + # so ensure there's at least one file inside to simulate a stale lock. + os.makedirs(locks, exist_ok=True) + Path(locks, "dummy.lock").touch() + assert os.listdir(locks), "expected at least one lock file" + + # run the CLI unlock command and verify it succeeds + sp.run( + [sys.executable, "-m", "snakemake", "--unlock"], + cwd=tmpdir, + check=True, + ) + # after unlocking the locks directory should be removed or empty + assert not os.path.exists(locks) or not os.listdir(locks) + finally: + shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS)Note: The diff also replaces
open(lockfile, "w").close()withPath(...).touch()sincePathis already imported.🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed. In `@tests/tests.py` around lines 73 - 92, The test_unlock_cli function should ensure tmpdir is always removed by wrapping the test body (after tmpdir = run(...)) in a try/finally where the finally calls shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS); also replace open(lockfile, "w").close() with Path(lockfile).touch() since Path is already imported and used elsewhere. Locate the test by the test_unlock_cli function and adjust the run/locks setup and the sp.run(... "--unlock") invocation to live inside the try, leaving only the shutil.rmtree(...) call in the finally block to guarantee cleanup on assertion or other failures.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Nitpick comments:
In `@tests/tests.py`:
- Around line 73-92: The test_unlock_cli function should ensure tmpdir is always
removed by wrapping the test body (after tmpdir = run(...)) in a try/finally
where the finally calls shutil.rmtree(tmpdir, ignore_errors=ON_WINDOWS); also
replace open(lockfile, "w").close() with Path(lockfile).touch() since Path is
already imported and used elsewhere. Locate the test by the test_unlock_cli
function and adjust the run/locks setup and the sp.run(... "--unlock")
invocation to live inside the try, leaving only the shutil.rmtree(...) call in
the finally block to guarantee cleanup on assertion or other failures.
Chiming in here, since I started this in a way :) |
58e219a to
23b6e59
Compare
|
OK, it's ready for merge. @johanneskoester do you think |
|
OK, we decided that |
🤖 I have created a release *beep* *boop* --- ## [9.17.0](v9.16.3...v9.17.0) (2026-03-13) ### Features * Allow storing snakemake metadata in files or databases ([#4012](#4012)) ([dd75f31](dd75f31)) * Allow to specify comparison command per-unit test ([#3956](#3956)) ([b88171c](b88171c)) * job table orderd topological when run is started ([#4018](#4018)) ([75cf506](75cf506)) * lambda functions for priority in rules ([#3253](#3253)) ([d2aa226](d2aa226)) * Make on... directive of modules accessible ([#4050](#4050)) ([e9f2e1c](e9f2e1c)) ### Bug Fixes * adjust conda tests to not fail on apple silicon; fix [#4040](#4040) ([#4049](#4049)) ([f5b0142](f5b0142)) * allow "--containerize apptainer" to output apptainer format instead of dockerfile ([#4030](#4030)) ([f5cac30](f5cac30)) * apptainer command not recognized when singularity is absent ([#4010](#4010)) ([b8162e2](b8162e2)) * capture stderr when tests fail ([#3995](#3995)) ([97d74ba](97d74ba)) * **docs:** make Data-dependent conditional execution a complete example ([#4043](#4043)) ([3a1d7f2](3a1d7f2)) * don't build the DAG when running unlock. Fixes [#4000](#4000) and [#198](#198) ([#4007](#4007)) ([acf79fd](acf79fd)) * Ensure pixi tasks may be run as advertised ([#4046](#4046)) ([88253c2](88253c2)) * fix checkpoint handling corner cases ([#3870](#3870) and [#3559](#3559)) ([#4015](#4015)) ([63f4257](63f4257)) * issue 3642 ([#4054](#4054)) ([76e6fc2](76e6fc2)) * issue 3815 ([#4026](#4026)) ([b0eec96](b0eec96)) * logging None in shellcmd context causes error ([#4064](#4064)) ([d0652cd](d0652cd)) * lookup function returns default value for empty DataFrame queries ([#4056](#4056)) ([f71de97](f71de97)) * make `cache: omit-software` a rule specific property ([#4085](#4085)) ([034a9e7](034a9e7)) * reduce number of tests leaving temporary files behind ([#4033](#4033)) ([a3a1c97](a3a1c97)) * regression in dynamic resource handling ([#4038](#4038)) ([f2c554a](f2c554a)) * somewhat shorter announce message ([#4080](#4080)) ([57efc71](57efc71)) ### Performance Improvements * switch reretry with tenacity; decouple container classes (with Python 3.7 compat for old scripts) from rest of the codebase (enabling moving to newer python versions) ([#4032](#4032)) ([ffb19e7](ffb19e7)) ### Documentation * Add AI-assisted contributions policy to contributing guidelines ([#4051](#4051)) ([dd70526](dd70526)) * **codebase:** Update & simplify plugin architecture section ([#4052](#4052)) ([176cf63](176cf63)) * Correct workflow.source_path() description in documentation ([#4036](#4036)) ([45883c5](45883c5)) * fixed wrong code example for collect() function ([#4037](#4037)) ([5c85ed8](5c85ed8)) * Minor docs improvements ([#4089](#4089)) ([29ea226](29ea226)) * switch to sphinx_design for tabs ([#3976](#3976)) ([9674614](9674614)) * typo in the migration table breaking a pip install command ([#4024](#4024)) ([66f9dda](66f9dda)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
This fixes #4000 and #198 by simply omitting the prepare DAG and build DAG steps in the unlock method in the workflow class.
QC
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).Summary by CodeRabbit
Bug Fixes
Tests
Chores