Skip to content
Permalink

Comparing changes

Choose two branches to see what’s changed or to start a new pull request. If you need to, you can also or learn more about diff comparisons.

Open a pull request

Create a new pull request by comparing changes across two branches. If you need to, you can also . Learn more about diff comparisons here.
base repository: snakemake/snakemake
Failed to load repositories. Confirm that selected base ref is valid, then try again.
Loading
base: v9.11.3
Choose a base ref
...
head repository: snakemake/snakemake
Failed to load repositories. Confirm that selected head ref is valid, then try again.
Loading
compare: v9.11.4
Choose a head ref
  • 3 commits
  • 22 files changed
  • 4 contributors

Commits on Sep 19, 2025

  1. fix: properly handle temp files in group jobs that are not needed out… 

    …side of the group (do not upload them to storage and delete them early) (#3730)

### Description

<!--Add a description of your PR here-->

### QC
<!-- Make sure that you can tick the boxes below. -->

* [x] The PR contains a test case for the changes or the changes are
already covered by an existing test case.
* [x] The documentation (`docs/`) is updated to reflect the changes or
this is not necessary (e.g. if the change does neither modify the
language nor the behavior or functionalities of Snakemake).


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Temp-file cleanup is now group-aware and recursively handles grouped
jobs, respecting group boundaries.

* **Bug Fixes**
* Prevents removal of temp files still needed by downstream jobs outside
a group, avoiding downstream failures.
* Improved diagnostic logging for temp-file decisions and group
handling.

* **Tests**
* Added an end-to-end test, expected-result assets, and a helper script
to validate group-aware temp-file behavior.

* **Chores**
* Spawned job invocations no longer force a notemp flag, restoring
default temp-file behavior.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
    @johanneskoester @coderabbitai
    johanneskoester and coderabbitai[bot] authored Sep 19, 2025
    Configuration menu
    Copy the full SHA
    3ffd8e1 View commit details
    Browse the repository at this point in the history

  2. fix: Confusion with Overriding input After Snakemake Modularization (#… 

    …3714)

will fix #3713

## 1: Support recursive module modification via `WorkflowModifier`
To make workflows recursively apply modifiers like path prefix and rule
name changes across nested modules, I introduced the following changes:

- src/snakemake/ruleinfo.py:
- Made `RuleInfo` recursively apply `parent_modifier` to adjust paths
and rule names.
- src/snakemake/path_modifier.py:
- Added `inner_modifier` to support chained path transformations in
nested modules.
- src/snakemake/modules.py:
- Modified `get_name_modifier_func` and
`WorkflowModifier.modify_rulename` to support recursive rule renaming.
- Replaced `WorkflowModifier.skip_rule` with `avail_rulename` to ensure
that `rule_whitelist` and `rule_exclude_list` are respected at every
module level.

## 2: Make multiple use statements with specific_rule more reliable and
strict
My understanding is that `rule.name` should be unique in a workflow.
Therefore, a second `use rule from ... with ...` should only be allowed
if the rule was previously imported with the same name via a wildcard
(`use * from ...`) and is now being refined.
- to clarify:
- A given rule from a module can be used **multiple times** in the same
workflow, as long as each usage assigns it a **unique name**.
- However, using the **same rule name** more than once is **not
allowed**, to prevent accidental overwrites, just like any rule rule
defined **out of** module **cannot** be overwrite.
- That’s the restriction behind the “only once” statement -- it applies
per final rule name, not per source rule.

- src/snakemake/rules.py:
- The old logic stored rule dependencies as objects. If a rule was
modified afterward, those changes wouldn't propagate.
- Now, rule dependencies are resolved dynamically by name, ensuring
correct updated rule with parameters modified.
- src/snakemake/modules.py:
- Stricter checks for rules overwriting. `ModuleInfo.use_rules` now
`allow_rule_overwrite` for specific_rule only if it was previously
imported via `use * from ...` and not already customized.
- src/snakemake/workflow.py:
  - Adjusted to work with the new `WorkflowModifier`.

## Other changes:
- Simplify `WorkflowModifier.__init__`. The module’s `__name__` is now
assigned as `WorkflowModifier.namespace`, making workflow development
and debugging easier.


### QC
* [x] The PR contains a test case for the changes or the changes are
already covered by an existing test case.
* [x] The documentation (`docs/`) is updated to reflect the changes or
this is not necessary (e.g. if the change does neither modify the
language nor the behavior or functionalities of Snakemake).


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

* **New Features**
* Multi-stage path modifiers, a public module registry, and improved
name/path/wrapper modifier lifecycle for more predictable module
imports.

* **Bug Fixes**
* Prevents unintended duplicate rule creation from wildcard/name
modifications and enforces explicit overwrite semantics; tighter
path-replacement guards and more reliable dependency resolution.

* **Tests**
* Updated fixtures for renamed, aliased and nested modules/rules to
reflect modifier behavior changes.

* **Documentation**
* Expanded module-import and rule-customization guidance with examples
and conflict-resolution advice.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Johannes Köster <johannes.koester@tu-dortmund.de>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
    @Hocnonsense @johanneskoester @coderabbitai
    3 people authored Sep 19, 2025
    Configuration menu
    Copy the full SHA
    bd94dc1 View commit details
    Browse the repository at this point in the history

  3. chore(main): release 9.11.4 (#3746) 

    🤖 I have created a release *beep* *boop*
---


##
[9.11.4](v9.11.3...v9.11.4)
(2025-09-19)


### Bug Fixes

* Confusion with Overriding input After Snakemake Modularization
([#3714](#3714))
([bd94dc1](bd94dc1))
* properly handle temp files in group jobs that are not needed outside
of the group (do not upload them to storage and delete them early)
([#3730](#3730))
([3ffd8e1](3ffd8e1))

---
This PR was generated with [Release
Please](https://github.com/googleapis/release-please). See
[documentation](https://github.com/googleapis/release-please#release-please).
    @snakemake-bot
    snakemake-bot authored Sep 19, 2025
    Configuration menu
    Copy the full SHA
    889dd6f View commit details
    Browse the repository at this point in the history
Loading