feat: multiext support for named input/output#3372
feat: multiext support for named input/output#3372johanneskoester merged 29 commits intosnakemake:mainfrom
Conversation
📝 WalkthroughWalkthroughThis pull request introduces detailed documentation for the use of the Changes
Sequence Diagram(s)sequenceDiagram
participant User as User
participant Rule as Rule Processor
participant M as multiext() Function
participant A as apply_wildcards()
participant P as PathModifier
User->>Rule: Define rule with multiext outputs
Rule->>M: Process multiext(prefix, *extensions, **named_extensions)
M-->>Rule: Return MultiextValue instance(s) or raise error on mix
Rule->>A: Apply wildcards on MultiextValue.prefix
A-->>Rule: Return substituted output path(s)
Rule->>P: Process output path (checks for MultiextValue)
P-->>Rule: Return final output path(s)
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 0
🔭 Outside diff range comments (1)
docs/snakefiles/rules.rst (1)
312-332:⚠️ Potential issueDocumentation for named multiext parameters contains a spelling error.
The documentation clearly explains how to use the new named output feature with multiext. It provides a practical example showing both the syntax and how to reference the named outputs in shell commands.
However, there is a spelling error on line 331:
-Do note that all the multiext extensions should be named, or all of them should be unnamed (not both). Additionaly, if additional input/output statements are given, multiext should be treated as positional arguments (before other named input/output files). +Do note that all the multiext extensions should be named, or all of them should be unnamed (not both). Additionally, if additional input/output statements are given, multiext should be treated as positional arguments (before other named input/output files).🧰 Tools
🪛 GitHub Check: Check for spelling errors
[failure] 331-331:
Additionaly ==> Additionally🪛 GitHub Actions: Codespell
[error] 331-331: codespell: Additionaly ==> Additionally
🧹 Nitpick comments (3)
snakemake/io.py (1)
1439-1449: Consider simplifying theisnamedcheck.
Currently, lines 1445–1447 check whethernameis a string using an if/else block. You could directly returnbool(isinstance(self.name, str)), which mirrors the static analysis hint and simplifies readability.def isnamed(self): - if isinstance(self.name, str): - return True - return False + return bool(isinstance(self.name, str))🧰 Tools
🪛 Ruff (0.8.2)
1445-1447: Return the condition
bool(isinstance(self.name, str))directlyReplace with
return bool(isinstance(self.name, str))(SIM103)
snakemake/workflow.py (1)
1812-1812: Avoid comparing withFalsedirectly.
To comply with best practices and static analysis suggestions, consider using either a truthiness check or a direct identity check.- if not ruleinfo.template_engine and ruleinfo.container_img != False: + if not ruleinfo.template_engine and ruleinfo.container_img:Or, if you truly need to verify that
container_imgis exactlyFalse:- if not ruleinfo.template_engine and ruleinfo.container_img != False: + if not ruleinfo.template_engine and ruleinfo.container_img is not False:🧰 Tools
🪛 Ruff (0.8.2)
1812-1812: Avoid inequality comparisons to
False; useif ruleinfo.container_img:for truth checksReplace with
ruleinfo.container_img(E712)
snakemake/rules.py (1)
1-1: Suggestion: Format code to pass Black check.The pipeline failures indicate that Black formatting check failed with 2 files needing reformatting. Consider running Black on these files to comply with the project's formatting standards.
black snakemake/rules.py snakemake/script/__init__.py🧰 Tools
🪛 GitHub Actions: CI
[error] 1-1: Black formatting check failed. 2 files would be reformatted, 140 files would be left unchanged.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
tests/test_script_xsh/expected-results/test.outis excluded by!**/*.out
📒 Files selected for processing (20)
docs/snakefiles/rules.rst(1 hunks)snakemake/io.py(1 hunks)snakemake/rules.py(2 hunks)tests/test_multiext_named/Snakefile(1 hunks)docs/snakefiles/deployment.rst(2 hunks)docs/snakefiles/reporting.rst(5 hunks)docs/snakefiles/rules.rst(1 hunks)snakemake/cli.py(1 hunks)snakemake/script/__init__.py(3 hunks)snakemake/workflow.py(1 hunks)tests/test_conda_python_3_7_script/Snakefile(1 hunks)tests/test_conda_python_3_7_script/test_script_python_3_7.py(1 hunks)tests/test_conda_run/Snakefile(1 hunks)tests/test_conda_run/expected-results/test.txt(1 hunks)tests/test_conda_run/test_python_env.yaml(1 hunks)tests/test_conda_run/test_script_run.py(1 hunks)tests/test_script_xsh/Snakefile(1 hunks)tests/test_script_xsh/envs/xonsh.yaml(1 hunks)tests/test_script_xsh/scripts/test.xsh(1 hunks)tests/tests_using_conda.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
tests/test_conda_python_3_7_script/test_script_python_3_7.pytests/test_conda_run/test_script_run.pytests/tests_using_conda.pysnakemake/script/__init__.pysnakemake/workflow.pysnakemake/rules.pysnakemake/io.pysnakemake/cli.py
🪛 GitHub Check: Check for spelling errors
docs/snakefiles/rules.rst
[failure] 331-331:
Additionaly ==> Additionally
🪛 GitHub Actions: Codespell
docs/snakefiles/rules.rst
[error] 331-331: codespell: Additionaly ==> Additionally
[error] 331-331: codespell: Additionaly ==> Additionally
🪛 Ruff (0.8.2)
snakemake/workflow.py
1812-1812: Avoid inequality comparisons to False; use if ruleinfo.container_img: for truth checks
Replace with ruleinfo.container_img
(E712)
snakemake/io.py
1445-1447: Return the condition bool(isinstance(self.name, str)) directly
Replace with return bool(isinstance(self.name, str))
(SIM103)
🪛 GitHub Actions: CI
snakemake/rules.py
[error] 1-1: Black formatting check failed. 2 files would be reformatted, 140 files would be left unchanged.
snakemake/io.py
[error] 1-1: Black formatting check failed. 2 files would be reformatted, 140 files would be left unchanged.
🔇 Additional comments (34)
snakemake/io.py (3)
1456-1459: Validation of named extensions looks good.
RaisingWorkflowErrorwhen named extensions include path delimiters ensures correctness and prevents unexpected path handling. No issues detected.
1461-1465: Solid check to enforce consistent usage of extensions.
This logic properly ensures either positional or named extensions are used, but never both at once, aligning with the intended design.
1466-1469: Return statements for positional vs. named extensions are consistent.
The code cleanly returns appropriately flagged lists based on whether extensions are positional or named. Implementation is straightforward and flexible.snakemake/workflow.py (1)
1779-1787: Centralized validation logic is beneficial.
Placingcheck_may_use_software_deploymentinside the decorator promotes clarity and reuse. The logic to disallow these directives whentemplate_engineis set is coherent and prevents contradictory declarations.tests/test_conda_run/expected-results/test.txt (1)
1-1: Output file matches expected value.
This single-line file demonstrates the correct result of the conda-based script (3.0). No issues detected.tests/test_script_xsh/scripts/test.xsh (1)
1-2: Clear and concise script implementation.This script correctly uses Xonsh's variable expansion syntax
@()to access the Snakemake output variable and redirect the "Hello, world!" string to the designated output file. This is a good minimal example to test the Xonsh script execution functionality.tests/test_conda_run/test_script_run.py (1)
1-4: Appropriate implementation for conda environment testing.The script correctly imports numpy and writes the result of
numpy.log2(8)to the output file. This effectively tests both the Python script execution and conda environment integration.tests/test_conda_python_3_7_script/test_script_python_3_7.py (1)
4-4: Minor style change - single quotes to double quotes.The change from single quotes to double quotes for the filename is a minor style adjustment that maintains the same functionality.
tests/test_conda_run/test_python_env.yaml (1)
1-6: Well-structured conda environment definition.This YAML file correctly defines a conda environment with the necessary channels and dependencies. The specification of numpy and python is appropriate for the test case that imports and uses numpy functionality.
tests/test_script_xsh/Snakefile (1)
1-12: Well-structured Snakefile for testing Xonsh script execution.The Snakefile correctly defines:
- An "all" rule that serves as the workflow entry point
- A "test_xonsh" rule that:
- Specifies the output file
- References a conda environment for reproducibility
- Uses the script directive to execute the Xonsh script
This structure follows Snakemake best practices and provides a good test case for the Xonsh script integration.
tests/test_script_xsh/envs/xonsh.yaml (1)
1-5: Looks good!The environment file correctly specifies the necessary channels and dependency for testing Xonsh scripts. The order of channels is also appropriate, with conda-forge first for more up-to-date packages.
tests/test_conda_python_3_7_script/Snakefile (1)
7-7: Script path update looks correctThe change to use "test_script_python_3_7.py" is consistent with testing specific Python version functionality.
tests/test_conda_run/Snakefile (1)
1-13: Rule implementation is appropriate for testingThe rule is well-structured for testing the execution of Python scripts from a conda environment. The comments provide excellent context explaining that this pattern is only for testing and not recommended for regular workflows.
tests/tests_using_conda.py (2)
309-315: New Xonsh script test looks goodThe test function for Xonsh scripts is properly decorated with
@skip_on_windowsand@conda, appropriately testing the functionality with the conda deployment method.
318-320: Conda run test implementation is correctThis test properly verifies the functionality of executing scripts with the conda run directive, following the established testing pattern.
tests/test_multiext_named/Snakefile (3)
1-5: Theallrule demonstrates both positional and named usage of multiext.This rule properly shows both styles of using the multiext function:
- Line 3: Traditional positional parameters
- Line 4: New named parameters feature
6-13: First test rule using positional multiext syntax.This rule demonstrates the traditional multiext usage with positional parameters. The shell commands directly reference the output files by their full names, which is appropriate for positional parameters.
14-20: Second test rule correctly demonstrates the named multiext feature.This rule showcases the new named parameter feature for multiext. The implementation properly:
- Defines extensions with names (out1, out2)
- References these outputs in the shell command using the named syntax:
{output.out1}and{output.out2}This is an excellent test case that validates the core functionality being added in this PR.
docs/snakefiles/rules.rst (1)
1612-1632: New Xonsh script support documentation looks good.The added documentation for Xonsh script support is well-structured and clearly explains how to use Xonsh scripts within Snakemake rules.
snakemake/cli.py (1)
957-965: Documentation improvement for the report feature is clear and helpful.The expanded help text for the
--reportargument provides more detailed information about the report format options and their use cases. The description clarifies when to use HTML (smaller datasets) versus ZIP (larger datasets) formats, and includes instructions for accessing the report.docs/snakefiles/deployment.rst (2)
288-293: Documentation clarification about conda with therundirective is valuable.This update correctly explains that conda environments can now be used with the
rundirective, while clearly highlighting its special nature. The explanation of how thecondadirective only affectsshellfunction calls within therunscript is particularly important for users to understand the execution context.
464-467: Consistent documentation of container behavior withrundirective.This change properly aligns the documentation for container integration with the conda integration documentation, explaining the same constraints when using the
rundirective with thecontainerdirective. This consistency helps users understand the execution model across different deployment methods.snakemake/script/__init__.py (4)
1600-1603: Implementation of XonshScript class looks good.The new
XonshScriptclass inherits fromPythonScriptand simply overrides theexecute_scriptmethod to use the xonsh shell for execution instead of python. This is a clean implementation that follows the existing class hierarchy pattern.
1667-1668: Good addition of file extension recognition for Xonsh scripts.The implementation properly follows the pattern established for other script types by checking for the
.xshextension and assigning the appropriate language identifier.
1732-1732: Correct mapping of 'xonsh' language to XonshScript class.This addition to the executor mapping dictionary correctly connects the 'xonsh' language identifier to the newly created XonshScript class.
1735-1737: Updated error message to include Xonsh scripts.The error message for unsupported script types now includes Xonsh (.xsh) scripts in the list of supported formats, maintaining clear user communication.
snakemake/rules.py (3)
280-290: Implementation of multiext support for named input looks good.The code now properly checks if any items in the input have a 'multiext' flag and processes them accordingly. If the multiext is named, it correctly sets the name using the value from the flag.
347-359: Good implementation of multiext support for named output.The added code includes a helpful comment explaining that if the first multiext is named, all of them are named. The implementation properly checks for multiext flags and processes output files with the correct naming when appropriate.
347-347: Great explanatory comment for multiext behavior.The comment clearly explains the behavior that named multiext values have their name set under the flag, and clarifies that if the first one is named, all of them are named.
docs/snakefiles/reporting.rst (5)
7-10: Documentation improvements enhance clarity of report generation.The updated documentation now better describes the capabilities of Snakemake's HTML reports, explaining that they include runtime statistics, provenance information, and workflow topology by default. The distinction between single HTML files for simpler reports and ZIP archives for complex reports is also clearly explained.
13-16: Well-structured section for including results in reports.This new section heading and organization improves the documentation structure by clearly separating the explanation of how to include results in reports.
265-302: Clear explanation of self-contained report options.The documentation now clearly distinguishes between self-contained HTML files and ZIP archives, with good guidance on when to use each format based on report complexity and size.
304-317: Useful information about partial reports.This section explains how to generate reports for specific targets, which is valuable for users working with intermediate results or alternative targets.
318-330: Helpful guidance on custom report styling.The section on custom layouts with stylesheets provides valuable information for users who want to customize their reports with organization-specific branding.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (5)
docs/snakefiles/deployment.rst (2)
288-293: Enhanced Note on Conda Environments and therunDirectiveThe updated note clearly explains that conda environments can be used with the
shell,script,notebook,wrapper, andrundirectives while highlighting that therundirective is a special case (i.e. only shell functions called within its script are affected). This clarification improves the readability and understanding of how environment directives interact in this context. For even greater clarity, consider adding a brief cross-reference or an example scenario that illustrates this behavior in practice.
464-467: Clarification on Apptainer Integration with therunDirectiveThe note on apptainer integration now succinctly states that it is applicable to the
shell,script,wrapper, andrundirectives and clarifies the special handling of therundirective. One point to double-check: unlike the conda note, thenotebookdirective is not mentioned—if this omission is intentional, it might help to briefly state so; otherwise, consider aligning the two notes for consistency.snakemake/io.py (2)
1439-1450: Simplify the 'isnamed' check using direct boolean evaluation.Lines 1445-1447 rely on an explicit conditional return. Consider returning the condition directly for cleaner code and to address the static analysis hint.
Below is an example fix:
- def isnamed(self): - if isinstance(self.name, str): - return True - return False + def isnamed(self): + return bool(isinstance(self.name, str))🧰 Tools
🪛 Ruff (0.8.2)
1445-1447: Return the condition
bool(isinstance(self.name, str))directlyReplace with
return bool(isinstance(self.name, str))(SIM103)
1456-1469: Validate the prefix for path delimiters as well.While the existing checks correctly prohibit path delimiters in extensions, prefix itself could potentially contain unwanted delimiters. If that’s a concern, you might want to raise a similar error for prefix to guard against paths sneaking into the final output.
def multiext(prefix, *extensions, **named_extensions): + # Optionally, also check for path delimiters in prefix + if "/" in prefix or "\\" in prefix: + raise WorkflowError("Prefix for multiext may not contain path delimiters (/,\).")snakemake/workflow.py (1)
1812-1812: Use truthy checks instead of comparing to False.To conform to Pythonic style, remove the explicit comparison to False and rely on a truthy check.
- if not ruleinfo.template_engine and ruleinfo.container_img != False: + if not ruleinfo.template_engine and ruleinfo.container_img:🧰 Tools
🪛 Ruff (0.8.2)
1812-1812: Avoid inequality comparisons to
False; useif ruleinfo.container_img:for truth checksReplace with
ruleinfo.container_img(E712)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (3)
tests/test_script_xsh/expected-results/test.outis excluded by!**/*.outtests/test_script_xsh/expected-results/test.outis excluded by!**/*.outpyproject.tomlis excluded by!pyproject.toml
📒 Files selected for processing (39)
docs/snakefiles/rules.rst(1 hunks)snakemake/io.py(1 hunks)snakemake/rules.py(2 hunks)docs/snakefiles/deployment.rst(2 hunks)docs/snakefiles/reporting.rst(5 hunks)docs/snakefiles/rules.rst(1 hunks)snakemake/cli.py(1 hunks)snakemake/script/__init__.py(3 hunks)snakemake/workflow.py(1 hunks)tests/test_conda_python_3_7_script/Snakefile(1 hunks)tests/test_conda_python_3_7_script/test_script_python_3_7.py(1 hunks)tests/test_conda_run/Snakefile(1 hunks)tests/test_conda_run/expected-results/test.txt(1 hunks)tests/test_conda_run/test_python_env.yaml(1 hunks)tests/test_conda_run/test_script_run.py(1 hunks)tests/test_script_xsh/Snakefile(1 hunks)tests/test_script_xsh/envs/xonsh.yaml(1 hunks)tests/test_script_xsh/scripts/test.xsh(1 hunks)tests/tests_using_conda.py(1 hunks)snakemake/io.py(1 hunks)snakemake/rules.py(2 hunks)docs/snakefiles/rules.rst(1 hunks)docs/snakefiles/deployment.rst(2 hunks)docs/snakefiles/reporting.rst(5 hunks)docs/snakefiles/rules.rst(1 hunks)snakemake/cli.py(1 hunks)snakemake/script/__init__.py(3 hunks)snakemake/workflow.py(1 hunks)tests/test_conda_python_3_7_script/Snakefile(1 hunks)tests/test_conda_python_3_7_script/test_script_python_3_7.py(1 hunks)tests/test_conda_run/Snakefile(1 hunks)tests/test_conda_run/expected-results/test.txt(1 hunks)tests/test_conda_run/test_python_env.yaml(1 hunks)tests/test_conda_run/test_script_run.py(1 hunks)tests/test_script_xsh/Snakefile(1 hunks)tests/test_script_xsh/envs/xonsh.yaml(1 hunks)tests/test_script_xsh/scripts/test.xsh(1 hunks)tests/tests_using_conda.py(1 hunks)test-environment.yml(2 hunks)
✅ Files skipped from review due to trivial changes (1)
- tests/test_conda_python_3_7_script/test_script_python_3_7.py
🚧 Files skipped from review as they are similar to previous changes (28)
- tests/test_conda_run/expected-results/test.txt
- tests/test_script_xsh/scripts/test.xsh
- tests/test_conda_run/expected-results/test.txt
- tests/test_script_xsh/scripts/test.xsh
- tests/test_conda_python_3_7_script/test_script_python_3_7.py
- tests/test_conda_run/test_script_run.py
- tests/test_conda_run/test_python_env.yaml
- tests/test_conda_run/test_python_env.yaml
- tests/test_conda_run/test_script_run.py
- tests/test_script_xsh/envs/xonsh.yaml
- tests/tests_using_conda.py
- tests/test_conda_python_3_7_script/Snakefile
- tests/test_script_xsh/envs/xonsh.yaml
- docs/snakefiles/rules.rst
- tests/test_script_xsh/Snakefile
- snakemake/io.py
- tests/test_conda_run/Snakefile
- tests/test_script_xsh/Snakefile
- tests/test_conda_run/Snakefile
- snakemake/workflow.py
- snakemake/rules.py
- docs/snakefiles/rules.rst
- tests/tests_using_conda.py
- docs/snakefiles/rules.rst
- snakemake/cli.py
- docs/snakefiles/rules.rst
- docs/snakefiles/reporting.rst
- tests/test_conda_python_3_7_script/Snakefile
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
snakemake/cli.pysnakemake/io.pysnakemake/script/__init__.pysnakemake/rules.pysnakemake/workflow.py
🪛 Ruff (0.8.2)
snakemake/io.py
1445-1447: Return the condition bool(isinstance(self.name, str)) directly
Replace with return bool(isinstance(self.name, str))
(SIM103)
snakemake/workflow.py
1812-1812: Avoid inequality comparisons to False; use if ruleinfo.container_img: for truth checks
Replace with ruleinfo.container_img
(E712)
⏰ Context from checks skipped due to timeout of 90000ms (11)
- GitHub Check: testing-windows (10)
- GitHub Check: testing-windows (8)
- GitHub Check: testing-windows (7)
- GitHub Check: testing-windows (6)
- GitHub Check: testing-windows (5)
- GitHub Check: testing-windows (4)
- GitHub Check: testing-windows (3)
- GitHub Check: testing (1, 3.12, dash)
- GitHub Check: testing-windows (2)
- GitHub Check: testing-windows (1)
- GitHub Check: testing (1, 3.11, bash)
🔇 Additional comments (24)
test-environment.yml (2)
7-7: Good addition of setuptools dependency.Adding setuptools as a dependency ensures consistent environment setup across different systems.
61-62: Proper relocation of cwltool and cwl-utils dependencies.Moving these dependencies from pip to conda-forge channel is a good practice as it ensures better compatibility with the rest of the conda environment.
snakemake/script/__init__.py (4)
1600-1603: Clean implementation of XonshScript class.The XonshScript implementation follows the established pattern by inheriting from PythonScript and only overriding the execute_script method. The approach is simple and effective.
1667-1668: Proper extension of language detection for .xsh files.The addition of the Xonsh file extension detection follows the existing pattern for language identification.
1732-1732: Correct mapping of Xonsh to its executor class.Adding Xonsh to the language-to-executor mapping is implemented correctly.
1736-1737: Updated error message to include Xonsh.The error message for unsupported scripts has been appropriately updated to include Xonsh in the list of supported script types.
snakemake/cli.py (1)
957-966: Excellent update to the report help text!This expanded help text greatly improves user understanding of the report feature by:
- Explaining that reports include statistics, provenance information, and user-specified results
- Clarifying the difference between HTML and ZIP output formats
- Providing guidance on when to use each format based on dataset size
- Including instructions on how to access the report after generation
These changes align perfectly with the updated documentation in
docs/snakefiles/reporting.rst.docs/snakefiles/reporting.rst (14)
7-10: Excellent enhancement to the introduction section!The updated introduction clearly explains the report functionality, highlighting that reports include runtime statistics, provenance information, and workflow topology by default. The distinction between smaller HTML-based reports and more complex ZIP-based reports prepares the reader for the detailed explanations that follow.
13-17: Good addition of an explicit section for including results!Adding a dedicated section with proper RST heading for "Including results in a report" improves the document structure and makes it easier for users to find this critical information.
104-106: Minor formatting improvement for consistency.The change to use proper RST heading formatting (^ characters matching the length of the heading) for the "Defining file labels" section improves document structure consistency.
141-143: Minor formatting improvement for consistency.Similar to the previous heading change, this properly formats the heading for "Determining category, subcategory, and labels dynamically via functions" using the appropriate RST heading level.
148-150: Formatting improvement for consistency.The heading for "Linking between items" is now properly formatted with ^ characters matching the length of the heading text.
151-153: Better subheading structure for improved navigation.The "From captions" subsection is now properly formatted with " characters, creating a clear hierarchy in the document structure.
163-165: Better subheading structure for improved navigation.The "From results" subsection is now properly formatted with " characters to maintain document structure consistency.
257-258: Important addition of a reference anchor.Adding the reference anchor
.. _snakefiles-rendering_reports:enables cross-referencing this section from other parts of the documentation, improving overall navigation.
262-264: Helpful clarification on metadata collection.This new paragraph explains to users how the report metadata is automatically collected, enhancing their understanding of the report generation process.
265-269: Good addition of a reference anchor and subsection.Adding the reference anchor and proper subsection for "Self-contained HTML file" creates a clear document structure and enables cross-referencing, which is used in the introduction.
270-285: Excellent detailed guidance for HTML reports.This new section provides clear instructions for generating basic HTML reports, including examples with default and custom filenames. The note about the limitations of HTML reports for larger outputs helps users make appropriate choices.
286-302: Valuable new section on ZIP archive reports.This section clearly explains when and how to use ZIP archive reports, with practical examples and guidance on accessing the report after generation. This information is crucial for users working with larger workflows.
304-317: Helpful explanation of partial reports.The addition of the "Partial reports" section provides useful information on generating reports for specific targets, which is valuable for workflows that haven't completed or have multiple alternative target rules.
318-331: Great addition of custom layout instructions.The "Custom layout" section provides clear instructions on how to customize report appearance using stylesheets, complete with an example and links to sample code. This enhances the documentation's practical value.
snakemake/workflow.py (1)
1779-1787: Centralized method usage check looks good.This new helper method is well-structured. It uniformly checks whether a software deployment context may be used with template engine restrictions. No further issues found here.
snakemake/rules.py (2)
280-290: Enhancement to handle named multiext input items correctly.The implementation adds proper support for named input files with the 'multiext' flag by recursively checking each file in the collection and conditionally passing the name parameter to
_set_inoutput_itemwhen the multiext is named.
348-359: Enhancement to handle named multiext output items correctly.This implementation mirrors the input handling logic for output files, correctly processing named multiext output flags. The implementation includes helpful comments explaining how named multiext items work, which improves maintainability.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
snakemake/io.py (1)
1439-1449: Consider clarifying thenameattribute type and simplifyingisnamed().Currently, the data class marks
nameasstrbut initializes it toNone. For clearer semantics, it might be preferable to declare it asOptional[str] = None. In addition, you can simplifyisnamed()by returningbool(self.name), since the check is effectively for a non-Nonestring.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
snakemake/io.py(1 hunks)snakemake/io.py(1 hunks)snakemake/io.py(1 hunks)snakemake/path_modifier.py(1 hunks)snakemake/path_modifier.py(1 hunks)snakemake/io.py(1 hunks)snakemake/path_modifier.py(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- snakemake/io.py
🚧 Files skipped from review as they are similar to previous changes (4)
- snakemake/path_modifier.py
- snakemake/path_modifier.py
- snakemake/path_modifier.py
- snakemake/io.py
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
snakemake/io.py
⏰ Context from checks skipped due to timeout of 90000ms (26)
- GitHub Check: testing (10, 3.12, bash)
- GitHub Check: testing (10, 3.11, bash)
- GitHub Check: testing (7, 3.12, bash)
- GitHub Check: testing (7, 3.11, bash)
- GitHub Check: testing (6, 3.12, bash)
- GitHub Check: testing (6, 3.11, bash)
- GitHub Check: testing (5, 3.12, bash)
- GitHub Check: testing-windows (10)
- GitHub Check: testing (5, 3.11, bash)
- GitHub Check: testing-windows (9)
- GitHub Check: testing (4, 3.12, bash)
- GitHub Check: testing-windows (8)
- GitHub Check: testing-windows (7)
- GitHub Check: testing (4, 3.11, bash)
- GitHub Check: testing-windows (6)
- GitHub Check: testing (3, 3.12, bash)
- GitHub Check: testing-windows (5)
- GitHub Check: testing (3, 3.11, bash)
- GitHub Check: testing-windows (4)
- GitHub Check: testing (2, 3.12, bash)
- GitHub Check: testing-windows (3)
- GitHub Check: testing (2, 3.11, bash)
- GitHub Check: testing-windows (2)
- GitHub Check: testing (1, 3.12, bash)
- GitHub Check: testing-windows (1)
- GitHub Check: testing (1, 3.11, bash)
🔇 Additional comments (1)
snakemake/io.py (1)
1450-1469: Robust handling of named/positional extensions.The function correctly ensures that all extensions are either positional or named and raises an error if path delimiters appear or if a mix is encountered. This well-structured approach improves consistency in defining multiple extensions.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
tests/tests.py (1)
1350-1352: Consider adding an assertion or docstring for clarity.While this new test function correctly exercises the
test_multiext_namedscenario through therunfunction, adding a brief docstring or an explicit assertion will help clarify the expected outcome and strengthen test coverage.snakemake/rules.py (1)
347-359: Refactor duplicated logic for handling multiext output files.The logic for expanding named/un-named
'multiext'items is nearly identical to the one used inset_input. To reduce maintenance overhead, consider extracting the repeated parts into a helper method that handles both input and output in a unified way.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (8)
snakemake/rules.py(2 hunks)snakemake/rules.py(2 hunks)snakemake/rules.py(1 hunks)snakemake/rules.py(2 hunks)snakemake/rules.py(2 hunks)snakemake/rules.py(2 hunks)snakemake/rules.py(2 hunks)tests/tests.py(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (6)
- snakemake/rules.py
- snakemake/rules.py
- snakemake/rules.py
- snakemake/rules.py
- snakemake/rules.py
- snakemake/rules.py
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
tests/tests.pysnakemake/rules.py
⏰ Context from checks skipped due to timeout of 90000ms (30)
- GitHub Check: testing (10, 3.12, bash)
- GitHub Check: testing (10, 3.11, bash)
- GitHub Check: testing (9, 3.12, bash)
- GitHub Check: testing (9, 3.11, bash)
- GitHub Check: testing (8, 3.12, bash)
- GitHub Check: testing (8, 3.11, bash)
- GitHub Check: testing (7, 3.12, bash)
- GitHub Check: testing (7, 3.11, bash)
- GitHub Check: testing (6, 3.12, bash)
- GitHub Check: testing (6, 3.11, bash)
- GitHub Check: testing (5, 3.12, bash)
- GitHub Check: testing-windows (10)
- GitHub Check: testing-windows (9)
- GitHub Check: testing (5, 3.11, bash)
- GitHub Check: testing-windows (8)
- GitHub Check: testing (4, 3.12, bash)
- GitHub Check: testing-windows (7)
- GitHub Check: testing (4, 3.11, bash)
- GitHub Check: testing-windows (6)
- GitHub Check: testing (3, 3.12, bash)
- GitHub Check: testing-windows (5)
- GitHub Check: testing (3, 3.11, bash)
- GitHub Check: testing-windows (4)
- GitHub Check: testing (2, 3.12, bash)
- GitHub Check: testing-windows (3)
- GitHub Check: testing (2, 3.11, bash)
- GitHub Check: testing-windows (2)
- GitHub Check: testing (1, 3.12, bash)
- GitHub Check: testing-windows (1)
- GitHub Check: testing (1, 3.11, bash)
🔇 Additional comments (1)
snakemake/rules.py (1)
280-291: Verify handling of non-list inputs.Here, the code iterates over
itemif any sub-element flags contain'multiext'. Ifitemcan sometimes be a string (rather than a list/tuple of files), this loop would iterate over each character of the string, which is likely unintentional. Please confirm thatitemis always an iterable of files; otherwise, consider adding a type check or gracefully handling single-file strings.
|
|
@johanneskoester with the check for MultiextValue in apply_wildcards, replace_prefix and apply_default_storage we can leave the MultiextValue as flag, I guess this is a more appropriate solution ? |
…pply_default_storage
4bc0bf2 to
acf388e
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/snakemake/io.py (1)
1442-1445: Add docstring or inline comment to clarifyMultiextValue.While straightforward, a short inline comment or docstring on this dataclass would enhance clarity for future maintainers, clarifying its intended usage and constraints.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
docs/snakefiles/rules.rst(1 hunks)src/snakemake/io.py(2 hunks)src/snakemake/path_modifier.py(3 hunks)src/snakemake/rules.py(2 hunks)tests/test_multiext_named/Snakefile(1 hunks)tests/tests.py(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- tests/tests.py
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
src/snakemake/rules.pysrc/snakemake/path_modifier.pysrc/snakemake/io.py
🔇 Additional comments (10)
tests/test_multiext_named/Snakefile (2)
1-4: Input correctly uses named multiext extensions.The 'all' rule implements the new named multiext feature correctly, using named extensions 'out1' and 'out2' for the input files.
5-8: Shell command properly references named outputs.The rule correctly defines outputs using the multiext function with named extensions, and the shell command properly references these outputs using the dot notation ({output.out1} and {output.out2}).
src/snakemake/path_modifier.py (3)
10-17: Import of MultiextValue correctly added.The MultiextValue class has been properly imported from snakemake.io, maintaining the alphabetical order of the imports.
69-72: Special handling for MultiextValue in replace_prefix method.The replace_prefix method now correctly handles MultiextValue objects by returning them directly without modification, which preserves their special behavior.
129-132: Special case handling for MultiextValue in apply_default_storage.The apply_default_storage method has been modified to properly handle MultiextValue instances by constructing the query string using path.prefix instead of the path itself. This change aligns with the intention of the MultiextValue implementation.
docs/snakefiles/rules.rst (1)
311-332: Clear documentation for named multiext feature.The documentation clearly explains how to use the named multiext feature with a comprehensive example that demonstrates:
- How to define named extensions in the multiext function
- How to reference these named outputs in the shell command
- How to combine multiext with other types of outputs
The important restriction that "all the multiext extensions should be named, or all of them should be unnamed (not both)" is clearly stated.
src/snakemake/rules.py (2)
280-299: Validate naming collisions for multiext input iterables.You properly assign
_multiextvalue.nameto items flagged as multiext, but be mindful of potential naming collisions if the same name appears inkwinputor other items. It may be worthwhile to verify that no conflicts arise, especially when multiple multiext inputs are declared in the same rule.
355-367: Preserve consistency checks for multiext output.This logic mirrors the input handling, ensuring each extension is iterated and possibly assigned a
namefrom_multiextvalue. The approach is consistent. If you anticipate frequent mixing of named/un-named extensions, consider explicitly documenting or validating that behavior to avoid user confusion.src/snakemake/io.py (2)
1093-1095: Maintain wildcard substitution for MultiextValue.This addition correctly applies wildcard substitution to
pattern.prefixrather than the entireMultiextValueobject. Double-check that callingapply_wildcardson objects flagged for multi-extension cannot break code relying on the old approach.
1447-1476: Ensure prefix validations for mixed usage.This function enforces that all extensions are either named or unnamed. It's well-structured and raises helpful errors. Just confirm that allowing path-like prefixes doesn’t inadvertently introduce partial overlaps with the delimiter checks meant for extensions.
…t with the prefix attribute instead
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/snakemake/io.py (1)
1447-1477: Good implementation of named extensions in multiext function.This implementation adds support for named extensions while maintaining backward compatibility. The function properly validates inputs by checking for path delimiters and ensuring consistent usage patterns (either all named or all positional extensions).
The error message could be slightly improved by providing an example of correct usage:
- raise WorkflowError( - "multiext should be given with all named extensions or all not-named extensions, not a mix." - ) + raise WorkflowError( + "multiext should be given with either all named extensions or all positional extensions, not a mix. " + "For example, either use: multiext('prefix', '.ext1', '.ext2') or multiext('prefix', ext1='.ext1', ext2='.ext2')" + )
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
src/snakemake/io.py(3 hunks)src/snakemake/rules.py(3 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
src/snakemake/io.pysrc/snakemake/rules.py
⏰ Context from checks skipped due to timeout of 90000ms (41)
- GitHub Check: tests (10, windows-latest, py312, bash)
- GitHub Check: tests (10, windows-latest, py311, bash)
- GitHub Check: tests (10, ubuntu-latest, py312, bash)
- GitHub Check: tests (10, ubuntu-latest, py311, bash)
- GitHub Check: tests (9, windows-latest, py312, bash)
- GitHub Check: tests (9, windows-latest, py311, bash)
- GitHub Check: tests (9, ubuntu-latest, py312, bash)
- GitHub Check: tests (9, ubuntu-latest, py311, bash)
- GitHub Check: tests (8, windows-latest, py312, bash)
- GitHub Check: tests (8, windows-latest, py311, bash)
- GitHub Check: tests (8, ubuntu-latest, py312, bash)
- GitHub Check: tests (8, ubuntu-latest, py311, bash)
- GitHub Check: tests (7, windows-latest, py312, bash)
- GitHub Check: tests (7, windows-latest, py311, bash)
- GitHub Check: tests (7, ubuntu-latest, py312, bash)
- GitHub Check: tests (7, ubuntu-latest, py311, bash)
- GitHub Check: tests (6, windows-latest, py312, bash)
- GitHub Check: tests (6, windows-latest, py311, bash)
- GitHub Check: tests (6, ubuntu-latest, py312, bash)
- GitHub Check: tests (6, ubuntu-latest, py311, bash)
- GitHub Check: tests (5, windows-latest, py312, bash)
- GitHub Check: tests (5, windows-latest, py311, bash)
- GitHub Check: tests (5, ubuntu-latest, py312, bash)
- GitHub Check: tests (5, ubuntu-latest, py311, bash)
- GitHub Check: tests (4, windows-latest, py312, bash)
- GitHub Check: tests (4, windows-latest, py311, bash)
- GitHub Check: tests (4, ubuntu-latest, py312, bash)
- GitHub Check: tests (4, ubuntu-latest, py311, bash)
- GitHub Check: tests (3, windows-latest, py312, bash)
- GitHub Check: tests (3, windows-latest, py311, bash)
- GitHub Check: tests (3, ubuntu-latest, py312, bash)
- GitHub Check: tests (3, ubuntu-latest, py311, bash)
- GitHub Check: tests (2, windows-latest, py312, bash)
- GitHub Check: tests (2, windows-latest, py311, bash)
- GitHub Check: tests (2, ubuntu-latest, py312, bash)
- GitHub Check: tests (2, ubuntu-latest, py311, bash)
- GitHub Check: tests (1, windows-latest, py312, bash)
- GitHub Check: tests (1, windows-latest, py311, bash)
- GitHub Check: tests (1, ubuntu-latest, py312, bash)
- GitHub Check: tests (1, ubuntu-latest, py311, bash)
- GitHub Check: apidocs
🔇 Additional comments (5)
src/snakemake/rules.py (2)
277-293: Enhanced input handling for multiext files.This implementation successfully handles multiext items by iterating through each file and setting the appropriate name. The logic clearly separates handling of multiext items from regular items, which makes the code more maintainable.
349-359: Properly implemented named output support for multiext.The implementation correctly handles multiext output items by checking with
is_multiext_itemsand iterating through each file in the item. The comments nicely explain the behavior, helping future developers understand the code.src/snakemake/io.py (3)
393-394: Updated multiext_prefix property to handle MultiextValue.The property now correctly retrieves the prefix from the MultiextValue object returned by get_flag_value.
1441-1445: Well-designed MultiextValue data class.Good choice using a dataclass to represent the multiext value with appropriate attributes for prefix and optional name.
1479-1486: Well-implemented multiext item detection function.The
is_multiext_itemsfunction provides a clear and efficient way to identify multiext items. The type annotations are thorough and help ensure proper usage.
🤖 I have created a release *beep* *boop* --- ## [9.0.0](v8.30.0...v9.0.0) (2025-03-14) ### ⚠ BREAKING CHANGES * Logging refactor & add LoggerPluginInterface ([#3107](#3107)) ### Features * [#3412](#3412) - keep shadow folder of failed job if --keep-incomplete flag is set. ([#3430](#3430)) ([22978c3](22978c3)) * add flag --report-after-run to automatically generate the report after a successfull workflow run ([#3428](#3428)) ([b0a7f03](b0a7f03)) * add flatten function to IO utils ([#3424](#3424)) ([67fa392](67fa392)) * add helper functions to parse input files ([#2918](#2918)) ([63e45a7](63e45a7)) * Add option to print redacted file names ([#3089](#3089)) ([ba4d264](ba4d264)) * add support for validation of polars dataframe and lazyframe ([#3262](#3262)) ([c7473a6](c7473a6)) * added support for rendering dag with mermaid js ([#3409](#3409)) ([7bf8381](7bf8381)) * adding --replace-workflow-config to fully replace workflow configs (from config: directive) with --configfile, instead of merging them ([#3381](#3381)) ([47504a0](47504a0)) * Dynamic module name ([#3401](#3401)) ([024dc32](024dc32)) * Enable saving and reloading IOCache object ([#3386](#3386)) ([c935953](c935953)) * files added in rule params with workflow.source_path will be available in used containers ([#3385](#3385)) ([a6e45bf](a6e45bf)) * Fix keep_local in storage directive and more freedom over remote retrieval behaviour ([#3410](#3410)) ([67b4739](67b4739)) * inherit parameters of use rule and extend/replace individual items them when using 'with' directive ([#3365](#3365)) ([93e4b92](93e4b92)) * Logging refactor & add LoggerPluginInterface ([#3107](#3107)) ([86f1d6e](86f1d6e)) * Maximal file size for checksums ([#3368](#3368)) ([b039f8a](b039f8a)) * Modernize package configuration using Pixi ([#3369](#3369)) ([77992d8](77992d8)) * multiext support for named input/output ([#3372](#3372)) ([05e1378](05e1378)) * optionally auto-group jobs via temp files in case of remote execution ([#3378](#3378)) ([cc9bba2](cc9bba2)) ### Bug Fixes * `--delete-all-output` ignores `--dry-run` ([#3265](#3265)) ([23fef82](23fef82)) * 3342 faster touch runs and warning messages for non-existing files ([#3398](#3398)) ([cd9c3c3](cd9c3c3)) * add default value to max-jobs-per-timespan ([#3043](#3043)) ([2959abe](2959abe)) * checkpoints inside modules are overwritten ([#3359](#3359)) ([fba3ac7](fba3ac7)) * Convert Path to IOFile ([#3405](#3405)) ([c58684c](c58684c)) * Do not perform storage object cleanup with --keep-storage-local-copies set ([#3358](#3358)) ([9a6d14b](9a6d14b)) * edgecases of source deployment in case of remote execution ([#3396](#3396)) ([5da13be](5da13be)) * enhance error message formatting for strict DAG-building mode ([#3376](#3376)) ([a1c39ee](a1c39ee)) * fix bug in checkpoint handling that led to exceptions in case checkpoint output was missing upon rerun ([#3423](#3423)) ([8cf4a2f](8cf4a2f)) * force check all required outputs ([#3341](#3341)) ([495a4e7](495a4e7)) * group job formatting ([#3442](#3442)) ([f0b10a3](f0b10a3)) * in remote jobs, upload storage in topological order such that modification dates are preserved (e.g. in case of group jobs) ([#3377](#3377)) ([eace08f](eace08f)) * only skip eval when resource depends on input ([#3374](#3374)) ([4574c92](4574c92)) * Prevent execution of conda in apptainer when not explicitly requested in software deployment method ([#3388](#3388)) ([c43c5c0](c43c5c0)) * print filenames with quotes around them in RuleException ([#3269](#3269)) ([6baeda5](6baeda5)) * Re-evaluation of free resources ([#3399](#3399)) ([6371293](6371293)) * ReadTheDocs layout issue due to src directory change ([#3419](#3419)) ([695b127](695b127)) * robustly escaping quotes in generated bash scripts (v2) ([#3297](#3297)) ([#3389](#3389)) ([58720bd](58720bd)) * Show apptainer image URL in snakemake report ([#3407](#3407)) ([45f0450](45f0450)) * Update ReadTheDocs configuration for documentation build to use Pixi ([#3433](#3433)) ([3f227a6](3f227a6)) ### Documentation * Add pixi setup instructions to general use tutorial ([#3382](#3382)) ([115e81b](115e81b)) * fix contribution section heading levels, fix docs testing setup order ([#3360](#3360)) ([051dc53](051dc53)) * fix link to github.com/snakemake/poetry-snakemake-plugin ([#3436](#3436)) ([ec6d97c](ec6d97c)) * fix quoting ([#3394](#3394)) ([b40f599](b40f599)) * fix rerun-triggers default ([#3403](#3403)) ([4430e23](4430e23)) * fix typo 'safe' -> 'save' ([#3384](#3384)) ([7755861](7755861)) * mention code formatting in the contribution section ([#3431](#3431)) ([e8682b7](e8682b7)) * remove duplicated 'functions'. ([#3356](#3356)) ([7c595db](7c595db)) * update broken links documentation ([#3437](#3437)) ([e3d0d88](e3d0d88)) * Updating contributing guidelines with new pixi dev setup ([#3415](#3415)) ([8e95a12](8e95a12)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: snakemake-bot <snakemake-bot-admin@googlegroups.com>
This PR brings named input/output to multiext and resolves snakemake#1335. In brief: ``` rule named: input: 'ab.txt' output: multiext("coln", out1 = ".1", out2 = ".2"), shell:''' cut -f1 {input} > {output.out1} cut -f2 {input} > {output.out2} ''' ``` ### QC <!-- Make sure that you can tick the boxes below. --> * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **Documentation** - Introduced a new section explaining how to define Snakemake rules with the `multiext` function, including usage examples. - **New Features** - Enhanced support for specifying multiple output files in a single rule with consistent naming through the `multiext` function. - **Tests** - Expanded test scenarios to ensure the reliability of the new `multiext` functionality. - **Refactor** - Improved file input/output processing for clearer rule definitions and robust error handling related to multiple extensions. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Johannes Koester <johannes.koester@uni-due.de>
🤖 I have created a release *beep* *boop* --- ## [9.0.0](snakemake/snakemake@v8.30.0...v9.0.0) (2025-03-14) ### ⚠ BREAKING CHANGES * Logging refactor & add LoggerPluginInterface ([snakemake#3107](snakemake#3107)) ### Features * [snakemake#3412](snakemake#3412) - keep shadow folder of failed job if --keep-incomplete flag is set. ([snakemake#3430](snakemake#3430)) ([22978c3](snakemake@22978c3)) * add flag --report-after-run to automatically generate the report after a successfull workflow run ([snakemake#3428](snakemake#3428)) ([b0a7f03](snakemake@b0a7f03)) * add flatten function to IO utils ([snakemake#3424](snakemake#3424)) ([67fa392](snakemake@67fa392)) * add helper functions to parse input files ([snakemake#2918](snakemake#2918)) ([63e45a7](snakemake@63e45a7)) * Add option to print redacted file names ([snakemake#3089](snakemake#3089)) ([ba4d264](snakemake@ba4d264)) * add support for validation of polars dataframe and lazyframe ([snakemake#3262](snakemake#3262)) ([c7473a6](snakemake@c7473a6)) * added support for rendering dag with mermaid js ([snakemake#3409](snakemake#3409)) ([7bf8381](snakemake@7bf8381)) * adding --replace-workflow-config to fully replace workflow configs (from config: directive) with --configfile, instead of merging them ([snakemake#3381](snakemake#3381)) ([47504a0](snakemake@47504a0)) * Dynamic module name ([snakemake#3401](snakemake#3401)) ([024dc32](snakemake@024dc32)) * Enable saving and reloading IOCache object ([snakemake#3386](snakemake#3386)) ([c935953](snakemake@c935953)) * files added in rule params with workflow.source_path will be available in used containers ([snakemake#3385](snakemake#3385)) ([a6e45bf](snakemake@a6e45bf)) * Fix keep_local in storage directive and more freedom over remote retrieval behaviour ([snakemake#3410](snakemake#3410)) ([67b4739](snakemake@67b4739)) * inherit parameters of use rule and extend/replace individual items them when using 'with' directive ([snakemake#3365](snakemake#3365)) ([93e4b92](snakemake@93e4b92)) * Logging refactor & add LoggerPluginInterface ([snakemake#3107](snakemake#3107)) ([86f1d6e](snakemake@86f1d6e)) * Maximal file size for checksums ([snakemake#3368](snakemake#3368)) ([b039f8a](snakemake@b039f8a)) * Modernize package configuration using Pixi ([snakemake#3369](snakemake#3369)) ([77992d8](snakemake@77992d8)) * multiext support for named input/output ([snakemake#3372](snakemake#3372)) ([05e1378](snakemake@05e1378)) * optionally auto-group jobs via temp files in case of remote execution ([snakemake#3378](snakemake#3378)) ([cc9bba2](snakemake@cc9bba2)) ### Bug Fixes * `--delete-all-output` ignores `--dry-run` ([snakemake#3265](snakemake#3265)) ([23fef82](snakemake@23fef82)) * 3342 faster touch runs and warning messages for non-existing files ([snakemake#3398](snakemake#3398)) ([cd9c3c3](snakemake@cd9c3c3)) * add default value to max-jobs-per-timespan ([snakemake#3043](snakemake#3043)) ([2959abe](snakemake@2959abe)) * checkpoints inside modules are overwritten ([snakemake#3359](snakemake#3359)) ([fba3ac7](snakemake@fba3ac7)) * Convert Path to IOFile ([snakemake#3405](snakemake#3405)) ([c58684c](snakemake@c58684c)) * Do not perform storage object cleanup with --keep-storage-local-copies set ([snakemake#3358](snakemake#3358)) ([9a6d14b](snakemake@9a6d14b)) * edgecases of source deployment in case of remote execution ([snakemake#3396](snakemake#3396)) ([5da13be](snakemake@5da13be)) * enhance error message formatting for strict DAG-building mode ([snakemake#3376](snakemake#3376)) ([a1c39ee](snakemake@a1c39ee)) * fix bug in checkpoint handling that led to exceptions in case checkpoint output was missing upon rerun ([snakemake#3423](snakemake#3423)) ([8cf4a2f](snakemake@8cf4a2f)) * force check all required outputs ([snakemake#3341](snakemake#3341)) ([495a4e7](snakemake@495a4e7)) * group job formatting ([snakemake#3442](snakemake#3442)) ([f0b10a3](snakemake@f0b10a3)) * in remote jobs, upload storage in topological order such that modification dates are preserved (e.g. in case of group jobs) ([snakemake#3377](snakemake#3377)) ([eace08f](snakemake@eace08f)) * only skip eval when resource depends on input ([snakemake#3374](snakemake#3374)) ([4574c92](snakemake@4574c92)) * Prevent execution of conda in apptainer when not explicitly requested in software deployment method ([snakemake#3388](snakemake#3388)) ([c43c5c0](snakemake@c43c5c0)) * print filenames with quotes around them in RuleException ([snakemake#3269](snakemake#3269)) ([6baeda5](snakemake@6baeda5)) * Re-evaluation of free resources ([snakemake#3399](snakemake#3399)) ([6371293](snakemake@6371293)) * ReadTheDocs layout issue due to src directory change ([snakemake#3419](snakemake#3419)) ([695b127](snakemake@695b127)) * robustly escaping quotes in generated bash scripts (v2) ([snakemake#3297](snakemake#3297)) ([snakemake#3389](snakemake#3389)) ([58720bd](snakemake@58720bd)) * Show apptainer image URL in snakemake report ([snakemake#3407](snakemake#3407)) ([45f0450](snakemake@45f0450)) * Update ReadTheDocs configuration for documentation build to use Pixi ([snakemake#3433](snakemake#3433)) ([3f227a6](snakemake@3f227a6)) ### Documentation * Add pixi setup instructions to general use tutorial ([snakemake#3382](snakemake#3382)) ([115e81b](snakemake@115e81b)) * fix contribution section heading levels, fix docs testing setup order ([snakemake#3360](snakemake#3360)) ([051dc53](snakemake@051dc53)) * fix link to github.com/snakemake/poetry-snakemake-plugin ([snakemake#3436](snakemake#3436)) ([ec6d97c](snakemake@ec6d97c)) * fix quoting ([snakemake#3394](snakemake#3394)) ([b40f599](snakemake@b40f599)) * fix rerun-triggers default ([snakemake#3403](snakemake#3403)) ([4430e23](snakemake@4430e23)) * fix typo 'safe' -> 'save' ([snakemake#3384](snakemake#3384)) ([7755861](snakemake@7755861)) * mention code formatting in the contribution section ([snakemake#3431](snakemake#3431)) ([e8682b7](snakemake@e8682b7)) * remove duplicated 'functions'. ([snakemake#3356](snakemake#3356)) ([7c595db](snakemake@7c595db)) * update broken links documentation ([snakemake#3437](snakemake#3437)) ([e3d0d88](snakemake@e3d0d88)) * Updating contributing guidelines with new pixi dev setup ([snakemake#3415](snakemake#3415)) ([8e95a12](snakemake@8e95a12)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: snakemake-bot <snakemake-bot-admin@googlegroups.com>
This PR brings named input/output to multiext and resolves #1335.
In brief:
QC
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).Summary by CodeRabbit
multiextfunction, including usage examples.multiextfunction.multiextfunctionality.