fix: enhance error message formatting for strict DAG-building mode#3376
Conversation
…nException even when a rule is not strictly required. Added warning for when the option is not set.
…dcards even when the rule responsible is not strictly required. Added warning for when the options are not set.
📝 WalkthroughWalkthroughThe changes introduce a new command-line argument, Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant CLI
participant ArgumentParser
participant DAGEvaluator
participant ExceptionHandler
User->>CLI: Execute command with --strict-dag-evaluation [values]
CLI->>ArgumentParser: Parse command-line options
ArgumentParser->>DAGEvaluator: Pass strict evaluation settings
DAGEvaluator->>DAGEvaluator: Evaluate DAG with strict mode
alt Strict error condition detected
DAGEvaluator->>ExceptionHandler: Raise appropriate exception (e.g., cyclic graph or periodic wildcard)
ExceptionHandler->>User: Display strict evaluation warning message
else No strict errors
DAGEvaluator->>User: Continue normal execution
end
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
|
Please format your code with black: |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
snakemake/workflow.py (1)
1812-1813: Boolean comparison can be simplified.The comparison with
Falseshould be replaced with a direct boolean check for better readability and to follow Python best practices.- if not ruleinfo.template_engine and ruleinfo.container_img != False: + if not ruleinfo.template_engine and ruleinfo.container_img:🧰 Tools
🪛 Ruff (0.8.2)
1812-1812: Avoid inequality comparisons to
False; useif ruleinfo.container_img:for truth checksReplace with
ruleinfo.container_img(E712)
snakemake/dag.py (1)
1049-1049: Consider improving the TODO comment clarityThe TODO comment is a bit vague. Consider clarifying the intention behind potentially including these additional exceptions in the same handling block.
-WorkflowError, #TODO maybe CyclicGraphException, PeriodicWildcardError, Workflow? +WorkflowError, #TODO: Consider extending this exception handling to include CyclicGraphException and PeriodicWildcardError for consistent error reporting
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (3)
pyproject.tomlis excluded by!pyproject.tomltests/test_script_xsh/expected-results/test.outis excluded by!**/*.outCHANGELOG.mdis excluded by!CHANGELOG.md
📒 Files selected for processing (28)
snakemake/cli.py(2 hunks)snakemake/dag.py(3 hunks)snakemake/exceptions.py(1 hunks)snakemake/settings/types.py(1 hunks)snakemake/cli.py(2 hunks)snakemake/dag.py(3 hunks)snakemake/settings/types.py(1 hunks)docs/snakefiles/deployment.rst(2 hunks)docs/snakefiles/reporting.rst(5 hunks)docs/snakefiles/rules.rst(1 hunks)snakemake/cli.py(1 hunks)snakemake/script/__init__.py(3 hunks)snakemake/workflow.py(1 hunks)test-environment.yml(2 hunks)tests/test_conda_python_3_7_script/Snakefile(1 hunks)tests/test_conda_python_3_7_script/test_script_python_3_7.py(1 hunks)tests/test_conda_run/Snakefile(1 hunks)tests/test_conda_run/expected-results/test.txt(1 hunks)tests/test_conda_run/test_python_env.yaml(1 hunks)tests/test_conda_run/test_script_run.py(1 hunks)tests/test_script_xsh/Snakefile(1 hunks)tests/test_script_xsh/envs/xonsh.yaml(1 hunks)tests/test_script_xsh/scripts/test.xsh(1 hunks)tests/tests_using_conda.py(1 hunks)snakemake/exceptions.py(0 hunks)snakemake/dag.py(6 hunks)snakemake/exceptions.py(4 hunks)setup.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
tests/test_conda_python_3_7_script/test_script_python_3_7.pysetup.pytests/test_conda_run/test_script_run.pysnakemake/exceptions.pysnakemake/settings/types.pysnakemake/dag.pytests/tests_using_conda.pysnakemake/workflow.pysnakemake/cli.pysnakemake/script/__init__.py
🪛 Ruff (0.8.2)
snakemake/dag.py
1051-1051: A length-one tuple literal is redundant in exception handlers
Replace with except CyclicGraphException
(B013)
1056-1056: A length-one tuple literal is redundant in exception handlers
Replace with except PeriodicWildcardError
(B013)
1087-1089: Multiple isinstance calls for e, merge into a single call
Merge isinstance calls for e
(SIM101)
1196-1198: Multiple isinstance calls for ex, merge into a single call
Merge isinstance calls for ex
(SIM101)
1051-1051: A length-one tuple literal is redundant in exception handlers
Replace with except CyclicGraphException
(B013)
1056-1056: A length-one tuple literal is redundant in exception handlers
Replace with except PeriodicWildcardError
(B013)
snakemake/workflow.py
1812-1812: Avoid inequality comparisons to False; use if ruleinfo.container_img: for truth checks
Replace with ruleinfo.container_img
(E712)
snakemake/exceptions.py
91-91: Multiple isinstance calls for ex, merge into a single call
Merge isinstance calls for ex
(SIM101)
144-147: Combine if branches using logical or operator
Combine if branches
(SIM114)
🔇 Additional comments (38)
tests/test_conda_run/expected-results/test.txt (1)
1-1: LGTM - Expected test result file.This file contains the expected output (3.0) from running
numpy.log2(8)in the test script. The value is correctly specified for test verification purposes.tests/test_script_xsh/scripts/test.xsh (1)
1-2: LGTM - Xonsh script correctly implemented.The script properly demonstrates Xonsh integration with Snakemake by writing "Hello, world!" to the output file specified in the Snakemake workflow. The syntax uses Xonsh's
@()operator to access the Snakemake output path.tests/test_conda_python_3_7_script/test_script_python_3_7.py (1)
4-4: String quote style change looks good.Changed from single quotes to double quotes in the filename string. This is a purely stylistic change with no functional impact.
tests/test_conda_run/test_script_run.py (1)
1-5: LGTM - Test script correctly implements NumPy log2 calculation.This script properly imports NumPy, calculates log base 2 of 8 (which equals 3.0), and writes the result to "test.txt". The implementation is simple and serves its testing purpose well.
setup.py (1)
33-33: Appropriate addition to package_data.The inclusion of "unit_tests/templates/*" ensures that necessary test template files are packaged with the distribution, which is essential for testing the new error formatting functionality.
tests/test_script_xsh/envs/xonsh.yaml (1)
1-5: LGTM - Proper Conda environment configuration for Xonsh.The environment file correctly specifies the necessary channels and dependencies for running Xonsh scripts. This aligns with the PR's addition of Xonsh script execution support.
tests/test_conda_python_3_7_script/Snakefile (1)
7-7: Script reference updated with clearer naming.The change from "test_script.py" to "test_script_python_3_7.py" improves clarity by explicitly indicating the Python version the script is designed for.
tests/test_conda_run/test_python_env.yaml (1)
1-6: Well-structured Conda environment configuration.The environment file properly defines channels and dependencies needed for the Python test environment. This supports the new testing capabilities introduced in this PR.
tests/test_script_xsh/Snakefile (1)
1-13: Well-structured Snakefile for testing Xonsh script executionThe Snakefile correctly defines a workflow with two rules: an
allrule that establishes the workflow dependency graph and atest_xonshrule that demonstrates Xonsh script execution with conda environment integration. The structure follows Snakemake best practices with clean dependency definition.tests/test_conda_run/Snakefile (1)
1-13: Well-documented test rule with educational commentsThe rule implementation serves its testing purpose well. The comments provide excellent guidance on why this pattern (calling a script via shell) is only used for testing purposes and what would be the preferred approach in actual workflows. This helps maintainers understand the intent behind this implementation choice.
tests/tests_using_conda.py (2)
309-315: LGTM: Test function properly skips windows platformThe test function for Xonsh script execution correctly includes the
@skip_on_windowsdecorator, which aligns with potential platform compatibility limitations for Xonsh scripts.
318-320: LGTM: Test function for conda run maintains cross-platform compatibilityUnlike the Xonsh test, this test function doesn't include the
@skip_on_windowsdecorator, suggesting this functionality is expected to work across all platforms including Windows. This is a good approach for ensuring cross-platform compatibility.snakemake/settings/types.py (1)
206-207: Looks good, added new flags for strict evaluation modes.These two new boolean attributes in the
DAGSettingsclass align with the PR objectives for enhancing error message formatting for strict DAG-building mode. The default values ofFalsemaintain backward compatibility while allowing explicit activation.test-environment.yml (2)
7-7: Added setuptools as a dependency.Adding setuptools is a good practice to ensure consistent packaging support.
61-62: Moved cwltool and cwl-utils from pip to conda dependencies.Moving these dependencies from the pip section to the main dependencies list makes sense for better dependency management through conda.
docs/snakefiles/rules.rst (1)
1591-1610: Added documentation for Xonsh script integration.The documentation for Xonsh integration is clear and follows the same format as the existing documentation for other script types. It provides an explanation of what Xonsh is, links to the official site, and includes a practical example that demonstrates its usage in Snakemake rules.
snakemake/workflow.py (4)
1779-1787: Good addition of a helper function for validation checks.This new function centralizes the deployment validation logic, which makes the code more maintainable and consistent. It clearly validates that certain deployment methods cannot be used with template engines.
1794-1794: Simplifying validation with the new helper function.Good refactoring - replacing inline validation with the centralized helper function improves code maintainability and readability.
1800-1800: Consistent approach for conda validation.Using the helper function here as well maintains consistency in how deployment method validation is handled across different directives.
1808-1808: Consistent approach for container validation.Good application of the helper function for container/singularity validation, which completes the refactoring of all deployment-related validation checks.
docs/snakefiles/deployment.rst (2)
288-293: Improved documentation for conda environment usage with run directive.The documentation now correctly explains that conda environments can be used with the
rundirective, and clearly explains the special case behavior - that it only affectsshellfunction calls executed from within therunscript. This clarifies an important detail for users.
464-467: Consistent documentation for apptainer usage with run directive.Similar to the conda documentation update, this section now correctly explains the constraints around using apptainer with the
rundirective. The consistent documentation style between conda and apptainer makes it easier for users to understand the parallel behaviors.snakemake/script/__init__.py (4)
1600-1603: Good implementation of Xonsh script support.The new
XonshScriptclass neatly inherits fromPythonScriptand properly overrides just theexecute_scriptmethod to run xonsh scripts. This is a clean implementation that follows the existing pattern for script types.
1667-1668: Added detection for Xonsh script files.The code now correctly identifies files with the
.xshextension as Xonsh scripts. This change integrates well with the existing language detection logic.
1732-1732: Added support for executing Xonsh scripts.The Xonsh script type is now properly registered in the script execution framework, enabling users to use Xonsh scripts in their workflows.
1736-1737: Updated error message to include Xonsh scripts.The error message for unsupported script types has been updated to include Xonsh scripts, providing users with complete information about supported script types.
snakemake/dag.py (2)
54-54: Import addition looks goodThe addition of
print_exception_warningfrom snakemake.exceptions is appropriate for the enhanced error handling approach introduced in this PR.
2265-2278: Good implementation of strict functions evaluationThis implementation correctly enforces strict validation of InputFunctionException when the
strict_functions_evaluationflag is enabled, while providing helpful warning messages when running in non-strict mode. The message clearly guides users on how to enable strict mode if needed.snakemake/cli.py (3)
869-870: Minor improvement to help text wording.The help text for
--strict-functions-evaluationhas been refined by replacing "strictly required to produce output files" with "strictly required to produce the output files" adding a definite article for clarity.
871-886: Good addition of new strict evaluation flags.The addition of two new CLI arguments enhances the strict validation capabilities of Snakemake:
--strict-cyclic-graph-evaluationfor enabling strict cycle detection--strict-periodic-wildcards-evaluationfor enabling strict periodic wildcards detectionBoth arguments follow the established naming convention and include appropriate aliases that match the existing pattern. The help text is clear and descriptive, explaining the purpose and behavior of each flag.
2118-2120: Correctly integrated new arguments with DAGSettings.The new CLI arguments are properly integrated with the
DAGSettingsobject, providing the values to the corresponding parameters:
strict_functions_evaluation(existing)strict_cycle_evaluation(new)strict_wildcards_recursion_evaluation(new)This implementation ensures that user selections from the command line are correctly reflected in the DAG building process.
docs/snakefiles/reporting.rst (7)
7-10: Improved report generation description.The introduction to reporting capabilities has been enhanced to better describe the purpose and benefits of Snakemake reports. The text now clarifies that reports are self-contained HTML documents that include runtime statistics, provenance information, and workflow topology by default. It also introduces the distinction between simpler reports (single HTML files) and more complex reports (ZIP archives), with appropriate cross-references to the detailed sections.
13-16: Added section header for including results in reports.Added a proper section with a reference anchor for the "Including results in a report" section, which improves document structure and allows direct linking to this specific topic.
257-264: Enhanced reporting introduction with metadata explanation.Added helpful context about how report metadata is automatically collected from the
.snakemakedirectory, which clarifies the source of runtime statistics and other information shown in reports.
265-285: Added comprehensive section on self-contained HTML reports.New section with clear instructions for generating basic HTML reports, including:
- Default usage with
--report- Custom report naming
- Explanation of when this format is appropriate (smaller reports)
This section provides practical guidance that helps users understand and utilize the HTML report feature effectively.
286-303: Added detailed section on ZIP archive reports.New section explaining the benefits and usage of ZIP archive reports:
- Explains when to use ZIP format (complex reports with many samples/outputs)
- Shows command-line usage
- Describes the portability benefits
- Identifies the main entry point file
This addition provides valuable guidance for users working with larger workflows and datasets.
304-317: Added information on generating partial reports.New section that explains how to generate reports for specific targets, which is useful for:
- Working with incomplete workflows
- Examining intermediate outputs
- Handling workflows with alternative target rules
This practical information helps users make better use of the reporting functionality during workflow development.
318-331: Added section on customizing report appearance.New section describing how to apply custom stylesheets to reports:
- Command-line usage for specifying a custom stylesheet
- Example use case (adding organization logo)
- Links to example stylesheet and report
This valuable addition helps users create branded reports appropriate for their organization or institution.
johanneskoester
left a comment
There was a problem hiding this comment.
I would suggest to use an enum and a single arg for this instead, e.g.
snakemake --strict-dag-evaluation periodic-wildcards cyclic-graph functions
The enum should be defined under snakemake.settings.enums. If you use the same base class as the other enums there (SettingsEnumBase), you can also copy the way the enum is used to offer the choices for the argument parser, e.g. copy from --rerun-triggers.
There was a problem hiding this comment.
Caution
Inline review comments failed to post. This is likely due to GitHub's limits when posting large numbers of comments. If you are seeing this consistently it is likely a permissions issue. Please check "Moderation" -> "Code review limits" under your organization settings.
Actionable comments posted: 1
🔭 Outside diff range comments (1)
snakemake/cli.py (1)
2106-2109:⚠️ Potential issueDuplicate/conflicting code detected.
These lines appear to be part of a different implementation approach than what's used above. Line 2102 passes
strict_functions_evaluationtoDAGSettings, but here there's a completely different parameterstrict_evaluationbeing passed the value ofargs.strict_dag_evaluation.One of these implementations should be removed to maintain consistency. Based on the PR objectives, either implement the three separate parameters or use the consolidated approach, but not both.
🧹 Nitpick comments (1)
snakemake/dag.py (1)
1046-1065: Enhanced exception handling with configurable strictness.The updated exception handling now captures
WorkflowErrorand implements conditional raising ofCyclicGraphExceptionandPeriodicWildcardErrorbased on strict evaluation settings. This makes the code more robust while providing configurability.However, there's a minor style issue with the tuple syntax in the exception clauses.
According to the static analysis hints, length-one tuples are redundant in exception handlers. Consider this change:
-except ( - CyclicGraphException, -) as ex: +except CyclicGraphException as ex: -except ( - PeriodicWildcardError, -) as ex: +except PeriodicWildcardError as ex:🧰 Tools
🪛 Ruff (0.8.2)
1051-1051: A length-one tuple literal is redundant in exception handlers
Replace with
except CyclicGraphException(B013)
1059-1059: A length-one tuple literal is redundant in exception handlers
Replace with
except PeriodicWildcardError(B013)
🛑 Comments failed to post (1)
snakemake/cli.py (1)
2102-2103: 💡 Verification agent
🧩 Analysis chain
Inconsistency with strict evaluation implementation.
The code at this line passes the new
strict_functions_evaluationparameter toDAGSettings, but there appears to be another implementation approach at lines 2107-2108 that uses a consolidatedstrict_evaluationparameter instead (passed the value ofargs.strict_dag_evaluation). However, I don't see whereargs.strict_dag_evaluationis defined in this file.This suggests there's an inconsistency between implementations or a transition from individual parameters to a consolidated one. According to the PR objectives, there should be three parameters:
strict-functions-evaluation,strict-cyclic-graph-evaluation, andstrict-periodic-wildcards-evaluation, but only one is implemented here.
🏁 Script executed:
#!/bin/bash # Check if strict_dag_evaluation parameter is defined elsewhere in the codebase echo "Searching for strict_dag_evaluation parameter definition..." rg --type py 'strict.?dag.?evaluation' --ignore-caseLength of output: 1117
Action Required: Resolve the Strict Evaluation Parameter Inconsistency
After verifying the code, note the following:
- The parameter
--strict-dag-evaluationis indeed defined and parsed insnakemake/cli.py(via theStrictDagEvaluationsettings), so the claim thatargs.strict_dag_evaluationis missing is incorrect.- However, there is still a discrepancy: the code passes a new
strict_functions_evaluationparameter toDAGSettingswhile simultaneously using a consolidated strict evaluation parameter (strict_evaluation=args.strict_dag_evaluation). This creates an inconsistency.- According to the PR objectives, the strict evaluation mechanism should be split into three separate parameters (
strict-functions-evaluation,strict-cyclic-graph-evaluation, andstrict-periodic-wildcards-evaluation), yet only the consolidated approach is implemented.Next Steps:
- Clarify the Design Intent: Confirm whether the transition to a single consolidated strict evaluation parameter is intentional.
- Adjust Implementation: If the original intent (three distinct parameters) must be maintained, update the implementation accordingly by splitting the consolidated parameter into the three separate ones.
|
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
snakemake/dag.py (3)
1053-1069: Enhanced exception handling for CyclicGraphException and PeriodicWildcardErrorThe code now checks specific values from the StrictDagEvaluation enum before deciding whether to raise exceptions or append them to the exceptions list. This implements the core functionality described in the PR objectives.
However, there are some syntax improvements that could be made:
- except (CyclicGraphException,) as ex: + except CyclicGraphException as ex:- except (PeriodicWildcardError,) as ex: + except PeriodicWildcardError as ex:The parentheses and trailing commas for length-one tuples in exception handlers are redundant.
🧰 Tools
🪛 Ruff (0.8.2)
1053-1053: A length-one tuple literal is redundant in exception handlers
Replace with
except CyclicGraphException(B013)
1061-1061: A length-one tuple literal is redundant in exception handlers
Replace with
except PeriodicWildcardError(B013)
1094-1098: Added warning message printing for collected exceptionsThe code now prints warning messages for CyclicGraphException and PeriodicWildcardError exceptions after they've been collected, which provides better feedback to users when strict evaluation is not enforced.
The implementation could be improved by combining the isinstance checks:
- if isinstance(e, CyclicGraphException) or isinstance( - e, PeriodicWildcardError - ): + if isinstance(e, (CyclicGraphException, PeriodicWildcardError)):This is more efficient and follows Python best practices for type checking.
🧰 Tools
🪛 Ruff (0.8.2)
1095-1097: Multiple
isinstancecalls fore, merge into a single callMerge
isinstancecalls fore(SIM101)
1204-1207: Added warning message printing for exceptions in update_ methodSimilar to the previous comment, the code now prints warning messages for exceptions in the update_ method. The implementation could be improved:
- if isinstance(ex, CyclicGraphException) or isinstance( - ex, PeriodicWildcardError - ): + if isinstance(ex, (CyclicGraphException, PeriodicWildcardError)):This change would make the code more concise and follow Python best practices.
🧰 Tools
🪛 Ruff (0.8.2)
1204-1206: Multiple
isinstancecalls forex, merge into a single callMerge
isinstancecalls forex(SIM101)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
snakemake/cli.py(3 hunks)snakemake/dag.py(6 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- snakemake/cli.py
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
snakemake/dag.py
🪛 Ruff (0.8.2)
snakemake/dag.py
1053-1053: A length-one tuple literal is redundant in exception handlers
Replace with except CyclicGraphException
(B013)
1061-1061: A length-one tuple literal is redundant in exception handlers
Replace with except PeriodicWildcardError
(B013)
1095-1097: Multiple isinstance calls for e, merge into a single call
Merge isinstance calls for e
(SIM101)
1204-1206: Multiple isinstance calls for ex, merge into a single call
Merge isinstance calls for ex
(SIM101)
⏰ Context from checks skipped due to timeout of 90000ms (30)
- GitHub Check: testing (10, 3.12, bash)
- GitHub Check: testing (10, 3.11, bash)
- GitHub Check: testing (9, 3.12, bash)
- GitHub Check: testing (9, 3.11, bash)
- GitHub Check: testing (8, 3.12, bash)
- GitHub Check: testing (8, 3.11, bash)
- GitHub Check: testing (7, 3.12, bash)
- GitHub Check: testing (7, 3.11, bash)
- GitHub Check: testing (6, 3.12, bash)
- GitHub Check: testing (6, 3.11, bash)
- GitHub Check: testing (5, 3.12, bash)
- GitHub Check: testing-windows (10)
- GitHub Check: testing (5, 3.11, bash)
- GitHub Check: testing-windows (9)
- GitHub Check: testing (4, 3.12, bash)
- GitHub Check: testing-windows (8)
- GitHub Check: testing (4, 3.11, bash)
- GitHub Check: testing-windows (7)
- GitHub Check: testing (3, 3.12, bash)
- GitHub Check: testing-windows (6)
- GitHub Check: testing (3, 3.11, bash)
- GitHub Check: testing-windows (5)
- GitHub Check: testing (2, 3.12, bash)
- GitHub Check: testing-windows (4)
- GitHub Check: testing (2, 3.11, bash)
- GitHub Check: testing-windows (3)
- GitHub Check: testing (1, 3.12, bash)
- GitHub Check: testing-windows (2)
- GitHub Check: testing (1, 3.11, bash)
- GitHub Check: testing-windows (1)
🔇 Additional comments (3)
snakemake/dag.py (3)
38-38: Added import for StrictDagEvaluation enum from settings.typesThis import is required to support the new strict DAG evaluation functionality. It's properly placed with related imports.
54-54: Added import for print_exception_warning from exceptionsThis function is now needed for printing warning messages when non-strict mode is used, which aligns well with the PR objective to enhance error message formatting.
2291-2307: Enhanced handling of InputFunctionExceptionThis change implements the check for the StrictDagEvaluation.FUNCTIONS flag and properly raises the exception when strict mode is enabled. When strict mode is not enabled, it provides helpful warning messages.
I notice that this call to print_exception_warning includes an additional parameter with usage instructions, which is a nice touch for improved user experience:
print_exception_warning( e, self.workflow.linemaps, "Use --strict-dag-evaluation to force strict mode.", )This helps users understand how to switch to strict mode if needed.
🤖 I have created a release *beep* *boop* --- ## [9.0.0](v8.30.0...v9.0.0) (2025-03-14) ### ⚠ BREAKING CHANGES * Logging refactor & add LoggerPluginInterface ([#3107](#3107)) ### Features * [#3412](#3412) - keep shadow folder of failed job if --keep-incomplete flag is set. ([#3430](#3430)) ([22978c3](22978c3)) * add flag --report-after-run to automatically generate the report after a successfull workflow run ([#3428](#3428)) ([b0a7f03](b0a7f03)) * add flatten function to IO utils ([#3424](#3424)) ([67fa392](67fa392)) * add helper functions to parse input files ([#2918](#2918)) ([63e45a7](63e45a7)) * Add option to print redacted file names ([#3089](#3089)) ([ba4d264](ba4d264)) * add support for validation of polars dataframe and lazyframe ([#3262](#3262)) ([c7473a6](c7473a6)) * added support for rendering dag with mermaid js ([#3409](#3409)) ([7bf8381](7bf8381)) * adding --replace-workflow-config to fully replace workflow configs (from config: directive) with --configfile, instead of merging them ([#3381](#3381)) ([47504a0](47504a0)) * Dynamic module name ([#3401](#3401)) ([024dc32](024dc32)) * Enable saving and reloading IOCache object ([#3386](#3386)) ([c935953](c935953)) * files added in rule params with workflow.source_path will be available in used containers ([#3385](#3385)) ([a6e45bf](a6e45bf)) * Fix keep_local in storage directive and more freedom over remote retrieval behaviour ([#3410](#3410)) ([67b4739](67b4739)) * inherit parameters of use rule and extend/replace individual items them when using 'with' directive ([#3365](#3365)) ([93e4b92](93e4b92)) * Logging refactor & add LoggerPluginInterface ([#3107](#3107)) ([86f1d6e](86f1d6e)) * Maximal file size for checksums ([#3368](#3368)) ([b039f8a](b039f8a)) * Modernize package configuration using Pixi ([#3369](#3369)) ([77992d8](77992d8)) * multiext support for named input/output ([#3372](#3372)) ([05e1378](05e1378)) * optionally auto-group jobs via temp files in case of remote execution ([#3378](#3378)) ([cc9bba2](cc9bba2)) ### Bug Fixes * `--delete-all-output` ignores `--dry-run` ([#3265](#3265)) ([23fef82](23fef82)) * 3342 faster touch runs and warning messages for non-existing files ([#3398](#3398)) ([cd9c3c3](cd9c3c3)) * add default value to max-jobs-per-timespan ([#3043](#3043)) ([2959abe](2959abe)) * checkpoints inside modules are overwritten ([#3359](#3359)) ([fba3ac7](fba3ac7)) * Convert Path to IOFile ([#3405](#3405)) ([c58684c](c58684c)) * Do not perform storage object cleanup with --keep-storage-local-copies set ([#3358](#3358)) ([9a6d14b](9a6d14b)) * edgecases of source deployment in case of remote execution ([#3396](#3396)) ([5da13be](5da13be)) * enhance error message formatting for strict DAG-building mode ([#3376](#3376)) ([a1c39ee](a1c39ee)) * fix bug in checkpoint handling that led to exceptions in case checkpoint output was missing upon rerun ([#3423](#3423)) ([8cf4a2f](8cf4a2f)) * force check all required outputs ([#3341](#3341)) ([495a4e7](495a4e7)) * group job formatting ([#3442](#3442)) ([f0b10a3](f0b10a3)) * in remote jobs, upload storage in topological order such that modification dates are preserved (e.g. in case of group jobs) ([#3377](#3377)) ([eace08f](eace08f)) * only skip eval when resource depends on input ([#3374](#3374)) ([4574c92](4574c92)) * Prevent execution of conda in apptainer when not explicitly requested in software deployment method ([#3388](#3388)) ([c43c5c0](c43c5c0)) * print filenames with quotes around them in RuleException ([#3269](#3269)) ([6baeda5](6baeda5)) * Re-evaluation of free resources ([#3399](#3399)) ([6371293](6371293)) * ReadTheDocs layout issue due to src directory change ([#3419](#3419)) ([695b127](695b127)) * robustly escaping quotes in generated bash scripts (v2) ([#3297](#3297)) ([#3389](#3389)) ([58720bd](58720bd)) * Show apptainer image URL in snakemake report ([#3407](#3407)) ([45f0450](45f0450)) * Update ReadTheDocs configuration for documentation build to use Pixi ([#3433](#3433)) ([3f227a6](3f227a6)) ### Documentation * Add pixi setup instructions to general use tutorial ([#3382](#3382)) ([115e81b](115e81b)) * fix contribution section heading levels, fix docs testing setup order ([#3360](#3360)) ([051dc53](051dc53)) * fix link to github.com/snakemake/poetry-snakemake-plugin ([#3436](#3436)) ([ec6d97c](ec6d97c)) * fix quoting ([#3394](#3394)) ([b40f599](b40f599)) * fix rerun-triggers default ([#3403](#3403)) ([4430e23](4430e23)) * fix typo 'safe' -> 'save' ([#3384](#3384)) ([7755861](7755861)) * mention code formatting in the contribution section ([#3431](#3431)) ([e8682b7](e8682b7)) * remove duplicated 'functions'. ([#3356](#3356)) ([7c595db](7c595db)) * update broken links documentation ([#3437](#3437)) ([e3d0d88](e3d0d88)) * Updating contributing guidelines with new pixi dev setup ([#3415](#3415)) ([8e95a12](8e95a12)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: snakemake-bot <snakemake-bot-admin@googlegroups.com>
…nakemake#3376) ### Description This PR adds three CLI parameters to enhance debugging capabilities through strict evaluation modes: * strict-functions-evaluation * strict-cyclic-graph-evaluation * strict-periodic-wildcards-evaluation These flags are stored in types.DAGSettings and when enabled, enforce strict checking of exceptions that would otherwise be ignored if the rules producing them are not strictly required. The affected exceptions are: * InputFunctionException * CyclicGraphException * PeriodicWildcardError When these flags are not set, appropriate warning messages are now displayed for these exceptions. Additionally, small refactor to exceptions.py: the code to stringify exceptions is moved from print_exception into a new function named format_exception_to_string, which is now used to build consistent strings for both error reporting and logging purposes. Closes snakemake#3331 <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Enabled execution of Xonsh scripts in workflows. - Consolidated strict evaluation options into a single parameter for improved flexibility. - **Documentation** - Updated help text and user guides for report generation, highlighting options for self-contained HTML and ZIP reports. - Refined deployment instructions for using conda and container integrations with workflow directives. - Added guidance on integrating Xonsh scripts within rules. - **Tests** - Expanded test coverage for conda deployments and script executions, including support for Python 3.7 and Xonsh workflows. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: Johannes Köster <johannes.koester@tu-dortmund.de>
🤖 I have created a release *beep* *boop* --- ## [9.0.0](snakemake/snakemake@v8.30.0...v9.0.0) (2025-03-14) ### ⚠ BREAKING CHANGES * Logging refactor & add LoggerPluginInterface ([snakemake#3107](snakemake#3107)) ### Features * [snakemake#3412](snakemake#3412) - keep shadow folder of failed job if --keep-incomplete flag is set. ([snakemake#3430](snakemake#3430)) ([22978c3](snakemake@22978c3)) * add flag --report-after-run to automatically generate the report after a successfull workflow run ([snakemake#3428](snakemake#3428)) ([b0a7f03](snakemake@b0a7f03)) * add flatten function to IO utils ([snakemake#3424](snakemake#3424)) ([67fa392](snakemake@67fa392)) * add helper functions to parse input files ([snakemake#2918](snakemake#2918)) ([63e45a7](snakemake@63e45a7)) * Add option to print redacted file names ([snakemake#3089](snakemake#3089)) ([ba4d264](snakemake@ba4d264)) * add support for validation of polars dataframe and lazyframe ([snakemake#3262](snakemake#3262)) ([c7473a6](snakemake@c7473a6)) * added support for rendering dag with mermaid js ([snakemake#3409](snakemake#3409)) ([7bf8381](snakemake@7bf8381)) * adding --replace-workflow-config to fully replace workflow configs (from config: directive) with --configfile, instead of merging them ([snakemake#3381](snakemake#3381)) ([47504a0](snakemake@47504a0)) * Dynamic module name ([snakemake#3401](snakemake#3401)) ([024dc32](snakemake@024dc32)) * Enable saving and reloading IOCache object ([snakemake#3386](snakemake#3386)) ([c935953](snakemake@c935953)) * files added in rule params with workflow.source_path will be available in used containers ([snakemake#3385](snakemake#3385)) ([a6e45bf](snakemake@a6e45bf)) * Fix keep_local in storage directive and more freedom over remote retrieval behaviour ([snakemake#3410](snakemake#3410)) ([67b4739](snakemake@67b4739)) * inherit parameters of use rule and extend/replace individual items them when using 'with' directive ([snakemake#3365](snakemake#3365)) ([93e4b92](snakemake@93e4b92)) * Logging refactor & add LoggerPluginInterface ([snakemake#3107](snakemake#3107)) ([86f1d6e](snakemake@86f1d6e)) * Maximal file size for checksums ([snakemake#3368](snakemake#3368)) ([b039f8a](snakemake@b039f8a)) * Modernize package configuration using Pixi ([snakemake#3369](snakemake#3369)) ([77992d8](snakemake@77992d8)) * multiext support for named input/output ([snakemake#3372](snakemake#3372)) ([05e1378](snakemake@05e1378)) * optionally auto-group jobs via temp files in case of remote execution ([snakemake#3378](snakemake#3378)) ([cc9bba2](snakemake@cc9bba2)) ### Bug Fixes * `--delete-all-output` ignores `--dry-run` ([snakemake#3265](snakemake#3265)) ([23fef82](snakemake@23fef82)) * 3342 faster touch runs and warning messages for non-existing files ([snakemake#3398](snakemake#3398)) ([cd9c3c3](snakemake@cd9c3c3)) * add default value to max-jobs-per-timespan ([snakemake#3043](snakemake#3043)) ([2959abe](snakemake@2959abe)) * checkpoints inside modules are overwritten ([snakemake#3359](snakemake#3359)) ([fba3ac7](snakemake@fba3ac7)) * Convert Path to IOFile ([snakemake#3405](snakemake#3405)) ([c58684c](snakemake@c58684c)) * Do not perform storage object cleanup with --keep-storage-local-copies set ([snakemake#3358](snakemake#3358)) ([9a6d14b](snakemake@9a6d14b)) * edgecases of source deployment in case of remote execution ([snakemake#3396](snakemake#3396)) ([5da13be](snakemake@5da13be)) * enhance error message formatting for strict DAG-building mode ([snakemake#3376](snakemake#3376)) ([a1c39ee](snakemake@a1c39ee)) * fix bug in checkpoint handling that led to exceptions in case checkpoint output was missing upon rerun ([snakemake#3423](snakemake#3423)) ([8cf4a2f](snakemake@8cf4a2f)) * force check all required outputs ([snakemake#3341](snakemake#3341)) ([495a4e7](snakemake@495a4e7)) * group job formatting ([snakemake#3442](snakemake#3442)) ([f0b10a3](snakemake@f0b10a3)) * in remote jobs, upload storage in topological order such that modification dates are preserved (e.g. in case of group jobs) ([snakemake#3377](snakemake#3377)) ([eace08f](snakemake@eace08f)) * only skip eval when resource depends on input ([snakemake#3374](snakemake#3374)) ([4574c92](snakemake@4574c92)) * Prevent execution of conda in apptainer when not explicitly requested in software deployment method ([snakemake#3388](snakemake#3388)) ([c43c5c0](snakemake@c43c5c0)) * print filenames with quotes around them in RuleException ([snakemake#3269](snakemake#3269)) ([6baeda5](snakemake@6baeda5)) * Re-evaluation of free resources ([snakemake#3399](snakemake#3399)) ([6371293](snakemake@6371293)) * ReadTheDocs layout issue due to src directory change ([snakemake#3419](snakemake#3419)) ([695b127](snakemake@695b127)) * robustly escaping quotes in generated bash scripts (v2) ([snakemake#3297](snakemake#3297)) ([snakemake#3389](snakemake#3389)) ([58720bd](snakemake@58720bd)) * Show apptainer image URL in snakemake report ([snakemake#3407](snakemake#3407)) ([45f0450](snakemake@45f0450)) * Update ReadTheDocs configuration for documentation build to use Pixi ([snakemake#3433](snakemake#3433)) ([3f227a6](snakemake@3f227a6)) ### Documentation * Add pixi setup instructions to general use tutorial ([snakemake#3382](snakemake#3382)) ([115e81b](snakemake@115e81b)) * fix contribution section heading levels, fix docs testing setup order ([snakemake#3360](snakemake#3360)) ([051dc53](snakemake@051dc53)) * fix link to github.com/snakemake/poetry-snakemake-plugin ([snakemake#3436](snakemake#3436)) ([ec6d97c](snakemake@ec6d97c)) * fix quoting ([snakemake#3394](snakemake#3394)) ([b40f599](snakemake@b40f599)) * fix rerun-triggers default ([snakemake#3403](snakemake#3403)) ([4430e23](snakemake@4430e23)) * fix typo 'safe' -> 'save' ([snakemake#3384](snakemake#3384)) ([7755861](snakemake@7755861)) * mention code formatting in the contribution section ([snakemake#3431](snakemake#3431)) ([e8682b7](snakemake@e8682b7)) * remove duplicated 'functions'. ([snakemake#3356](snakemake#3356)) ([7c595db](snakemake@7c595db)) * update broken links documentation ([snakemake#3437](snakemake#3437)) ([e3d0d88](snakemake@e3d0d88)) * Updating contributing guidelines with new pixi dev setup ([snakemake#3415](snakemake#3415)) ([8e95a12](snakemake@8e95a12)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: snakemake-bot <snakemake-bot-admin@googlegroups.com>
Description
This PR adds three CLI parameters to enhance debugging capabilities through strict evaluation modes:
strict-functions-evaluation
strict-cyclic-graph-evaluation
strict-periodic-wildcards-evaluation
These flags are stored in types.DAGSettings and when enabled, enforce strict checking of exceptions that would otherwise be ignored if the rules producing them are not strictly required. The affected exceptions are:
InputFunctionException
CyclicGraphException
PeriodicWildcardError
When these flags are not set, appropriate warning messages are now displayed for these exceptions.
Additionally, small refactor to exceptions.py: the code to stringify exceptions is moved from print_exception into a new function named format_exception_to_string, which is now used to build consistent strings for both error reporting and logging purposes.
Closes #3331
Summary by CodeRabbit
New Features
Refactor