feat: adding --replace-workflow-config to fully replace workflow configs (from config: directive) with --configfile, instead of merging them#3381
Conversation
📝 WalkthroughWalkthroughThis pull request introduces configuration-driven workflows and new testing scenarios. Multiple Snakefiles and YAML configuration files have been added to define pipeline behaviors, with expected result files updated accordingly. Test cases have been expanded to verify workflow behavior under different configuration replacement settings. In the core application code, a new command-line flag ( Changes
Sequence Diagram(s)sequenceDiagram
participant CLI as Command Line
participant API as Snakemake API
participant WF as Workflow
participant Logger as Logger
CLI->>API: Pass --replace-workflow-config flag
API->>WF: Call configfile() with config_settings.replace_workflow_config
alt replace_workflow_config is true
WF->>Logger: Log "Config replaced" (merge_action="replaced")
else replace_workflow_config is false
WF->>Logger: Log "Config extended" (merge_action="extended")
end
Tip ⚡🧪 Multi-step agentic review comment chat (experimental)
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (11)
🚧 Files skipped from review as they are similar to previous changes (6)
🧰 Additional context used📓 Path-based instructions (1)`**/*.py`: Do not try to improve formatting. Do not suggest ...
⏰ Context from checks skipped due to timeout of 90000ms (40)
🔇 Additional comments (7)
✨ Finishing Touches
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (3)
snakemake/settings/types.py (1)
367-372: Fix missing space in warning message.There's a missing space between "via" and "command" in the warning message, which could affect readability.
- "--replace-workflow-config was used but no config entries are provided via" - "command line. This flag will be ignored." + "--replace-workflow-config was used but no config entries are provided via " + "command line. This flag will be ignored."tests/test_config_replacing_nocli/Snakefile (1)
1-1: Unnecessary importThe
jsonmodule is imported but not used anywhere in the file.-import jsontests/test_config_replacing/Snakefile (1)
1-1: Unnecessary importThe
jsonmodule is imported but not used anywhere in the file.-import json
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (13)
snakemake/cli.py(2 hunks)snakemake/settings/types.py(1 hunks)snakemake/workflow.py(1 hunks)tests/test_config_replacing/Snakefile(1 hunks)tests/test_config_replacing/cli-config.yaml(1 hunks)tests/test_config_replacing/expected-results/result.txt(1 hunks)tests/test_config_replacing/workflow-config.yaml(1 hunks)tests/tests.py(1 hunks)tests/test_config_replacing_nocli/Snakefile(1 hunks)tests/test_config_replacing_nocli/expected-results/result.txt(1 hunks)tests/test_config_replacing_nocli/workflow-config.yaml(1 hunks)tests/tests.py(1 hunks)snakemake/settings/types.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
tests/tests.pysnakemake/settings/types.pysnakemake/workflow.pysnakemake/cli.py
🪛 GitHub Actions: CI
tests/tests.py
[error] 373-373: Black formatting check failed. The string should use double quotes.
[error] 373-373: Black formatting check failed. The string should use double quotes.
snakemake/workflow.py
[error] 1549-1549: Black formatting check failed. The line should use double quotes.
snakemake/cli.py
[error] 2053-2053: Black formatting check failed. The line should end with a comma.
⏰ Context from checks skipped due to timeout of 90000ms (1)
- GitHub Check: apidocs
🔇 Additional comments (15)
snakemake/settings/types.py (1)
351-351: Boolean attribute for replacing workflow config added.The new
replace_workflowattribute with a default value ofFalsehas been added to theConfigSettingsclass. This will be used to determine whether to replace the workflow's existing configuration instead of merging it.tests/test_config_replacing_nocli/workflow-config.yaml (1)
1-2: LGTM: Simple config file for testing.This YAML configuration file provides a simple test case with just a 'name' entry that will be used for testing the new
--replace-workflow-configfeature.tests/test_config_replacing_nocli/expected-results/result.txt (1)
1-2: LGTM: Expected test output.This file contains the expected output for the test case where the workflow configuration is replaced without any command-line configuration. The output format "name-noid-novalue" suggests the workflow is correctly using the 'name' from the configuration while using default values for 'id' and 'value'.
tests/test_config_replacing/workflow-config.yaml (1)
1-2: LGTM: Duplicate config file for another test case.This YAML configuration file is identical to the one in the
test_config_replacing_noclidirectory and provides a simple test case with a 'name' entry. This is likely used for a different test scenario involving the--replace-workflow-configfeature.tests/test_config_replacing/expected-results/result.txt (1)
1-2: LGTM: Expected test output fileThis file contains the expected output for the configuration replacement test, which appears to be correctly formatted according to the test case requirements.
tests/test_config_replacing/cli-config.yaml (1)
1-3: LGTM: Appropriately structured test configurationThis YAML configuration file correctly defines the test values that will be used to validate the new
--replace-workflow-configfunctionality.tests/test_config_replacing_nocli/Snakefile (2)
4-6: LGTM: Safe configuration access with defaultsGood practice using fallback values when accessing configuration keys.
8-18: LGTM: Well-structured Snakemake rulesThe rules are defined appropriately for testing the configuration replacement functionality.
tests/test_config_replacing/Snakefile (2)
4-6: LGTM: Safe configuration access with defaultsGood practice using fallback values when accessing configuration keys.
8-18: LGTM: Well-structured Snakemake rulesThe rules are defined appropriately for testing the configuration replacement functionality.
tests/tests.py (1)
373-379:⚠️ Potential issueNew test function looks good, but fix the string quotes to match Black formatting.
The new test function properly verifies the behavior of the
--replace-workflow-configflag when used without any CLI configuration entries. This complements the existingtest_config_replacingfunction.Fix the string quotes in line 373 to comply with Black formatting requirements:
- shellcmd='snakemake -j 1 --configfile cli-config.yaml --config "value=value2" --replace-workflow-config', + shellcmd="snakemake -j 1 --configfile cli-config.yaml --config \"value=value2\" --replace-workflow-config",Likely an incorrect or invalid review comment.
🧰 Tools
🪛 GitHub Actions: CI
[error] 373-373: Black formatting check failed. The string should use double quotes.
snakemake/workflow.py (2)
1550-1559: Good implementation of workflow config replacement functionality.The changes properly implement the ability to replace the entire workflow configuration rather than just extending it. The code correctly discards the global config before merging when the
replace_workflowflag is enabled, and the log message is appropriately updated to indicate whether the configuration was extended or replaced.
1556-1558:⚠️ Potential issueFix string quotes to match Black formatting requirements.
Line 1557 has a formatting issue detected by Black. Fix the string quotes:
- "Config file {} is {} by additional config specified via the command line.".format( + f"Config file {fp} is {merge_action} by additional config specified via the command line."This would also simplify the code by using an f-string instead of
.format().Likely an incorrect or invalid review comment.
snakemake/cli.py (2)
688-697: Well-implemented new command-line optionThe new
--replace-workflow-configflag is well-implemented with clear help text that explains its purpose and behavior. This flag allows users to fully replace the workflow configuration with the config files provided via the command line, rather than extending the existing configuration.
2055-2059:Details
❓ Verification inconclusive
Verify this feature works with the settings API
The integration of the new flag with the ConfigSettings API looks good. Ensure that the
replace_workflowparameter in ConfigSettings is properly handled in the downstream code to fully replace workflow configurations as expected.
🏁 Script executed:
#!/bin/bash # Check if the ConfigSettings class correctly accepts and handles the replace_workflow parameter rg "replace_workflow" -A 5 -B 5 "settings/types.py"Length of output: 107
Manual Verification Required: Validate Replace Workflow Integration
The new flag wiring in
snakemake/cli.pyappears correctly set up, mappingargs.replace_workflow_configto thereplace_workflowparameter. However, our check for a corresponding update insettings/types.pyreturned no results, indicating that this integration may be implemented in a different module or might require further manual inspection. Please confirm that theConfigSettingsAPI correctly accepts and processes thereplace_workflowparameter—ensuring any downstream usage fully replaces workflow configurations as intended.
- Verify the updated handling of the
replace_workflowparameter by searching the entire repository (e.g., viarg "replace_workflow") to locate its implementation.- Confirm that any changes to
ConfigSettingsregarding this parameter are properly reflected in downstream modules.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
tests/test_config_replacing/Snakefile (1)
1-1: Unused import detected.The
jsonmodule is imported but not used anywhere in this Snakefile. Consider removing unused imports to maintain code cleanliness.-import jsontests/test_config_replacing_nocli/Snakefile (2)
1-1: Unused import detected.The
jsonmodule is imported but not used anywhere in the file. Consider removing this import to keep the code clean.-import json
1-17: Consider adding comments to clarify test purpose.This test file would benefit from comments explaining how it relates to the
--replace-workflow-configfeature being implemented. Adding context about what scenario this is testing would make the code more maintainable.+# Test case for configuration loading without CLI overrides +# This tests the default behavior without the --replace-workflow-config flag import json configfile: "workflow-config.yaml"
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
tests/test_config_replacing/Snakefile(1 hunks)tests/test_config_replacing_nocli/Snakefile(1 hunks)tests/test_config_replacing/Snakefile(0 hunks)tests/test_config_replacing_nocli/Snakefile(0 hunks)tests/test_config_replacing/Snakefile(1 hunks)tests/test_config_replacing_nocli/Snakefile(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
- tests/test_config_replacing_nocli/Snakefile
- tests/test_config_replacing_nocli/Snakefile
- tests/test_config_replacing/Snakefile
- tests/test_config_replacing/Snakefile
⏰ Context from checks skipped due to timeout of 90000ms (26)
- GitHub Check: testing (10, 3.12, bash)
- GitHub Check: testing (10, 3.11, bash)
- GitHub Check: testing (8, 3.11, bash)
- GitHub Check: testing (7, 3.11, bash)
- GitHub Check: testing (6, 3.12, bash)
- GitHub Check: testing (6, 3.11, bash)
- GitHub Check: testing (5, 3.12, bash)
- GitHub Check: testing (5, 3.11, bash)
- GitHub Check: testing-windows (10)
- GitHub Check: testing (4, 3.12, bash)
- GitHub Check: testing-windows (9)
- GitHub Check: testing (4, 3.11, bash)
- GitHub Check: testing-windows (8)
- GitHub Check: testing (3, 3.12, bash)
- GitHub Check: testing-windows (7)
- GitHub Check: testing (3, 3.11, bash)
- GitHub Check: testing-windows (6)
- GitHub Check: testing (2, 3.12, bash)
- GitHub Check: testing-windows (5)
- GitHub Check: testing (2, 3.11, bash)
- GitHub Check: testing-windows (4)
- GitHub Check: testing (1, 3.12, bash)
- GitHub Check: testing-windows (3)
- GitHub Check: testing (1, 3.11, bash)
- GitHub Check: testing-windows (2)
- GitHub Check: testing-windows (1)
🔇 Additional comments (6)
tests/test_config_replacing/Snakefile (2)
2-6: LGTM: Clean configuration handling with fallbacks.This is a well-structured approach for loading configuration values with appropriate fallback defaults, which aligns perfectly with testing the
--replace-workflow-configfunctionality mentioned in the PR objectives.
8-17: LGTM: Appropriate test structure for configuration replacement testing.The rules are well-designed to test the configuration replacement feature. The
createrule outputs the configuration values to a file, allowing verification of how the configuration values are processed with and without the new--replace-workflow-configflag.tests/test_config_replacing_nocli/Snakefile (4)
2-2: LGTM: Config file loading looks correct.This correctly uses Snakemake's
configfiledirective to load the workflow configuration from a YAML file.
4-6: LGTM: Appropriate fallback values provided.Good defensive programming practice to check if keys exist in the config dictionary and provide sensible default values if they don't. This will help make the test robust against different configuration scenarios.
8-11: LGTM: Rule definition follows best practices.The
allrule correctly specifies the expected final output of the workflow.
12-17: LGTM: Output creation uses config values correctly.The
createrule properly uses the extracted configuration values in the shell command. This will be useful for testing whether the configuration was loaded and processed correctly.
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
snakemake/cli.py(2 hunks)snakemake/settings/types.py(1 hunks)snakemake/workflow.py(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- snakemake/workflow.py
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
snakemake/cli.pysnakemake/settings/types.py
🪛 Ruff (0.8.2)
snakemake/settings/types.py
359-359: snakemake.logging.logger imported but unused
Remove unused import: snakemake.logging.logger
(F401)
⏰ Context from checks skipped due to timeout of 90000ms (30)
- GitHub Check: testing (10, 3.12, bash)
- GitHub Check: testing (10, 3.11, bash)
- GitHub Check: testing (9, 3.12, bash)
- GitHub Check: testing (9, 3.11, bash)
- GitHub Check: testing (8, 3.12, bash)
- GitHub Check: testing (8, 3.11, bash)
- GitHub Check: testing (7, 3.12, bash)
- GitHub Check: testing (7, 3.11, bash)
- GitHub Check: testing (6, 3.12, bash)
- GitHub Check: testing (6, 3.11, bash)
- GitHub Check: testing (5, 3.12, bash)
- GitHub Check: testing-windows (10)
- GitHub Check: testing (5, 3.11, bash)
- GitHub Check: testing-windows (9)
- GitHub Check: testing (4, 3.12, bash)
- GitHub Check: testing-windows (8)
- GitHub Check: testing (4, 3.11, bash)
- GitHub Check: testing-windows (7)
- GitHub Check: testing (3, 3.12, bash)
- GitHub Check: testing-windows (6)
- GitHub Check: testing (3, 3.11, bash)
- GitHub Check: testing-windows (5)
- GitHub Check: testing (2, 3.12, bash)
- GitHub Check: testing-windows (4)
- GitHub Check: testing (2, 3.11, bash)
- GitHub Check: testing-windows (3)
- GitHub Check: testing (1, 3.12, bash)
- GitHub Check: testing-windows (2)
- GitHub Check: testing (1, 3.11, bash)
- GitHub Check: testing-windows (1)
🔇 Additional comments (3)
snakemake/settings/types.py (1)
351-351: LGTM: New configuration attribute added.The
replace_workflow_configattribute with a default value ofFalseis a good addition to control whether the config dictionary from the command line should replace or extend the workflow's config dictionary.snakemake/cli.py (2)
688-697: LGTM: Well-documented command line argument for replacing workflow config.The new
--replace-workflow-configflag is clearly documented with a good explanation of its purpose and behavior. This implementation addresses issue #730 as mentioned in the PR objectives.
2058-2058: LGTM: Proper integration with the API.The
replace_workflow_configparameter is correctly passed from the command line arguments to theConfigSettingsclass.
snakemake/settings/types.py
Outdated
| from snakemake.logging import logger | ||
|
|
There was a problem hiding this comment.
Remove unused import.
The logger import is not being used in the _get_overwrite_config method.
- from snakemake.logging import logger
-📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| from snakemake.logging import logger |
🧰 Tools
🪛 Ruff (0.8.2)
359-359: snakemake.logging.logger imported but unused
Remove unused import: snakemake.logging.logger
(F401)
|
The flag changes the behavior of configfile merging. When providing configfiles via cli, they replace workflow configs instead of extending/updating them. Adresses snakemake#730.
Adds a testcase that checks for workflow config keys to be discarded if not redefined in the cli configfiles.
Checks that the flag does not break the workflow config if no configfile is provided via cli.
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
64d9624 to
aca2a27
Compare
🤖 I have created a release *beep* *boop* --- ## [9.0.0](v8.30.0...v9.0.0) (2025-03-14) ### ⚠ BREAKING CHANGES * Logging refactor & add LoggerPluginInterface ([#3107](#3107)) ### Features * [#3412](#3412) - keep shadow folder of failed job if --keep-incomplete flag is set. ([#3430](#3430)) ([22978c3](22978c3)) * add flag --report-after-run to automatically generate the report after a successfull workflow run ([#3428](#3428)) ([b0a7f03](b0a7f03)) * add flatten function to IO utils ([#3424](#3424)) ([67fa392](67fa392)) * add helper functions to parse input files ([#2918](#2918)) ([63e45a7](63e45a7)) * Add option to print redacted file names ([#3089](#3089)) ([ba4d264](ba4d264)) * add support for validation of polars dataframe and lazyframe ([#3262](#3262)) ([c7473a6](c7473a6)) * added support for rendering dag with mermaid js ([#3409](#3409)) ([7bf8381](7bf8381)) * adding --replace-workflow-config to fully replace workflow configs (from config: directive) with --configfile, instead of merging them ([#3381](#3381)) ([47504a0](47504a0)) * Dynamic module name ([#3401](#3401)) ([024dc32](024dc32)) * Enable saving and reloading IOCache object ([#3386](#3386)) ([c935953](c935953)) * files added in rule params with workflow.source_path will be available in used containers ([#3385](#3385)) ([a6e45bf](a6e45bf)) * Fix keep_local in storage directive and more freedom over remote retrieval behaviour ([#3410](#3410)) ([67b4739](67b4739)) * inherit parameters of use rule and extend/replace individual items them when using 'with' directive ([#3365](#3365)) ([93e4b92](93e4b92)) * Logging refactor & add LoggerPluginInterface ([#3107](#3107)) ([86f1d6e](86f1d6e)) * Maximal file size for checksums ([#3368](#3368)) ([b039f8a](b039f8a)) * Modernize package configuration using Pixi ([#3369](#3369)) ([77992d8](77992d8)) * multiext support for named input/output ([#3372](#3372)) ([05e1378](05e1378)) * optionally auto-group jobs via temp files in case of remote execution ([#3378](#3378)) ([cc9bba2](cc9bba2)) ### Bug Fixes * `--delete-all-output` ignores `--dry-run` ([#3265](#3265)) ([23fef82](23fef82)) * 3342 faster touch runs and warning messages for non-existing files ([#3398](#3398)) ([cd9c3c3](cd9c3c3)) * add default value to max-jobs-per-timespan ([#3043](#3043)) ([2959abe](2959abe)) * checkpoints inside modules are overwritten ([#3359](#3359)) ([fba3ac7](fba3ac7)) * Convert Path to IOFile ([#3405](#3405)) ([c58684c](c58684c)) * Do not perform storage object cleanup with --keep-storage-local-copies set ([#3358](#3358)) ([9a6d14b](9a6d14b)) * edgecases of source deployment in case of remote execution ([#3396](#3396)) ([5da13be](5da13be)) * enhance error message formatting for strict DAG-building mode ([#3376](#3376)) ([a1c39ee](a1c39ee)) * fix bug in checkpoint handling that led to exceptions in case checkpoint output was missing upon rerun ([#3423](#3423)) ([8cf4a2f](8cf4a2f)) * force check all required outputs ([#3341](#3341)) ([495a4e7](495a4e7)) * group job formatting ([#3442](#3442)) ([f0b10a3](f0b10a3)) * in remote jobs, upload storage in topological order such that modification dates are preserved (e.g. in case of group jobs) ([#3377](#3377)) ([eace08f](eace08f)) * only skip eval when resource depends on input ([#3374](#3374)) ([4574c92](4574c92)) * Prevent execution of conda in apptainer when not explicitly requested in software deployment method ([#3388](#3388)) ([c43c5c0](c43c5c0)) * print filenames with quotes around them in RuleException ([#3269](#3269)) ([6baeda5](6baeda5)) * Re-evaluation of free resources ([#3399](#3399)) ([6371293](6371293)) * ReadTheDocs layout issue due to src directory change ([#3419](#3419)) ([695b127](695b127)) * robustly escaping quotes in generated bash scripts (v2) ([#3297](#3297)) ([#3389](#3389)) ([58720bd](58720bd)) * Show apptainer image URL in snakemake report ([#3407](#3407)) ([45f0450](45f0450)) * Update ReadTheDocs configuration for documentation build to use Pixi ([#3433](#3433)) ([3f227a6](3f227a6)) ### Documentation * Add pixi setup instructions to general use tutorial ([#3382](#3382)) ([115e81b](115e81b)) * fix contribution section heading levels, fix docs testing setup order ([#3360](#3360)) ([051dc53](051dc53)) * fix link to github.com/snakemake/poetry-snakemake-plugin ([#3436](#3436)) ([ec6d97c](ec6d97c)) * fix quoting ([#3394](#3394)) ([b40f599](b40f599)) * fix rerun-triggers default ([#3403](#3403)) ([4430e23](4430e23)) * fix typo 'safe' -> 'save' ([#3384](#3384)) ([7755861](7755861)) * mention code formatting in the contribution section ([#3431](#3431)) ([e8682b7](e8682b7)) * remove duplicated 'functions'. ([#3356](#3356)) ([7c595db](7c595db)) * update broken links documentation ([#3437](#3437)) ([e3d0d88](e3d0d88)) * Updating contributing guidelines with new pixi dev setup ([#3415](#3415)) ([8e95a12](8e95a12)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: snakemake-bot <snakemake-bot-admin@googlegroups.com>
…igs (from config: directive) with --configfile, instead of merging them (snakemake#3381) <!--Add a description of your PR here--> Adds a new flag `--replace-workflow-config` to change the behavior of config file merging: Instead of extended und updating the values in the config dictionary of the workflow, the config dictionary provided via CLI replaces the workflow dictionary entirely. Keys that are not redefined via the CLI dictionary are not present in the final config. This addresses the request from snakemake#730 ### QC <!-- Make sure that you can tick the boxes below. --> * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit - **New Features** - Introduced a new command‐line option allowing users to fully replace the workflow configuration with a provided configuration file, ensuring only the specified settings are applied. - Enhanced logging now clearly indicates whether configurations are being extended or replaced, providing clearer insights during execution. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
🤖 I have created a release *beep* *boop* --- ## [9.0.0](snakemake/snakemake@v8.30.0...v9.0.0) (2025-03-14) ### ⚠ BREAKING CHANGES * Logging refactor & add LoggerPluginInterface ([snakemake#3107](snakemake#3107)) ### Features * [snakemake#3412](snakemake#3412) - keep shadow folder of failed job if --keep-incomplete flag is set. ([snakemake#3430](snakemake#3430)) ([22978c3](snakemake@22978c3)) * add flag --report-after-run to automatically generate the report after a successfull workflow run ([snakemake#3428](snakemake#3428)) ([b0a7f03](snakemake@b0a7f03)) * add flatten function to IO utils ([snakemake#3424](snakemake#3424)) ([67fa392](snakemake@67fa392)) * add helper functions to parse input files ([snakemake#2918](snakemake#2918)) ([63e45a7](snakemake@63e45a7)) * Add option to print redacted file names ([snakemake#3089](snakemake#3089)) ([ba4d264](snakemake@ba4d264)) * add support for validation of polars dataframe and lazyframe ([snakemake#3262](snakemake#3262)) ([c7473a6](snakemake@c7473a6)) * added support for rendering dag with mermaid js ([snakemake#3409](snakemake#3409)) ([7bf8381](snakemake@7bf8381)) * adding --replace-workflow-config to fully replace workflow configs (from config: directive) with --configfile, instead of merging them ([snakemake#3381](snakemake#3381)) ([47504a0](snakemake@47504a0)) * Dynamic module name ([snakemake#3401](snakemake#3401)) ([024dc32](snakemake@024dc32)) * Enable saving and reloading IOCache object ([snakemake#3386](snakemake#3386)) ([c935953](snakemake@c935953)) * files added in rule params with workflow.source_path will be available in used containers ([snakemake#3385](snakemake#3385)) ([a6e45bf](snakemake@a6e45bf)) * Fix keep_local in storage directive and more freedom over remote retrieval behaviour ([snakemake#3410](snakemake#3410)) ([67b4739](snakemake@67b4739)) * inherit parameters of use rule and extend/replace individual items them when using 'with' directive ([snakemake#3365](snakemake#3365)) ([93e4b92](snakemake@93e4b92)) * Logging refactor & add LoggerPluginInterface ([snakemake#3107](snakemake#3107)) ([86f1d6e](snakemake@86f1d6e)) * Maximal file size for checksums ([snakemake#3368](snakemake#3368)) ([b039f8a](snakemake@b039f8a)) * Modernize package configuration using Pixi ([snakemake#3369](snakemake#3369)) ([77992d8](snakemake@77992d8)) * multiext support for named input/output ([snakemake#3372](snakemake#3372)) ([05e1378](snakemake@05e1378)) * optionally auto-group jobs via temp files in case of remote execution ([snakemake#3378](snakemake#3378)) ([cc9bba2](snakemake@cc9bba2)) ### Bug Fixes * `--delete-all-output` ignores `--dry-run` ([snakemake#3265](snakemake#3265)) ([23fef82](snakemake@23fef82)) * 3342 faster touch runs and warning messages for non-existing files ([snakemake#3398](snakemake#3398)) ([cd9c3c3](snakemake@cd9c3c3)) * add default value to max-jobs-per-timespan ([snakemake#3043](snakemake#3043)) ([2959abe](snakemake@2959abe)) * checkpoints inside modules are overwritten ([snakemake#3359](snakemake#3359)) ([fba3ac7](snakemake@fba3ac7)) * Convert Path to IOFile ([snakemake#3405](snakemake#3405)) ([c58684c](snakemake@c58684c)) * Do not perform storage object cleanup with --keep-storage-local-copies set ([snakemake#3358](snakemake#3358)) ([9a6d14b](snakemake@9a6d14b)) * edgecases of source deployment in case of remote execution ([snakemake#3396](snakemake#3396)) ([5da13be](snakemake@5da13be)) * enhance error message formatting for strict DAG-building mode ([snakemake#3376](snakemake#3376)) ([a1c39ee](snakemake@a1c39ee)) * fix bug in checkpoint handling that led to exceptions in case checkpoint output was missing upon rerun ([snakemake#3423](snakemake#3423)) ([8cf4a2f](snakemake@8cf4a2f)) * force check all required outputs ([snakemake#3341](snakemake#3341)) ([495a4e7](snakemake@495a4e7)) * group job formatting ([snakemake#3442](snakemake#3442)) ([f0b10a3](snakemake@f0b10a3)) * in remote jobs, upload storage in topological order such that modification dates are preserved (e.g. in case of group jobs) ([snakemake#3377](snakemake#3377)) ([eace08f](snakemake@eace08f)) * only skip eval when resource depends on input ([snakemake#3374](snakemake#3374)) ([4574c92](snakemake@4574c92)) * Prevent execution of conda in apptainer when not explicitly requested in software deployment method ([snakemake#3388](snakemake#3388)) ([c43c5c0](snakemake@c43c5c0)) * print filenames with quotes around them in RuleException ([snakemake#3269](snakemake#3269)) ([6baeda5](snakemake@6baeda5)) * Re-evaluation of free resources ([snakemake#3399](snakemake#3399)) ([6371293](snakemake@6371293)) * ReadTheDocs layout issue due to src directory change ([snakemake#3419](snakemake#3419)) ([695b127](snakemake@695b127)) * robustly escaping quotes in generated bash scripts (v2) ([snakemake#3297](snakemake#3297)) ([snakemake#3389](snakemake#3389)) ([58720bd](snakemake@58720bd)) * Show apptainer image URL in snakemake report ([snakemake#3407](snakemake#3407)) ([45f0450](snakemake@45f0450)) * Update ReadTheDocs configuration for documentation build to use Pixi ([snakemake#3433](snakemake#3433)) ([3f227a6](snakemake@3f227a6)) ### Documentation * Add pixi setup instructions to general use tutorial ([snakemake#3382](snakemake#3382)) ([115e81b](snakemake@115e81b)) * fix contribution section heading levels, fix docs testing setup order ([snakemake#3360](snakemake#3360)) ([051dc53](snakemake@051dc53)) * fix link to github.com/snakemake/poetry-snakemake-plugin ([snakemake#3436](snakemake#3436)) ([ec6d97c](snakemake@ec6d97c)) * fix quoting ([snakemake#3394](snakemake#3394)) ([b40f599](snakemake@b40f599)) * fix rerun-triggers default ([snakemake#3403](snakemake#3403)) ([4430e23](snakemake@4430e23)) * fix typo 'safe' -> 'save' ([snakemake#3384](snakemake#3384)) ([7755861](snakemake@7755861)) * mention code formatting in the contribution section ([snakemake#3431](snakemake#3431)) ([e8682b7](snakemake@e8682b7)) * remove duplicated 'functions'. ([snakemake#3356](snakemake#3356)) ([7c595db](snakemake@7c595db)) * update broken links documentation ([snakemake#3437](snakemake#3437)) ([e3d0d88](snakemake@e3d0d88)) * Updating contributing guidelines with new pixi dev setup ([snakemake#3415](snakemake#3415)) ([8e95a12](snakemake@8e95a12)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: snakemake-bot <snakemake-bot-admin@googlegroups.com>
Adds a new flag
--replace-workflow-configto change the behavior of config file merging: Instead of extended und updating the values in the config dictionary of the workflow, the config dictionary provided via CLI replaces the workflow dictionary entirely. Keys that are not redefined via the CLI dictionary are not present in the final config.This addresses the request from #730
QC
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).Summary by CodeRabbit