feat: files added in rule params with workflow.source_path will be available in used containers #3385
Conversation
…vailable in the container by default by mounting the .cache/snakemake/snakemake/source-cache/ if no other --bind for the container is given
…de the home folder from the standard mounted folders
📝 WalkthroughWalkthroughThis pull request introduces new test functions and documentation enhancements for container-based Snakemake workflows. It adds two test functions in Changes
Sequence Diagram(s)sequenceDiagram
participant T as Test Function
participant R as Runner (run method)
participant S as Snakemake
participant D as Docker Container
T->>R: Invoke test_issue3361_pass/test_issue3361_fail
R->>S: Execute "snakemake --sdm apptainer" with target ["all"]
S->>D: Run workflow as defined in Snakefile (copy_in_container rule)
D->>S: Return generated output
S->>R: Relay command results
R->>T: Validate output or expected error condition
sequenceDiagram
participant SC as shellcmd Function
participant GA as get_appdirs()
participant FS as Filesystem
participant EX as Command Executor
SC->>GA: Query user cache directory
GA-->>SC: Return cache directory path
SC->>FS: Check existence of the cache directory
alt Directory exists
FS-->>SC: Directory found
SC->>SC: Append bind mount argument for source cache
else Directory not found
SC->>SC: Log debug message (skip bind mount)
end
SC->>EX: Execute constructed shell command
Possibly related PRs
Suggested reviewers
Tip ⚡🧪 Multi-step agentic review comment chat (experimental)
📜 Recent review detailsConfiguration used: CodeRabbit UI 📒 Files selected for processing (2)
💤 Files with no reviewable changes (1)
🚧 Files skipped from review as they are similar to previous changes (1)
⏰ Context from checks skipped due to timeout of 90000ms (43)
🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
snakemake/deployment/singularity.py (1)
130-134: Fix typos in comments and ensure proper error handling.The implementation correctly mounts the source cache directory by default, which aligns with the PR objective. However, there are typos in the comments, and it might be beneficial to add error handling for cases where the cache directory doesn't exist.
-# mount the snakemake cache into the container per detault so that -# params included with workflow.source_path are allways mounted in the conteiner +# mount the snakemake cache into the container by default so that +# params included with workflow.source_path are always mounted in the container if len(args) == 0: - args += "--bind " + os.path.join(get_appdirs().user_cache_dir, "snakemake/source-cache") + cache_path = os.path.join(get_appdirs().user_cache_dir, "snakemake/source-cache") + if os.path.exists(cache_path): + args += "--bind " + cache_pathAdditionally, consider adding a note in the PR description or documentation about the security implications of mounting the user's cache directory, especially when used with the
--no-homeflag as mentioned in the PR objectives.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (7)
snakemake/deployment/singularity.py(2 hunks)snakemake/workflow.py(1 hunks)tests/test_issue3361/Snakefile(1 hunks)tests/test_issue3361/expected-results/output.txt(1 hunks)tests/test_issue3361/input.txt(1 hunks)tests/test_issue3361/some_dir/some_other_input.txt(1 hunks)tests/tests.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
snakemake/workflow.pysnakemake/deployment/singularity.pytests/tests.py
🔇 Additional comments (6)
tests/test_issue3361/input.txt (1)
1-2: LGTM! Appropriate test input file.Simple test input file with the expected content for the container access test.
tests/test_issue3361/some_dir/some_other_input.txt (1)
1-2: LGTM! Appropriate test file in subdirectory.Well-positioned test file that will be accessed via workflow.source_path from the container, which directly tests the functionality being implemented in this PR.
tests/test_issue3361/expected-results/output.txt (1)
1-3: LGTM! Properly formatted expected test output.The expected output file correctly contains the concatenated content of both input files, which will be used to validate the test results.
tests/test_issue3361/Snakefile (1)
1-16: LGTM! Well-designed test workflow for container source path access.This Snakefile effectively tests the PR's main functionality by:
- Creating a simple workflow with container execution
- Using workflow.source_path() to reference a file in a subdirectory
- Verifying the concatenated output matches the expected result
The test directly validates that files specified with workflow.source_path in params are accessible within the container environment.
tests/tests.py (1)
2051-2054: LGTM: Good test addition for the new feature.The test properly validates that files specified with
workflow.source_pathare accessible in containers by running Snakemake with the Apptainer deployment method.snakemake/deployment/singularity.py (1)
21-21: LGTM: Appropriate import for accessing user cache directory.This import is necessary for accessing the user's cache directory where Snakemake stores its source cache.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
snakemake/deployment/singularity.py (2)
130-134: Fix spelling errors in the comments.There are spelling errors in the comments that should be corrected.
- # mount the snakemake cache into the container per detault so that - # params included with workflow.source_path are allways mounted in the conteiner + # mount the snakemake cache into the container per default so that + # params included with workflow.source_path are always mounted in the container if len(args) == 0: args += "--bind " + os.path.join(get_appdirs().user_cache_dir, "snakemake/source-cache")🧰 Tools
🪛 GitHub Check: Check for spelling errors
[failure] 130-130:
detault ==> default
[failure] 131-131:
allways ==> always
[failure] 131-131:
conteiner ==> container🪛 GitHub Actions: Codespell
[error] 130-130: codespell: 'detault' is a misspelling; suggested correction is 'default'.
130-134: Good implementation for automatically mounting source cache files.The implementation correctly handles mounting the Snakemake source cache directory when no additional arguments are provided, ensuring that files referenced with
workflow.source_pathare available in containers. This addresses the stated PR objective.Consider adding a more detailed comment or log message to make users aware of this behavior, especially considering the security implications of mounting directories from the user's home.
🧰 Tools
🪛 GitHub Check: Check for spelling errors
[failure] 130-130:
detault ==> default
[failure] 131-131:
allways ==> always
[failure] 131-131:
conteiner ==> container🪛 GitHub Actions: Codespell
[error] 130-130: codespell: 'detault' is a misspelling; suggested correction is 'default'.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
snakemake/deployment/singularity.py(2 hunks)snakemake/workflow.py(1 hunks)docs/snakefiles/deployment.rst(1 hunks)snakemake/deployment/singularity.py(1 hunks)snakemake/workflow.py(0 hunks)
🚧 Files skipped from review as they are similar to previous changes (3)
- snakemake/workflow.py
- snakemake/workflow.py
- snakemake/deployment/singularity.py
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
snakemake/deployment/singularity.py
🪛 GitHub Check: Check for spelling errors
snakemake/deployment/singularity.py
[failure] 130-130:
detault ==> default
[failure] 131-131:
allways ==> always
[failure] 131-131:
conteiner ==> container
🪛 GitHub Actions: Codespell
snakemake/deployment/singularity.py
[error] 130-130: codespell: 'detault' is a misspelling; suggested correction is 'default'.
🪛 GitHub Actions: CI
snakemake/deployment/singularity.py
[error] 19-19: Black formatting check failed. 2 files would be reformatted, 141 files would be left unchanged.
🔇 Additional comments (2)
snakemake/deployment/singularity.py (1)
21-21: Import added for supporting the cache mounting feature.The import for
get_appdirsis correctly added to access the user cache directory needed to implement the new functionality.docs/snakefiles/deployment.rst (1)
455-457: Documentation clearly explains the new functionality and security considerations.The added documentation clearly explains:
- That files mounted using
paramswithworkflow.source_pathare automatically available in containers- How this is implemented (by mounting the Snakemake cache)
- Important security considerations about container access to the home directory
This documentation aligns well with the code changes and provides users with the necessary information about both the functionality and security implications.
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
snakemake/deployment/singularity.py(2 hunks)tests/tests.py(1 hunks)snakemake/deployment/singularity.py(1 hunks)snakemake/deployment/singularity.py(2 hunks)tests/tests.py(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
- snakemake/deployment/singularity.py
- tests/tests.py
- snakemake/deployment/singularity.py
- tests/tests.py
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
snakemake/deployment/singularity.py
🪛 GitHub Check: Check for spelling errors
snakemake/deployment/singularity.py
[failure] 130-130:
detault ==> default
[failure] 131-131:
allways ==> always
[failure] 131-131:
conteiner ==> container
🪛 GitHub Actions: Codespell
snakemake/deployment/singularity.py
[error] 130-130: codespell: 'detault' is a misspelling of 'default'.
⏰ Context from checks skipped due to timeout of 90000ms (30)
- GitHub Check: testing (10, 3.12, bash)
- GitHub Check: testing (10, 3.11, bash)
- GitHub Check: testing (9, 3.12, bash)
- GitHub Check: testing (9, 3.11, bash)
- GitHub Check: testing (8, 3.12, bash)
- GitHub Check: testing (8, 3.11, bash)
- GitHub Check: testing (7, 3.12, bash)
- GitHub Check: testing (7, 3.11, bash)
- GitHub Check: testing (6, 3.12, bash)
- GitHub Check: testing (6, 3.11, bash)
- GitHub Check: testing (5, 3.12, bash)
- GitHub Check: testing (5, 3.11, bash)
- GitHub Check: testing (4, 3.12, bash)
- GitHub Check: testing (4, 3.11, bash)
- GitHub Check: testing-windows (10)
- GitHub Check: testing (3, 3.12, bash)
- GitHub Check: testing-windows (9)
- GitHub Check: testing-windows (8)
- GitHub Check: testing (3, 3.11, bash)
- GitHub Check: testing-windows (7)
- GitHub Check: testing (2, 3.12, bash)
- GitHub Check: testing-windows (6)
- GitHub Check: testing (2, 3.11, bash)
- GitHub Check: testing-windows (5)
- GitHub Check: testing (1, 3.12, bash)
- GitHub Check: testing-windows (4)
- GitHub Check: testing (1, 3.11, bash)
- GitHub Check: testing-windows (3)
- GitHub Check: testing-windows (2)
- GitHub Check: testing-windows (1)
🔇 Additional comments (1)
snakemake/deployment/singularity.py (1)
21-21: Good addition of necessary import.Adding the import for
get_appdirsfromsnakemake.commonis necessary for the new functionality that mounts the source cache.
| get_appdirs().user_cache_dir, "snakemake/source-cache" | ||
| ) | ||
|
|
||
| cmd = "{} singularity {} exec --home {} {} {} {} -c '{}'".format( |
There was a problem hiding this comment.
but here it binds --home doesn't it? How can it be that this does not already include the cache dir?
There was a problem hiding this comment.
Thanks for the feedback!
We just had a deeper look and it seems that the cwd is bound as "home" and not the home of the user. So we think binding the source-cache is still needed to make the sourced files accessible in the container. "--no-home" flag is then not needed since "home" is mounted as the current cwd which makes sense.
We also further tested the capability of singularity to write and see files in the higher level folders and this fails as expected. We think this makes the use of containers more save then expected and we also implemented a test for that.
We will provide the update in the next commit.
…estricted rights on the FS
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
snakemake/deployment/singularity.py (1)
130-136: Consider adding a user warning about container security implicationsGiven the PR objectives mention security concerns about giving containers access to the user's cache directory, consider adding a warning log message when this mount is added automatically, especially for users who might not be aware of this behavior.
if len(args) == 0: + logger.warning( + "Mounting Snakemake source cache into container. " + "Only use trusted containers as they will have access to cached workflow files." + ) args += "--bind " + os.path.join( get_appdirs().user_cache_dir, "snakemake/source-cache" )
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
docs/snakefiles/deployment.rst(1 hunks)snakemake/deployment/singularity.py(2 hunks)tests/tests.py(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- tests/tests.py
- docs/snakefiles/deployment.rst
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
snakemake/deployment/singularity.py
🔇 Additional comments (1)
snakemake/deployment/singularity.py (1)
20-20: Import addition for accessing user cache directoryThe new import of
get_appdirsfromsnakemake.commonis necessary for determining the location of the Snakemake source cache directory, which is used in the updatedshellcmdfunction.
…orce-add for git and reformated the tests.py after merge conflict
|
Update in upstream repo
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
src/snakemake/deployment/singularity.py(2 hunks)src/snakemake/workflow.py(0 hunks)tests/tests.py(1 hunks)
💤 Files with no reviewable changes (1)
- src/snakemake/workflow.py
🚧 Files skipped from review as they are similar to previous changes (1)
- tests/tests.py
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest ...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
src/snakemake/deployment/singularity.py
⏰ Context from checks skipped due to timeout of 90000ms (18)
- GitHub Check: tests (10, windows-latest, py312, bash)
- GitHub Check: tests (10, windows-latest, py311, bash)
- GitHub Check: tests (10, ubuntu-latest, py312, bash)
- GitHub Check: tests (10, ubuntu-latest, py311, bash)
- GitHub Check: tests (4, ubuntu-latest, py312, bash)
- GitHub Check: tests (4, ubuntu-latest, py311, bash)
- GitHub Check: tests (3, windows-latest, py312, bash)
- GitHub Check: tests (3, windows-latest, py311, bash)
- GitHub Check: tests (3, ubuntu-latest, py312, bash)
- GitHub Check: tests (3, ubuntu-latest, py311, bash)
- GitHub Check: tests (2, windows-latest, py312, bash)
- GitHub Check: tests (2, windows-latest, py311, bash)
- GitHub Check: tests (2, ubuntu-latest, py312, bash)
- GitHub Check: tests (2, ubuntu-latest, py311, bash)
- GitHub Check: tests (1, windows-latest, py312, bash)
- GitHub Check: tests (1, windows-latest, py311, bash)
- GitHub Check: tests (1, ubuntu-latest, py312, bash)
- GitHub Check: tests (1, ubuntu-latest, py311, bash)
🔇 Additional comments (1)
src/snakemake/deployment/singularity.py (1)
20-20: Good addition of required import.The import of
get_appdirsfromsnakemake.commonis appropriately added to support the new feature for mounting the Snakemake cache into containers.
As done recently in the singularity implementation, we need to mount the cache folder in the path where XDG expects to find the cache. The template will need to change when adding support for apptainer, but we will converge on that later. - Related: snakemake/snakemake#3385
🤖 I have created a release *beep* *boop* --- ## [9.0.0](v8.30.0...v9.0.0) (2025-03-14) ### ⚠ BREAKING CHANGES * Logging refactor & add LoggerPluginInterface ([#3107](#3107)) ### Features * [#3412](#3412) - keep shadow folder of failed job if --keep-incomplete flag is set. ([#3430](#3430)) ([22978c3](22978c3)) * add flag --report-after-run to automatically generate the report after a successfull workflow run ([#3428](#3428)) ([b0a7f03](b0a7f03)) * add flatten function to IO utils ([#3424](#3424)) ([67fa392](67fa392)) * add helper functions to parse input files ([#2918](#2918)) ([63e45a7](63e45a7)) * Add option to print redacted file names ([#3089](#3089)) ([ba4d264](ba4d264)) * add support for validation of polars dataframe and lazyframe ([#3262](#3262)) ([c7473a6](c7473a6)) * added support for rendering dag with mermaid js ([#3409](#3409)) ([7bf8381](7bf8381)) * adding --replace-workflow-config to fully replace workflow configs (from config: directive) with --configfile, instead of merging them ([#3381](#3381)) ([47504a0](47504a0)) * Dynamic module name ([#3401](#3401)) ([024dc32](024dc32)) * Enable saving and reloading IOCache object ([#3386](#3386)) ([c935953](c935953)) * files added in rule params with workflow.source_path will be available in used containers ([#3385](#3385)) ([a6e45bf](a6e45bf)) * Fix keep_local in storage directive and more freedom over remote retrieval behaviour ([#3410](#3410)) ([67b4739](67b4739)) * inherit parameters of use rule and extend/replace individual items them when using 'with' directive ([#3365](#3365)) ([93e4b92](93e4b92)) * Logging refactor & add LoggerPluginInterface ([#3107](#3107)) ([86f1d6e](86f1d6e)) * Maximal file size for checksums ([#3368](#3368)) ([b039f8a](b039f8a)) * Modernize package configuration using Pixi ([#3369](#3369)) ([77992d8](77992d8)) * multiext support for named input/output ([#3372](#3372)) ([05e1378](05e1378)) * optionally auto-group jobs via temp files in case of remote execution ([#3378](#3378)) ([cc9bba2](cc9bba2)) ### Bug Fixes * `--delete-all-output` ignores `--dry-run` ([#3265](#3265)) ([23fef82](23fef82)) * 3342 faster touch runs and warning messages for non-existing files ([#3398](#3398)) ([cd9c3c3](cd9c3c3)) * add default value to max-jobs-per-timespan ([#3043](#3043)) ([2959abe](2959abe)) * checkpoints inside modules are overwritten ([#3359](#3359)) ([fba3ac7](fba3ac7)) * Convert Path to IOFile ([#3405](#3405)) ([c58684c](c58684c)) * Do not perform storage object cleanup with --keep-storage-local-copies set ([#3358](#3358)) ([9a6d14b](9a6d14b)) * edgecases of source deployment in case of remote execution ([#3396](#3396)) ([5da13be](5da13be)) * enhance error message formatting for strict DAG-building mode ([#3376](#3376)) ([a1c39ee](a1c39ee)) * fix bug in checkpoint handling that led to exceptions in case checkpoint output was missing upon rerun ([#3423](#3423)) ([8cf4a2f](8cf4a2f)) * force check all required outputs ([#3341](#3341)) ([495a4e7](495a4e7)) * group job formatting ([#3442](#3442)) ([f0b10a3](f0b10a3)) * in remote jobs, upload storage in topological order such that modification dates are preserved (e.g. in case of group jobs) ([#3377](#3377)) ([eace08f](eace08f)) * only skip eval when resource depends on input ([#3374](#3374)) ([4574c92](4574c92)) * Prevent execution of conda in apptainer when not explicitly requested in software deployment method ([#3388](#3388)) ([c43c5c0](c43c5c0)) * print filenames with quotes around them in RuleException ([#3269](#3269)) ([6baeda5](6baeda5)) * Re-evaluation of free resources ([#3399](#3399)) ([6371293](6371293)) * ReadTheDocs layout issue due to src directory change ([#3419](#3419)) ([695b127](695b127)) * robustly escaping quotes in generated bash scripts (v2) ([#3297](#3297)) ([#3389](#3389)) ([58720bd](58720bd)) * Show apptainer image URL in snakemake report ([#3407](#3407)) ([45f0450](45f0450)) * Update ReadTheDocs configuration for documentation build to use Pixi ([#3433](#3433)) ([3f227a6](3f227a6)) ### Documentation * Add pixi setup instructions to general use tutorial ([#3382](#3382)) ([115e81b](115e81b)) * fix contribution section heading levels, fix docs testing setup order ([#3360](#3360)) ([051dc53](051dc53)) * fix link to github.com/snakemake/poetry-snakemake-plugin ([#3436](#3436)) ([ec6d97c](ec6d97c)) * fix quoting ([#3394](#3394)) ([b40f599](b40f599)) * fix rerun-triggers default ([#3403](#3403)) ([4430e23](4430e23)) * fix typo 'safe' -> 'save' ([#3384](#3384)) ([7755861](7755861)) * mention code formatting in the contribution section ([#3431](#3431)) ([e8682b7](e8682b7)) * remove duplicated 'functions'. ([#3356](#3356)) ([7c595db](7c595db)) * update broken links documentation ([#3437](#3437)) ([e3d0d88](e3d0d88)) * Updating contributing guidelines with new pixi dev setup ([#3415](#3415)) ([8e95a12](8e95a12)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: snakemake-bot <snakemake-bot-admin@googlegroups.com>
…ailable in used containers (snakemake#3385) If files are specified under `params` using workflow.source_path, they are also automatically available in the container. This was achieved by mounting the nextflow source cache in the container by default. This achieves a behavior which is rather natural, as the user probably expects that files which are specified in params using workflow.source_path are also available in the container. Due to the fact that the source cache is normally located under /home/<user>/.cache/snakemake/snakemake/source-cache and the container thus gets access to the home directory despite the use of the --no-home flag used in this PR, the documentation explicitly points out that only trustworthy containers should be used. ### QC * [ x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [ x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit This release improves container workflow execution and documentation, ensuring a smoother and more secure experience for users. - **New Features** - Enhanced container integration now automatically binds local cache directories to improve workflow performance. - Added new test scenarios to validate Snakemake execution with Apptainer deployment. - **Documentation** - Updated guidance on file accessibility within container environments, with added security recommendations to use trusted container images. - **Tests** - Introduced additional test functions to cover various Snakemake execution scenarios. - New input and expected output files added for comprehensive testing. <!-- end of auto-generated comment: release notes by coderabbit.ai -->
🤖 I have created a release *beep* *boop* --- ## [9.0.0](snakemake/snakemake@v8.30.0...v9.0.0) (2025-03-14) ### ⚠ BREAKING CHANGES * Logging refactor & add LoggerPluginInterface ([snakemake#3107](snakemake#3107)) ### Features * [snakemake#3412](snakemake#3412) - keep shadow folder of failed job if --keep-incomplete flag is set. ([snakemake#3430](snakemake#3430)) ([22978c3](snakemake@22978c3)) * add flag --report-after-run to automatically generate the report after a successfull workflow run ([snakemake#3428](snakemake#3428)) ([b0a7f03](snakemake@b0a7f03)) * add flatten function to IO utils ([snakemake#3424](snakemake#3424)) ([67fa392](snakemake@67fa392)) * add helper functions to parse input files ([snakemake#2918](snakemake#2918)) ([63e45a7](snakemake@63e45a7)) * Add option to print redacted file names ([snakemake#3089](snakemake#3089)) ([ba4d264](snakemake@ba4d264)) * add support for validation of polars dataframe and lazyframe ([snakemake#3262](snakemake#3262)) ([c7473a6](snakemake@c7473a6)) * added support for rendering dag with mermaid js ([snakemake#3409](snakemake#3409)) ([7bf8381](snakemake@7bf8381)) * adding --replace-workflow-config to fully replace workflow configs (from config: directive) with --configfile, instead of merging them ([snakemake#3381](snakemake#3381)) ([47504a0](snakemake@47504a0)) * Dynamic module name ([snakemake#3401](snakemake#3401)) ([024dc32](snakemake@024dc32)) * Enable saving and reloading IOCache object ([snakemake#3386](snakemake#3386)) ([c935953](snakemake@c935953)) * files added in rule params with workflow.source_path will be available in used containers ([snakemake#3385](snakemake#3385)) ([a6e45bf](snakemake@a6e45bf)) * Fix keep_local in storage directive and more freedom over remote retrieval behaviour ([snakemake#3410](snakemake#3410)) ([67b4739](snakemake@67b4739)) * inherit parameters of use rule and extend/replace individual items them when using 'with' directive ([snakemake#3365](snakemake#3365)) ([93e4b92](snakemake@93e4b92)) * Logging refactor & add LoggerPluginInterface ([snakemake#3107](snakemake#3107)) ([86f1d6e](snakemake@86f1d6e)) * Maximal file size for checksums ([snakemake#3368](snakemake#3368)) ([b039f8a](snakemake@b039f8a)) * Modernize package configuration using Pixi ([snakemake#3369](snakemake#3369)) ([77992d8](snakemake@77992d8)) * multiext support for named input/output ([snakemake#3372](snakemake#3372)) ([05e1378](snakemake@05e1378)) * optionally auto-group jobs via temp files in case of remote execution ([snakemake#3378](snakemake#3378)) ([cc9bba2](snakemake@cc9bba2)) ### Bug Fixes * `--delete-all-output` ignores `--dry-run` ([snakemake#3265](snakemake#3265)) ([23fef82](snakemake@23fef82)) * 3342 faster touch runs and warning messages for non-existing files ([snakemake#3398](snakemake#3398)) ([cd9c3c3](snakemake@cd9c3c3)) * add default value to max-jobs-per-timespan ([snakemake#3043](snakemake#3043)) ([2959abe](snakemake@2959abe)) * checkpoints inside modules are overwritten ([snakemake#3359](snakemake#3359)) ([fba3ac7](snakemake@fba3ac7)) * Convert Path to IOFile ([snakemake#3405](snakemake#3405)) ([c58684c](snakemake@c58684c)) * Do not perform storage object cleanup with --keep-storage-local-copies set ([snakemake#3358](snakemake#3358)) ([9a6d14b](snakemake@9a6d14b)) * edgecases of source deployment in case of remote execution ([snakemake#3396](snakemake#3396)) ([5da13be](snakemake@5da13be)) * enhance error message formatting for strict DAG-building mode ([snakemake#3376](snakemake#3376)) ([a1c39ee](snakemake@a1c39ee)) * fix bug in checkpoint handling that led to exceptions in case checkpoint output was missing upon rerun ([snakemake#3423](snakemake#3423)) ([8cf4a2f](snakemake@8cf4a2f)) * force check all required outputs ([snakemake#3341](snakemake#3341)) ([495a4e7](snakemake@495a4e7)) * group job formatting ([snakemake#3442](snakemake#3442)) ([f0b10a3](snakemake@f0b10a3)) * in remote jobs, upload storage in topological order such that modification dates are preserved (e.g. in case of group jobs) ([snakemake#3377](snakemake#3377)) ([eace08f](snakemake@eace08f)) * only skip eval when resource depends on input ([snakemake#3374](snakemake#3374)) ([4574c92](snakemake@4574c92)) * Prevent execution of conda in apptainer when not explicitly requested in software deployment method ([snakemake#3388](snakemake#3388)) ([c43c5c0](snakemake@c43c5c0)) * print filenames with quotes around them in RuleException ([snakemake#3269](snakemake#3269)) ([6baeda5](snakemake@6baeda5)) * Re-evaluation of free resources ([snakemake#3399](snakemake#3399)) ([6371293](snakemake@6371293)) * ReadTheDocs layout issue due to src directory change ([snakemake#3419](snakemake#3419)) ([695b127](snakemake@695b127)) * robustly escaping quotes in generated bash scripts (v2) ([snakemake#3297](snakemake#3297)) ([snakemake#3389](snakemake#3389)) ([58720bd](snakemake@58720bd)) * Show apptainer image URL in snakemake report ([snakemake#3407](snakemake#3407)) ([45f0450](snakemake@45f0450)) * Update ReadTheDocs configuration for documentation build to use Pixi ([snakemake#3433](snakemake#3433)) ([3f227a6](snakemake@3f227a6)) ### Documentation * Add pixi setup instructions to general use tutorial ([snakemake#3382](snakemake#3382)) ([115e81b](snakemake@115e81b)) * fix contribution section heading levels, fix docs testing setup order ([snakemake#3360](snakemake#3360)) ([051dc53](snakemake@051dc53)) * fix link to github.com/snakemake/poetry-snakemake-plugin ([snakemake#3436](snakemake#3436)) ([ec6d97c](snakemake@ec6d97c)) * fix quoting ([snakemake#3394](snakemake#3394)) ([b40f599](snakemake@b40f599)) * fix rerun-triggers default ([snakemake#3403](snakemake#3403)) ([4430e23](snakemake@4430e23)) * fix typo 'safe' -> 'save' ([snakemake#3384](snakemake#3384)) ([7755861](snakemake@7755861)) * mention code formatting in the contribution section ([snakemake#3431](snakemake#3431)) ([e8682b7](snakemake@e8682b7)) * remove duplicated 'functions'. ([snakemake#3356](snakemake#3356)) ([7c595db](snakemake@7c595db)) * update broken links documentation ([snakemake#3437](snakemake#3437)) ([e3d0d88](snakemake@e3d0d88)) * Updating contributing guidelines with new pixi dev setup ([snakemake#3415](snakemake#3415)) ([8e95a12](snakemake@8e95a12)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: snakemake-bot <snakemake-bot-admin@googlegroups.com>



If files are specified under
paramsusing workflow.source_path, they are also automatically available in the container. This was achieved by mounting the nextflow source cache in the container by default. This achieves a behavior which is rather natural, as the user probably expects that files which are specified in params using workflow.source_path are also available in the container.Due to the fact that the source cache is normally located under /home//.cache/snakemake/snakemake/source-cache and the container thus gets access to the home directory despite the use of the --no-home flag used in this PR, the documentation explicitly points out that only trustworthy containers should be used.
QC
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).Summary by CodeRabbit
This release improves container workflow execution and documentation, ensuring a smoother and more secure experience for users.