fix: enable docstring assignment in `use rule ... with:` block by Hocnonsense · Pull Request #3710 · snakemake/snakemake

Hocnonsense · 2025-08-26T09:17:12Z

will fix #1678 and fix #3677

enable assign a docstring after use rule ... with:

QC

The PR contains a test case for the changes or the changes are already covered by an existing test case.
The documentation (docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).

Summary by CodeRabbit

New Features
- Support docstrings inside use rule with: blocks, injecting workflow-level documentation.
Refactor
- Unified handling of run-related keywords (run, shell, script, notebook, wrapper, template_engine, cwl) for more consistent parsing.
Bug Fixes
- Clearer, more specific errors for duplicate or misplaced run-related keywords.
- Improved error message when with: blocks lack valid entries, guiding use of keywords, comments, or docstrings.
Tests
- Added test coverage for with: blocks containing docstrings on used rule aliases.

coderabbitai · 2025-08-26T09:17:20Z

📝 Walkthrough

Walkthrough

Parser refactors group run-related sub-automata into a shared mapping and unify keyword checks. UseRule parsing gains support for docstrings inside with-blocks by emitting @workflow.docstring directives. Error messages are updated accordingly. A test Snakefile is adjusted to include a docstring inside a with-clause.

Changes

Cohort / File(s)	Change Summary
Parser: run sub-automata refactor `src/snakemake/parser.py`	Added public `rule_run_subautomata` mapping. `Rule.subautomata` now merges `rule_run_subautomata` with existing `rule_property_subautomata`. Replaced explicit keyword checks with membership in `rule_run_subautomata`. Updated related error messages. Switched to `yield from` after `StopAutomaton`.
Parser: UseRule docstring support `src/snakemake/parser.py`	In `UseRule.block_content`, reordered token checks to prioritize names; added support for comments and strings. Strings within `with:` now emit `["@workflow.docstring(<string>)", "\n"]`. Updated error message to accept docstrings.
Tests: module nested `tests/test_module_nested/Snakefile`	Updated `use rule ... as ...` to `use rule ... as ... with:` and added a docstring line: `a rule from deep module`.

Sequence Diagram(s)

sequenceDiagram
    autonumber
    participant U as User Snakefile
    participant P as Parser
    participant UR as UseRule Automaton
    participant RA as Rule Subautomata
    participant WF as Workflow Builder

    U->>P: use rule X as Y with:
    P->>UR: Enter with-block
    alt Token is string (docstring)
        UR->>WF: emit @workflow.docstring("<text>")
        Note right of WF: Docstring recorded for alias rule
    else Token is name (keyword)
        UR->>RA: delegate to matching subautomaton
        RA-->>UR: property/run-related handling
        UR->>WF: emit corresponding workflow code
    else Token is comment
        UR->>WF: emit comment
    else Unexpected token
        UR-->>P: SyntaxError ("Expecting rule keyword, comment or docstrings...")
    end

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Assessment against linked issues

Objective	Addressed	Explanation
Support docstring-style strings inside use-rule with-blocks [#1678, #3677]	✅
Improve error message to acknowledge docstrings inside use-rule with-blocks [#1678, #3677]	✅

Assessment against linked issues: Out-of-scope changes

Code Change	Explanation
Introduce `rule_run_subautomata` and refactor run-keyword handling (src/snakemake/parser.py, lines not specified)	Not related to docstring support in `use rule ... with:`; modifies grouping and checks for run-related sub-automata.
Replace explicit StopAutomaton iteration with `yield from self.block(e.token)` (src/snakemake/parser.py, lines not specified)	Iteration style refactor unrelated to the linked issues’ objectives.
Error messages updated to dynamically list run-related keywords (src/snakemake/parser.py, lines not specified)	Messaging change for run-keyword duplication/ordering not tied to docstring handling in with-blocks.

Tip

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.

✨ Finishing Touches

📝 Generate Docstrings

🧪 Generate unit tests

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch use_rule_with_docstring

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

Review comments: Directly reply to a review comment made by CodeRabbit. Example:
- I pushed a fix in commit <commit_id>, please review it.
- Open a follow-up GitHub issue for this discussion.
Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
- @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
- @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (`.coderabbit.yaml`)

You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
Please see the configuration documentation for more information.
If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

Visit our Status Page to check the current availability of CodeRabbit.
Visit our Documentation for detailed information on how to use CodeRabbit.
Join our Discord Community to get help, request features, and share feedback.
Follow us on X/Twitter for updates and announcements.

coderabbitai

Actionable comments posted: 0

🧹 Nitpick comments (1)

src/snakemake/parser.py (1)

830-830: Consider using yield from consistently for readability.

Good use of yield from here for cleaner generator delegation.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

MCP integration is disabled by default for public repositories
Jira integration is disabled by default for public repositories
Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 26fcd38 and 161152b.

📒 Files selected for processing (2)

src/snakemake/parser.py (5 hunks)
tests/test_module_nested/Snakefile (1 hunks)

🧰 Additional context used

📓 Path-based instructions (1)

**/*.py

⚙️ CodeRabbit configuration file

**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.

Files:

src/snakemake/parser.py

🧠 Learnings (1)

📚 Learning: 2024-10-06T14:09:54.370Z

Learnt from: johanneskoester
PR: snakemake/snakemake#3117
File: tests/test_wrapper/Snakefile:11-11
Timestamp: 2024-10-06T14:09:54.370Z
Learning: Changes made within test cases, such as in `tests/test_wrapper/Snakefile`, are for testing purposes and do not require updates to the project documentation.

Applied to files:

tests/test_module_nested/Snakefile

🧬 Code graph analysis (1)

src/snakemake/parser.py (2)

src/snakemake/workflow.py (6)

run (2301-2302)

script (2252-2257)

notebook (2259-2264)

wrapper (2266-2271)

template_engine (2273-2278)

cwl (2280-2285)

src/snakemake/rules.py (1)

Rule (76-1336)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (55)

GitHub Check: tests (9, ubuntu-latest, py313)
GitHub Check: tests (8, ubuntu-latest, py312)
GitHub Check: tests (10, windows-latest, py312)
GitHub Check: tests (10, ubuntu-latest, py313)
GitHub Check: tests (10, windows-latest, py313)
GitHub Check: tests (7, ubuntu-latest, py312)
GitHub Check: tests (9, ubuntu-latest, py311)
GitHub Check: tests (9, windows-latest, py312)
GitHub Check: tests (9, macos-latest, py313)
GitHub Check: tests (10, ubuntu-latest, py312)
GitHub Check: tests (8, ubuntu-latest, py313)
GitHub Check: tests (6, windows-latest, py312)
GitHub Check: tests (7, windows-latest, py312)
GitHub Check: tests (9, ubuntu-latest, py312)
GitHub Check: tests (8, windows-latest, py313)
GitHub Check: tests (9, windows-latest, py313)
GitHub Check: tests (10, ubuntu-latest, py311)
GitHub Check: tests (7, ubuntu-latest, py313)
GitHub Check: tests (8, windows-latest, py312)
GitHub Check: tests (7, windows-latest, py313)
GitHub Check: tests (6, ubuntu-latest, py313)
GitHub Check: tests (8, ubuntu-latest, py311)
GitHub Check: tests (7, ubuntu-latest, py311)
GitHub Check: tests (6, ubuntu-latest, py312)
GitHub Check: tests (6, ubuntu-latest, py311)
GitHub Check: tests (4, windows-latest, py313)
GitHub Check: tests (5, windows-latest, py313)
GitHub Check: tests (6, windows-latest, py313)
GitHub Check: tests (5, windows-latest, py312)
GitHub Check: tests (5, ubuntu-latest, py313)
GitHub Check: tests (3, windows-latest, py312)
GitHub Check: tests (4, windows-latest, py312)
GitHub Check: tests (4, macos-latest, py313)
GitHub Check: tests (5, ubuntu-latest, py311)
GitHub Check: tests (4, ubuntu-latest, py312)
GitHub Check: tests (3, windows-latest, py313)
GitHub Check: tests (4, ubuntu-latest, py313)
GitHub Check: tests (5, ubuntu-latest, py312)
GitHub Check: tests (2, windows-latest, py313)
GitHub Check: tests (3, ubuntu-latest, py313)
GitHub Check: tests (3, ubuntu-latest, py311)
GitHub Check: tests (4, ubuntu-latest, py311)
GitHub Check: tests (1, macos-latest, py312)
GitHub Check: tests (2, ubuntu-latest, py312)
GitHub Check: tests (2, ubuntu-latest, py311)
GitHub Check: tests (3, ubuntu-latest, py312)
GitHub Check: tests (2, ubuntu-latest, py313)
GitHub Check: tests (2, macos-latest, py313)
GitHub Check: tests (1, windows-latest, py312)
GitHub Check: tests (2, windows-latest, py312)
GitHub Check: tests (1, windows-latest, py313)
GitHub Check: tests (1, macos-latest, py313)
GitHub Check: tests (1, ubuntu-latest, py311)
GitHub Check: tests (1, ubuntu-latest, py313)
GitHub Check: tests (1, ubuntu-latest, py312)

🔇 Additional comments (9)

tests/test_module_nested/Snakefile (1)

9-10: LGTM! Test case properly validates the new docstring functionality.

The test correctly verifies that a docstring can now be assigned within a use rule ... with: block, addressing the requirements from issues #1678 and #3677.
src/snakemake/parser.py (8)
743-751: Good refactoring: Extracted run-related keywords into a dedicated mapping.

This extraction improves code organization by grouping related subautomata and enables reuse across multiple places in the parser. The naming is clear and follows the existing pattern.

755-755: Clean composition using dictionary unpacking.

The refactored subautomata definition maintains the same functionality while improving readability through the use of unpacked dictionaries.

803-808: Improved error messages with dynamic keyword listing.

The error message now dynamically lists all run-related keywords using /.join(rule_run_subautomata)`, making it more maintainable and accurate.

814-816: Consistent error message formatting.

The error message for the "after run" scenario is also updated to use the dynamic keyword list, maintaining consistency with the previous change.

1222-1222: Logical reordering: Check token type before other conditions.

Moving the is_name(token) check first is a sensible optimization since name tokens are the most common case in with: blocks.

1236-1245: Core functionality: Successfully enables docstring support in use rule ... with: blocks.

The implementation correctly:

Detects string tokens (docstrings)

Wraps them with @workflow.docstring() directive

Extends the _with_block for later emission

This directly addresses the requirements from issues #1678 and #3677.

1248-1249: Updated error message accurately reflects the new capabilities.

The error message now correctly mentions "docstrings" as an expected token type within use rule ... with: statements, improving user guidance.

1240-1245: Docstring decorator verified and emission format is correct

Confirmed that the Workflow.docstring decorator exists in src/snakemake/workflow.py (lines 2035–2040) and correctly assigns ruleinfo.docstring = string.strip() when applied. The parser’s emission of
self._with_block.extend(
    [
        f"@workflow.docstring({token.string})",
        "\n",
    ]
)
at src/snakemake/parser.py:1240–1245 matches the decorator’s signature and will be handled as expected by the workflow system. No changes required.

johanneskoester · 2025-09-05T06:14:27Z

Thanks, as always!

🤖 I have created a release *beep* *boop* --- ## [9.11.0](v9.10.1...v9.11.0) (2025-09-05) ### Features * add landing page to the report containing custom metadata defined with a YTE yaml template ([#3452](#3452)) ([c754d80](c754d80)) * Issue a warning when unsupported characters are used in wildcard names ([#1587](#1587)) ([fa57355](fa57355)) ### Bug Fixes * avoid checking output files in immediate-submit mode ([#3569](#3569)) ([58c42cf](58c42cf)) * clarify --keep-going flag help text to distinguish runtime vs DAG construction errors ([#3705](#3705)) ([a3a8ef4](a3a8ef4)) * enable docstring assignment in `use rule ... with:` block ([#3710](#3710)) ([2191962](2191962)) * fix missing attribute error in greedy scheduler settings when using touch, dryrun or immediate submit. ([6471004](6471004)) * only trigger script with CODE label ([#3707](#3707)) ([2d01f8c](2d01f8c)) * parser.py incorrectly parsing multiline f-strings ([#3701](#3701)) ([06ece76](06ece76)) * parsing ordinary yaml strings ([#3506](#3506)) ([ddf334e](ddf334e)) * remove temp files when using checkpoints ([#3716](#3716)) ([5ff3e20](5ff3e20)) * Restore python rules changes triggering reruns. (GH: [#3213](#3213)) ([#3485](#3485)) ([2f663be](2f663be)) * unit testing ([#3680](#3680)) ([06ba7e6](06ba7e6)) * use queue_input_wait_time when updating queue input jobs ([#3712](#3712)) ([a945a19](a945a19)) ### Documentation * output files within output directories is an error ([#2848](#2848)) ([#2913](#2913)) ([37272e5](37272e5)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

…make#3710) will fix snakemake#1678 and fix snakemake#3677 enable assign a docstring after `use rule ... with:` ### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit * **New Features** * Support docstrings inside use rule with: blocks, injecting workflow-level documentation. * **Refactor** * Unified handling of run-related keywords (run, shell, script, notebook, wrapper, template_engine, cwl) for more consistent parsing. * **Bug Fixes** * Clearer, more specific errors for duplicate or misplaced run-related keywords. * Improved error message when with: blocks lack valid entries, guiding use of keywords, comments, or docstrings. * **Tests** * Added test coverage for with: blocks containing docstrings on used rule aliases.

🤖 I have created a release *beep* *boop* --- ## [9.11.0](snakemake/snakemake@v9.10.1...v9.11.0) (2025-09-05) ### Features * add landing page to the report containing custom metadata defined with a YTE yaml template ([snakemake#3452](snakemake#3452)) ([c754d80](snakemake@c754d80)) * Issue a warning when unsupported characters are used in wildcard names ([snakemake#1587](snakemake#1587)) ([fa57355](snakemake@fa57355)) ### Bug Fixes * avoid checking output files in immediate-submit mode ([snakemake#3569](snakemake#3569)) ([58c42cf](snakemake@58c42cf)) * clarify --keep-going flag help text to distinguish runtime vs DAG construction errors ([snakemake#3705](snakemake#3705)) ([a3a8ef4](snakemake@a3a8ef4)) * enable docstring assignment in `use rule ... with:` block ([snakemake#3710](snakemake#3710)) ([2191962](snakemake@2191962)) * fix missing attribute error in greedy scheduler settings when using touch, dryrun or immediate submit. ([6471004](snakemake@6471004)) * only trigger script with CODE label ([snakemake#3707](snakemake#3707)) ([2d01f8c](snakemake@2d01f8c)) * parser.py incorrectly parsing multiline f-strings ([snakemake#3701](snakemake#3701)) ([06ece76](snakemake@06ece76)) * parsing ordinary yaml strings ([snakemake#3506](snakemake#3506)) ([ddf334e](snakemake@ddf334e)) * remove temp files when using checkpoints ([snakemake#3716](snakemake#3716)) ([5ff3e20](snakemake@5ff3e20)) * Restore python rules changes triggering reruns. (GH: [snakemake#3213](snakemake#3213)) ([snakemake#3485](snakemake#3485)) ([2f663be](snakemake@2f663be)) * unit testing ([snakemake#3680](snakemake#3680)) ([06ba7e6](snakemake@06ba7e6)) * use queue_input_wait_time when updating queue input jobs ([snakemake#3712](snakemake#3712)) ([a945a19](snakemake@a945a19)) ### Documentation * output files within output directories is an error ([snakemake#2848](snakemake#2848)) ([snakemake#2913](snakemake#2913)) ([37272e5](snakemake@37272e5)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

fix: enable docstring assignment in use rule ... with: block

161152b

Hocnonsense requested a review from johanneskoester as a code owner August 26, 2025 09:17

coderabbitai bot reviewed Aug 26, 2025

View reviewed changes

Merge branch 'main' into use_rule_with_docstring

3bee4a2

johanneskoester approved these changes Sep 5, 2025

View reviewed changes

johanneskoester merged commit 2191962 into main Sep 5, 2025
79 checks passed

johanneskoester deleted the use_rule_with_docstring branch September 5, 2025 06:14

snakemake-bot mentioned this pull request Sep 5, 2025

chore(main): release 9.11.0 #3722

Merged

Hocnonsense self-assigned this Mar 18, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: enable docstring assignment in `use rule ... with:` block#3710

fix: enable docstring assignment in `use rule ... with:` block#3710
johanneskoester merged 2 commits intomainfrom
use_rule_with_docstring

Hocnonsense commented Aug 26, 2025 •

edited by coderabbitai bot

Loading

Uh oh!

coderabbitai bot commented Aug 26, 2025 •

edited

Loading

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Assessment against linked issues

Assessment against linked issues: Out-of-scope changes

Chat

Support

CodeRabbit Commands (Invoked using PR/Issue comments)

Other keywords and placeholders

CodeRabbit Configuration File (`.coderabbit.yaml`)

Status, Documentation and Community

Uh oh!

coderabbitai bot left a comment

Uh oh!

johanneskoester commented Sep 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

Hocnonsense commented Aug 26, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

QC

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Sequence Diagram(s)

Estimated code review effort

Assessment against linked issues

Assessment against linked issues: Out-of-scope changes

Chat

Support

CodeRabbit Commands (Invoked using PR/Issue comments)

Other keywords and placeholders

CodeRabbit Configuration File (.coderabbit.yaml)

Status, Documentation and Community

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

johanneskoester commented Sep 5, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Hocnonsense commented Aug 26, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Aug 26, 2025 •

edited

Loading

CodeRabbit Configuration File (`.coderabbit.yaml`)