Skip to content

refactor: Minor Logging Improvements#3604

Merged
johanneskoester merged 9 commits intosnakemake:mainfrom
ChristofferCOASD:patch-1
Jun 12, 2025
Merged

refactor: Minor Logging Improvements#3604
johanneskoester merged 9 commits intosnakemake:mainfrom
ChristofferCOASD:patch-1

Conversation

@ChristofferCOASD
Copy link
Copy Markdown
Contributor

@ChristofferCOASD ChristofferCOASD commented May 27, 2025

I am using SnakeMake with wildcards for a very large number of files. I observed that executimg snakemake --quiet rules would still leave blank lines corresponding to the space where the rules would have been.

image

I am not fully fammiliar with the snakemake logging setup but i narrowed down the problem to the emit mehod of ColorizingTextHandler. My solution was to check for an empty formatted_message.

Our large number of files also use a corresponding number of temp files which results in thousands of print messages about which temporary files would be remove. I purpose adding LogEvent.JOB_INFO to the temporary file messages in order to allow quieting these messages.

QC

  • The PR contains a test case for the changes or the changes are already covered by an existing test case.
  • The documentation (docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).

Summary by CodeRabbit

  • Bug Fixes
    • Improved logging to prevent empty or "None" messages from appearing in the output.
    • Refined logging behavior to respect quiet mode settings, reducing unnecessary log messages during temporary file removal and job scheduling progress updates.
    • Enhanced control over shell command and progress log visibility based on specific quietness flags for a clearer user experience.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai bot commented May 27, 2025

📝 Walkthrough

Walkthrough

The changes refine logging behavior by adding conditional checks to suppress temporary output removal messages when quietness settings include rule-related suppression. The logging handler is updated to skip emitting both "None" and empty string messages. Scheduler progress printing logic is adjusted for finer control based on specific quietness flags. No public API changes are introduced.

Changes

File(s) Change Summary
src/snakemake/dag.py Added extra parameter with quietness metadata to temporary output removal log calls to respect quietness rules.
src/snakemake/logging.py Moved shell command suppression from formatter to filter; updated filter to conditionally block shell command logs; handler skips empty string messages; adjusted quietness mapping for shell commands.
src/snakemake/scheduler.py Modified progress printing condition to consider specific quietness flags instead of a simple quiet mode check.

Sequence Diagram(s)

sequenceDiagram
    participant DAG
    participant Logger

    alt Quiet mode does not suppress RULES logs
        DAG->>Logger: Log "Would remove temporary output ..." with extra quietness=RULES
        alt Not dry-run
            DAG->>Logger: Log "Removing temporary output ..." with extra quietness=RULES
        end
    else
        DAG-->>Logger: (No log output)
    end
Loading
sequenceDiagram
    participant ColorizingTextHandler
    participant OutputStream

    ColorizingTextHandler->>ColorizingTextHandler: Format log message
    alt Message is "None" or ""
        ColorizingTextHandler-->>OutputStream: (No output)
    else
        ColorizingTextHandler->>OutputStream: Write formatted message
    end
Loading
sequenceDiagram
    participant Scheduler
    participant Output

    alt Not dry-run and (not quiet or quiet excludes PROGRESS)
        Scheduler->>Output: Print progress update
    else
        Scheduler-->>Output: (No progress output)
    end
Loading
✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@ChristofferCOASD ChristofferCOASD changed the title Minor Logging Improvements refactor: Minor Logging Improvements May 27, 2025
@cademirch cademirch self-assigned this May 27, 2025
@cademirch cademirch self-requested a review May 27, 2025 09:23
Copy link
Copy Markdown
Contributor

@cademirch cademirch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for opening this! A couple comments

@cademirch
Copy link
Copy Markdown
Contributor

I took a quick look at the blank lines. It seems they are actually coming from the shellcmd log call. Shellcmd is mapped to quietness.progress, not rules:

quietness_map = {
LogEvent.JOB_INFO: Quietness.RULES,
LogEvent.GROUP_INFO: Quietness.RULES,
LogEvent.JOB_ERROR: Quietness.RULES,
LogEvent.GROUP_ERROR: Quietness.RULES,
LogEvent.PROGRESS: Quietness.PROGRESS,
LogEvent.SHELLCMD: Quietness.PROGRESS,
LogEvent.JOB_FINISHED: Quietness.PROGRESS,
LogEvent.RESOURCES_INFO: Quietness.PROGRESS,
LogEvent.RUN_INFO: Quietness.PROGRESS,
}

This should be:

quietness_map = {
            LogEvent.JOB_INFO: Quietness.RULES,
            LogEvent.GROUP_INFO: Quietness.RULES,
            LogEvent.JOB_ERROR: Quietness.RULES,
            LogEvent.GROUP_ERROR: Quietness.RULES,
            LogEvent.PROGRESS: Quietness.PROGRESS,
-           LogEvent.SHELLCMD: Quietness.PROGRESS,
+           LogEvent.SHELLCMD: Quietness.RULES,
            LogEvent.JOB_FINISHED: Quietness.PROGRESS,
            LogEvent.RESOURCES_INFO: Quietness.PROGRESS,
            LogEvent.RUN_INFO: Quietness.PROGRESS,
        }

@ChristofferCOASD
Copy link
Copy Markdown
Contributor Author

ChristofferCOASD commented May 27, 2025

I guess checking for an empty is not really a direct solution to the issue i observed then. Would it make sense to still check for the log message being empty?

EDIT:
I see there is also the printshellcmds cli option which sets the message to being empty which causes these empty lines

@ChristofferCOASD
Copy link
Copy Markdown
Contributor Author

ChristofferCOASD commented May 27, 2025

It could be possible to move the printshellcmds check to where the filter is initialized:

image

and then we could add a quietness for shellcmds if the printshellcmds (self.settings.printshellcmds) flag is not present? And then remove the part where we set the message to empty?

@ChristofferCOASD
Copy link
Copy Markdown
Contributor Author

ChristofferCOASD commented May 27, 2025

This change would in some way also partially solve #2882 since progress would be properly handled

@cademirch
Copy link
Copy Markdown
Contributor

It could be possible to move the printshellcmds check to where the filter is initialized:

image

and then we could add a quietness for shellcmds if the printshellcmds (self.settings.printshellcmds) flag is not present? And then remove the part where we set the message to empty?

Yes! I think that would be a great solution. We should just filter the message properly. But I don't think we should add a new quietness level since its already confusing as is with the issue you referenced. I think we should just do something like:

class DefaultFilter:
    def __init__(self, quiet, debug_dag, dryrun, printshellcmds) -> None:
        if quiet is None:
            quiet = set()
        self.quiet = quiet
        self.debug_dag = debug_dag
        self.dryrun = dryrun
        self.printshellcmds = printshellcmds

    def filter(self, record):
        from snakemake.settings.enums import Quietness

        event, level = get_event_level(record)
        
        ...

        
        # handle shell cmd
        if event == LogEvent.SHELLCMD and not self.printshellcmds:
            return False

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
src/snakemake/logging.py (1)

509-509: Good defensive check, but verify the formatting condition logic.

The check for both "None" and empty string provides good defensive programming. However, the space in the condition formatted_message=="" lacks proper spacing.

Apply this diff to fix the formatting:

-                if formatted_message == "None" or formatted_message=="":
+                if formatted_message == "None" or formatted_message == "":

Additionally, consider using a more comprehensive check for falsy values:

-                if formatted_message == "None" or formatted_message == "":
+                if not formatted_message or formatted_message == "None":
📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 7b45be3 and 130b411.

📒 Files selected for processing (1)
  • src/snakemake/logging.py (5 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest type annotations for functions that are defined inside of functions or methods. Do not suggest type annotation of the `s...

**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.

  • src/snakemake/logging.py
🧬 Code Graph Analysis (1)
src/snakemake/logging.py (1)
src/snakemake/settings/enums.py (1)
  • Quietness (24-29)
🔇 Additional comments (5)
src/snakemake/logging.py (5)

246-246: Excellent simplification of shell command formatting.

The removal of conditional logic from format_shellcmd aligns perfectly with the architectural improvement of moving shell command filtering to the DefaultFilter class. This ensures cleaner separation of concerns.


369-369: Good architectural improvement by adding shell command control to the filter.

Adding the printshellcmds parameter to DefaultFilter is the correct approach for controlling shell command visibility at the filtering stage rather than in formatting.

Also applies to: 375-375


399-402: Excellent implementation of shell command filtering logic.

This is the core fix that addresses the blank line issue. By filtering out shell command events when printshellcmds is False, the formatter never receives empty messages to format, eliminating the blank lines.

The logic is clear and follows the filtering pattern used elsewhere in the class.


618-622: Proper integration of shell command filtering in logger setup.

The addition of self.settings.printshellcmds to the DefaultFilter constructor completes the refactoring by ensuring the filter receives the necessary configuration. This maintains the existing API while enabling the new filtering behavior.


393-393:

Details

❓ Verification inconclusive

Verify that changing shell command quietness level aligns with user expectations.

Changing LogEvent.SHELLCMD from Quietness.PROGRESS to Quietness.RULES means shell commands will now be suppressed when using --quiet rules instead of --quiet progress. This addresses the blank line issue mentioned in the PR objectives.

However, this is a behavioral change that might affect existing user workflows.

Please verify that this change aligns with user expectations by checking existing documentation or usage patterns:


🏁 Script executed:

#!/bin/bash
# Description: Check for documentation or examples that mention shell command quietness behavior

# Search for documentation about shell command logging and quietness levels
rg -A 3 -B 3 "shell.*command.*quiet|quiet.*shell.*command" --type md --type rst --type txt

# Search for test cases that might verify shell command quietness behavior
rg -A 5 -B 5 "SHELLCMD.*quiet|quiet.*SHELLCMD" --type py

# Search for any mentions of the old PROGRESS mapping for shell commands
rg -A 3 -B 3 "SHELLCMD.*PROGRESS|PROGRESS.*SHELLCMD" --type py

Length of output: 216


🏁 Script executed:

#!/bin/bash
# Inspect Quietness enum definition
rg -n "class Quietness" -A 10 -B 0 src/snakemake/logging.py

# Inspect CLI parser references to --quiet flag
rg -n "\-\-quiet" -A 5 -B 5 src/snakemake

# Inspect use of PROGRESS and RULES in logging.py
rg -n "PROGRESS" -A 5 -B 5 src/snakemake/logging.py
rg -n "RULES" -A 5 -B 5 src/snakemake/logging.py

Length of output: 8120


Confirm shell-command quietness mapping
We changed LogEvent.SHELLCMD from Quietness.PROGRESS to Quietness.RULES (src/snakemake/logging.py:393), so shell commands will now be suppressed by --quiet rules instead of --quiet progress.

• No user documentation or examples mention shell-command quietness behavior.
• No existing unit tests cover how --quiet progress vs. --quiet rules affects shell-command output.

Please verify that this new mapping matches intended workflows, update any user-facing docs or CLI help accordingly, and add a unit test to cover shell-command suppression under both --quiet progress and --quiet rules.

Copy link
Copy Markdown
Contributor

@cademirch cademirch left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for making these changes. Let's make sure tests pass but I think this is okay to merge.

ChristofferCOASD and others added 6 commits June 1, 2025 14:54
I added JOB_INFO LogEvent to the info aboout temp files. Previously there was no way to use the quiet flag to filter these messages. I would argue the the removal of temp files are related to the JOB_INFO logs.
Also check if formatted_message is empty otherwise logger would print empt lines for quieted rules
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
src/snakemake/logging.py (1)

509-509: Consider using 'in' operator for better readability

The current condition works correctly but could be more readable using the in operator as suggested by the static analysis tool.

-                if formatted_message == "None" or formatted_message=="":
+                if formatted_message in ("None", ""):

This change improves readability and follows Python best practices for multiple equality comparisons.

🧰 Tools
🪛 Pylint (3.3.7)

[refactor] 509-509: Consider merging these comparisons with 'in' by using 'formatted_message in ('None', '')'. Use a set instead if elements are hashable.

(R1714)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 130b411 and 077f989.

📒 Files selected for processing (3)
  • src/snakemake/dag.py (2 hunks)
  • src/snakemake/logging.py (5 hunks)
  • src/snakemake/scheduler.py (2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest type annotations for functions that are defined inside of functions or methods. Do not suggest type annotation of the `s...

**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.

  • src/snakemake/dag.py
  • src/snakemake/logging.py
  • src/snakemake/scheduler.py
🧬 Code Graph Analysis (2)
src/snakemake/logging.py (1)
src/snakemake/settings/enums.py (1)
  • Quietness (24-29)
src/snakemake/scheduler.py (1)
src/snakemake/settings/enums.py (1)
  • Quietness (24-29)
🪛 Pylint (3.3.7)
src/snakemake/dag.py

[convention] 33-33: Imports from package snakemake are not grouped

(C0412)


[convention] 964-964: Line too long (121/100)

(C0301)


[convention] 967-967: Line too long (121/100)

(C0301)


[warning] 965-965: Use lazy % formatting in logging functions

(W1203)


[warning] 968-968: Use lazy % formatting in logging functions

(W1203)

src/snakemake/logging.py

[refactor] 509-509: Consider merging these comparisons with 'in' by using 'formatted_message in ('None', '')'. Use a set instead if elements are hashable.

(R1714)

src/snakemake/scheduler.py

[convention] 73-73: Line too long (106/100)

(C0301)

🔇 Additional comments (7)
src/snakemake/dag.py (1)

33-33: LGTM!

The import of Quietness enum is necessary for the conditional logging logic implemented in the handle_temp method.

🧰 Tools
🪛 Pylint (3.3.7)

[convention] 33-33: Imports from package snakemake are not grouped

(C0412)

src/snakemake/scheduler.py (1)

73-73: LGTM: Improved granular quietness control for progress printing

This change enhances the quietness handling by allowing progress printing when the specific Quietness.PROGRESS flag is not set, even if other quietness flags are present. This provides more fine-grained control over what gets suppressed and aligns well with the broader logging improvements in this PR.

🧰 Tools
🪛 Pylint (3.3.7)

[convention] 73-73: Line too long (106/100)

(C0301)

src/snakemake/logging.py (5)

246-246: LGTM: Simplified shell command formatting

The shell command formatter now unconditionally returns the message, which is correct since the filtering logic has been moved to the DefaultFilter class. This separation of concerns makes the code cleaner and more maintainable.


369-369: LGTM: Enhanced filter constructor for shell command control

The addition of the printshellcmds parameter to the DefaultFilter constructor enables proper filtering of shell command messages based on the CLI flag. This is a much cleaner approach than the previous method of setting messages to empty strings.

Also applies to: 375-375


393-393: LGTM: Corrected quietness mapping for shell commands

Moving LogEvent.SHELLCMD from Quietness.PROGRESS to Quietness.RULES aligns shell command log events with the rules quietness level. This addresses the root cause of blank lines appearing when running snakemake --quiet rules, as mentioned in the PR discussion.


399-401: LGTM: Proper shell command filtering implementation

This filtering logic correctly implements the approach discussed in the PR comments. Shell command messages are now filtered out when printshellcmds is False, preventing empty messages from reaching the emit method and causing blank lines.


618-621: LGTM: Consistent parameter passing to filter

The LoggerManager correctly passes the printshellcmds setting to the DefaultFilter constructor, enabling the new filtering logic. This completes the implementation of the shell command filtering approach discussed in the PR.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
src/snakemake/dag.py (1)

964-975: Address code quality issues raised in previous reviews.

The conditional logging implementation correctly addresses the PR objectives, but multiple code quality issues remain unresolved from previous reviews, including line length violations, code duplication, and improper use of f-strings in logging functions.

The suggested refactor from previous reviews should be applied:

  • Extract the duplicated condition to a variable
  • Use lazy formatting (% formatting) in logger.info calls
  • Combine the logging statements to eliminate duplication
  • Keep lines under 100 characters

This maintains the correct logic while significantly improving code quality and readability.

🧰 Tools
🪛 Pylint (3.3.7)

[warning] 968-968: Use lazy % formatting in logging functions

(W1203)


[warning] 974-974: Use lazy % formatting in logging functions

(W1203)

🧹 Nitpick comments (2)
src/snakemake/dag.py (1)

33-33: Import grouping issue flagged by static analysis.

The import is functionally correct but static analysis suggests it should be grouped with other snakemake imports for better organization.

🧰 Tools
🪛 Pylint (3.3.7)

[convention] 33-33: Imports from package snakemake are not grouped

(C0412)

src/snakemake/logging.py (1)

509-509: Improve readability with 'in' operator.

The addition of the empty string check correctly prevents blank lines from being emitted. However, consider improving readability by using the 'in' operator as suggested by static analysis.

Apply this diff to improve readability:

-                if formatted_message == "None" or formatted_message == "":
+                if formatted_message in ("None", ""):
🧰 Tools
🪛 Pylint (3.3.7)

[refactor] 509-509: Consider merging these comparisons with 'in' by using 'formatted_message in ('None', '')'. Use a set instead if elements are hashable.

(R1714)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 077f989 and c9327dc.

📒 Files selected for processing (3)
  • src/snakemake/dag.py (2 hunks)
  • src/snakemake/logging.py (5 hunks)
  • src/snakemake/scheduler.py (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/snakemake/scheduler.py
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest type annotations for functions that are defined inside of functions or methods. Do not suggest type annotation of the `s...

**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.

  • src/snakemake/dag.py
  • src/snakemake/logging.py
🧬 Code Graph Analysis (1)
src/snakemake/logging.py (1)
src/snakemake/settings/enums.py (1)
  • Quietness (24-29)
🪛 Pylint (3.3.7)
src/snakemake/dag.py

[convention] 33-33: Imports from package snakemake are not grouped

(C0412)


[warning] 968-968: Use lazy % formatting in logging functions

(W1203)


[warning] 974-974: Use lazy % formatting in logging functions

(W1203)

src/snakemake/logging.py

[refactor] 509-509: Consider merging these comparisons with 'in' by using 'formatted_message in ('None', '')'. Use a set instead if elements are hashable.

(R1714)

🔇 Additional comments (5)
src/snakemake/logging.py (5)

244-246: LGTM! Proper separation of concerns.

The simplification of format_shellcmd to unconditionally return the message is correct. Moving the conditional logic to the filtering stage improves the architecture by keeping formatters focused on formatting and filters focused on filtering decisions.


369-375: LGTM! Necessary infrastructure for shell command filtering.

The addition of the printshellcmds parameter to the DefaultFilter constructor and its storage as an instance attribute properly supports the new filtering logic for shell commands.


393-393: LGTM! Improved quietness level alignment.

Mapping LogEvent.SHELLCMD to Quietness.RULES instead of Quietness.PROGRESS properly aligns shell command logging with rule-related quietness control. This change ensures that --quiet rules will suppress shell commands as expected.

Note: This may change behavior for users who previously used --quiet progress to suppress shell commands.


399-401: LGTM! Correct implementation of shell command filtering.

The new filtering logic properly filters out shell command log events when printshellcmds is False. This addresses the core issue of blank lines appearing in quiet mode by preventing shell command messages from reaching the formatter when they shouldn't be displayed.


618-621: LGTM! Proper wiring of printshellcmds setting.

The addition of self.settings.printshellcmds to the DefaultFilter constructor call correctly connects the settings to the filtering logic, enabling the new shell command filtering behavior.

Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
src/snakemake/logging.py (2)

387-390: Consider simplifying the nested if statement.

The logic is correct, but the nested if statements can be simplified for better readability.

-        if hasattr(record, "quietness"):
-            if record.quietness in self.quiet:
-                return False
+        if hasattr(record, "quietness") and record.quietness in self.quiet:
+            return False
🧰 Tools
🪛 Ruff (0.11.9)

387-388: Use a single if statement instead of nested if statements

Combine if statements using and

(SIM102)


513-513: Consider using 'in' operator for cleaner comparison.

The logic is correct for preventing empty message emission, but the comparison can be simplified.

-                if formatted_message == "None" or formatted_message == "":
+                if formatted_message in ("None", ""):
🧰 Tools
🪛 Pylint (3.3.7)

[refactor] 513-513: Consider merging these comparisons with 'in' by using 'formatted_message in ('None', '')'. Use a set instead if elements are hashable.

(R1714)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c9327dc and d62dd50.

📒 Files selected for processing (2)
  • src/snakemake/dag.py (2 hunks)
  • src/snakemake/logging.py (5 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/snakemake/dag.py
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest type annotations for functions that are defined inside of functions or methods. Do not suggest type annotation of the `s...

**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.

  • src/snakemake/logging.py
🧬 Code Graph Analysis (1)
src/snakemake/logging.py (1)
src/snakemake/settings/enums.py (1)
  • Quietness (24-29)
🪛 Ruff (0.11.9)
src/snakemake/logging.py

387-388: Use a single if statement instead of nested if statements

Combine if statements using and

(SIM102)

🪛 Pylint (3.3.7)
src/snakemake/logging.py

[refactor] 513-513: Consider merging these comparisons with 'in' by using 'formatted_message in ('None', '')'. Use a set instead if elements are hashable.

(R1714)

🔇 Additional comments (5)
src/snakemake/logging.py (5)

246-246: LGTM! Shell command formatting now unconditional.

This change aligns with the PR objective to move shell command filtering from the formatting stage to the filtering stage. The formatter now always returns the message, allowing the filter to handle visibility logic.


369-375: LGTM! Filter now handles printshellcmds parameter.

Good refactor moving the printshellcmds logic from the formatter to the filter. This allows proper filtering of shell command messages instead of setting them to empty strings, which was causing the blank lines issue.


397-397: LGTM! Correct quietness level mapping for shell commands.

This change fixes the root cause identified in the PR comments. Mapping SHELLCMD to Quietness.RULES instead of Quietness.PROGRESS ensures shell command messages are properly filtered when using --quiet rules.


403-406: LGTM! Proper shell command filtering based on printshellcmds flag.

This is the key fix that prevents blank lines. Instead of setting shell command messages to empty strings in the formatter, they are now properly filtered out when printshellcmds is False.


622-626: LGTM! LoggerManager now passes printshellcmds to filter.

Correct update to pass the printshellcmds parameter to the DefaultFilter constructor, completing the refactor that moves shell command filtering logic from formatter to filter.

@johanneskoester johanneskoester enabled auto-merge (squash) June 12, 2025 14:39
@johanneskoester johanneskoester merged commit 6dee2b5 into snakemake:main Jun 12, 2025
48 checks passed
kjohnsen pushed a commit to kjohnsen/snakemake that referenced this pull request Dec 15, 2025
I am using SnakeMake with wildcards for a very large number of files. I
observed that executimg `snakemake --quiet rules` would still leave
blank lines corresponding to the space where the rules would have been.


![image](https://github.com/user-attachments/assets/528e72d7-f6cd-4096-82df-5a10b3d5068f)

I am not fully fammiliar with the snakemake logging setup but i narrowed
down the problem to the `emit` mehod of `ColorizingTextHandler`. My
solution was to check for an empty `formatted_message`.


Our large number of files also use a corresponding number of temp files
which results in thousands of print messages about which temporary files
would be remove. I purpose adding `LogEvent.JOB_INFO` to the temporary
file messages in order to allow quieting these messages.


### QC
<!-- Make sure that you can tick the boxes below. -->

* [x] The PR contains a test case for the changes or the changes are
already covered by an existing test case.
* [x] The documentation (`docs/`) is updated to reflect the changes or
this is not necessary (e.g. if the change does neither modify the
language nor the behavior or functionalities of Snakemake).


<!-- This is an auto-generated comment: release notes by coderabbit.ai
-->
## Summary by CodeRabbit

- **Bug Fixes**
- Improved logging to prevent empty or "None" messages from appearing in
the output.
- Refined logging behavior to respect quiet mode settings, reducing
unnecessary log messages during temporary file removal and job
scheduling progress updates.
- Enhanced control over shell command and progress log visibility based
on specific quietness flags for a clearer user experience.
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

---------

Co-authored-by: Christoffer <christofferrs@protonmail.com>
Co-authored-by: Cade Mirchandani <cmirchan@ucsc.edu>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants