fix: parsing ordinary yaml strings#3506
Conversation
📝 WalkthroughWalkthroughAdds Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant T as Test / Caller
participant E as eval_resource_expression
participant G as generic_callable
participant R as is_ordinary_string
T->>E: evaluate resource value (val)
alt value is callable expression
E->>G: delegate evaluation
G->>R: is_ordinary_string(val)?
alt ordinary string (True)
R-->>G: True
G-->>E: return original val (passthrough)
else not ordinary (False)
R-->>G: False
G-->>G: attempt to evaluate expression
alt evaluation raises NameError/SyntaxError
G-->>E: return original val (error handled)
else success
G-->>E: return evaluated result
end
end
else value not callable
E-->>T: return value
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes Possibly related PRs
✨ Finishing Touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
SupportNeed help? Create a ticket on our support page for assistance with any issues or questions. CodeRabbit Commands (Invoked using PR/Issue comments)Type Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/snakemake/resources.py (1)
686-688: Consider simplifying the return statement.The condition can be simplified by directly returning the result of
is_ordinary_string().- if is_ordinary_string(value): - return True + return is_ordinary_string(value)Note: This is a minor style improvement. Since
is_ordinary_string()already returns a boolean, there's no need for the explicitifcondition.🧰 Tools
🪛 Ruff (0.8.2)
686-689: Return the condition
bool(is_ordinary_string(value))directlyReplace with
return bool(is_ordinary_string(value))(SIM103)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/snakemake/resources.py(3 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest type annotations for functions that are defined inside of functions or methods. Do not suggest type annotation of the `s...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
src/snakemake/resources.py
🪛 Ruff (0.8.2)
src/snakemake/resources.py
686-689: Return the condition bool(is_ordinary_string(value)) directly
Replace with return bool(is_ordinary_string(value))
(SIM103)
⏰ Context from checks skipped due to timeout of 90000ms (34)
- GitHub Check: tests (10, windows-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py311)
- GitHub Check: tests (9, windows-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py311)
- GitHub Check: tests (8, windows-latest, py312)
- GitHub Check: tests (8, ubuntu-latest, py312)
- GitHub Check: tests (8, ubuntu-latest, py311)
- GitHub Check: tests (7, macos-latest, py312)
- GitHub Check: tests (7, windows-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py311)
- GitHub Check: tests (6, macos-latest, py312)
- GitHub Check: tests (6, windows-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py311)
- GitHub Check: tests (5, macos-latest, py312)
- GitHub Check: tests (5, windows-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py311)
- GitHub Check: tests (4, windows-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py311)
- GitHub Check: tests (3, windows-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py311)
- GitHub Check: tests (2, windows-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py311)
- GitHub Check: tests (1, macos-latest, py312)
- GitHub Check: tests (1, windows-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py311)
🔇 Additional comments (2)
src/snakemake/resources.py (2)
662-668: Good implementation of the newis_ordinary_stringhelper.The function is well-defined with a clear docstring that explains its purpose. It correctly identifies strings that should be treated as ordinary YAML strings rather than Python expressions, which addresses the PR objective of fixing parsing for resource strings.
544-545: Appropriate usage of the new helper ingeneric_callable.This change correctly prevents ordinary strings from being unnecessarily evaluated as Python expressions, which would lead to errors. Now strings like
"gpu:1"can be passed through directly, addressing the issue described in the PR objectives.
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/snakemake/resources.py(2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest type annotations for functions that are defined inside of functions or methods. Do not suggest type annotation of the `s...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
src/snakemake/resources.py
⏰ Context from checks skipped due to timeout of 90000ms (35)
- GitHub Check: tests (10, windows-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py311)
- GitHub Check: tests (9, windows-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py311)
- GitHub Check: tests (8, windows-latest, py312)
- GitHub Check: tests (8, ubuntu-latest, py312)
- GitHub Check: tests (8, ubuntu-latest, py311)
- GitHub Check: tests (7, windows-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py311)
- GitHub Check: tests (6, windows-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py311)
- GitHub Check: tests (5, macos-latest, py312)
- GitHub Check: tests (5, windows-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py311)
- GitHub Check: tests (4, windows-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py311)
- GitHub Check: tests (3, macos-latest, py312)
- GitHub Check: tests (3, windows-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py311)
- GitHub Check: tests (2, macos-latest, py312)
- GitHub Check: tests (2, windows-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py311)
- GitHub Check: tests (1, macos-latest, py312)
- GitHub Check: tests (1, windows-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py311)
- GitHub Check: apidocs
🔇 Additional comments (1)
src/snakemake/resources.py (1)
660-666: Looks good!This new function provides a clear way to identify ordinary string values (not function calls) that should be treated as literal strings rather than Python expressions.
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/snakemake/resources.py (1)
684-687: Consider the logical implications of the 'or' operator.Using
orinis_ordinary_string(value) or eval(value)means thateval(value)won't be executed ifis_ordinary_string(value)returnsTrue. This is good for performance, but ifeval(value)evaluates to a falsy value (0, empty string, etc.) andis_ordinary_string(value)isFalse, the function will return that falsy value instead ofTrue.Consider explicitly returning
Trueif needed:- return is_ordinary_string(value) or eval(value) + if is_ordinary_string(value): + return True + return bool(eval(value))
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/snakemake/resources.py(2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest type annotations for functions that are defined inside of functions or methods. Do not suggest type annotation of the `s...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
src/snakemake/resources.py
⏰ Context from checks skipped due to timeout of 90000ms (25)
- GitHub Check: tests (10, windows-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py311)
- GitHub Check: tests (7, windows-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py311)
- GitHub Check: tests (6, windows-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py311)
- GitHub Check: tests (5, windows-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py311)
- GitHub Check: tests (4, windows-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py311)
- GitHub Check: tests (3, windows-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py311)
- GitHub Check: tests (2, windows-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py311)
- GitHub Check: tests (1, windows-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py311)
- GitHub Check: apidocs
🔇 Additional comments (2)
src/snakemake/resources.py (2)
660-667: Function logic looks correct for checking ordinary strings.The new
is_ordinary_stringfunction appropriately checks if a string value should be treated literally rather than evaluated as a Python expression. The regex pattern catches the common case of function-like strings.
684-687: Handle potential eval() exceptions.The modification now attempts to evaluate string values using
eval()without proper error handling. This could cause unexpected exceptions if the string is not a valid Python expression, especially since this function is called within an exception handler ineval_resource_expression()(line 542).Consider adding try/except around the eval call:
- return is_ordinary_string(value) or eval(value) + try: + return is_ordinary_string(value) or eval(value) + except (NameError, SyntaxError): + return False
|
@johanneskoester is my attempt futile - or are there any ideas ;-) |
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/snakemake/resources.py (1)
686-691: Simplify the conditional return statement.The current implementation can be simplified to directly return the boolean result.
- if is_ordinary_string(value): - return True - else: - return False + return is_ordinary_string(value)🧰 Tools
🪛 Ruff (0.8.2)
688-691: Return the condition
bool(is_ordinary_string(value))directlyReplace with
return bool(is_ordinary_string(value))(SIM103)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/snakemake/resources.py(2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest type annotations for functions that are defined inside of functions or methods. Do not suggest type annotation of the `s...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
src/snakemake/resources.py
🪛 Ruff (0.8.2)
src/snakemake/resources.py
688-691: Return the condition bool(is_ordinary_string(value)) directly
Replace with return bool(is_ordinary_string(value))
(SIM103)
⏰ Context from checks skipped due to timeout of 90000ms (36)
- GitHub Check: tests (10, windows-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py311)
- GitHub Check: tests (9, windows-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py311)
- GitHub Check: tests (8, windows-latest, py312)
- GitHub Check: tests (8, ubuntu-latest, py312)
- GitHub Check: tests (8, ubuntu-latest, py311)
- GitHub Check: tests (7, windows-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py311)
- GitHub Check: tests (6, windows-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py311)
- GitHub Check: tests (5, macos-latest, py312)
- GitHub Check: tests (5, windows-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py311)
- GitHub Check: tests (4, macos-latest, py312)
- GitHub Check: tests (4, windows-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py311)
- GitHub Check: tests (3, macos-latest, py312)
- GitHub Check: tests (3, windows-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py311)
- GitHub Check: tests (2, macos-latest, py312)
- GitHub Check: tests (2, windows-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py311)
- GitHub Check: tests (1, macos-latest, py312)
- GitHub Check: tests (1, windows-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py312)
- GitHub Check: apidocs
- GitHub Check: tests (1, ubuntu-latest, py311)
🔇 Additional comments (1)
src/snakemake/resources.py (1)
660-668: Implementation looks good, but regex pattern could be more explicit.The new function
is_ordinary_stringcorrectly identifies strings that shouldn't be evaluated as Python expressions. The regex pattern matches function calls, dictionary literals, and lambda expressions to exclude them from being considered "ordinary".Consider adding a comment explaining the specific patterns being matched in the regex for better maintainability.
|
oops! The last attempt passed? Then I am putting this PR to review. @johanneskoester |
johanneskoester
left a comment
There was a problem hiding this comment.
I like this. Two things to get this merged.
- we need very comprehensive test cases that clearly show that this does not cause regressions and always works as expected.
- The regex needs a comment that explains the rationale behind its elements.
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/snakemake/resources.py (1)
710-715: Simplify the conditional return statementThe conditional return can be simplified for better readability.
- if is_ordinary_string(value): - return True - else: - return False + return is_ordinary_string(value)🧰 Tools
🪛 Ruff (0.8.2)
712-715: Return the condition
bool(is_ordinary_string(value))directlyReplace with
return bool(is_ordinary_string(value))(SIM103)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/snakemake/resources.py(2 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest type annotations for functions that are defined inside of functions or methods. Do not suggest type annotation of the `s...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
src/snakemake/resources.py
🪛 Ruff (0.8.2)
src/snakemake/resources.py
712-715: Return the condition bool(is_ordinary_string(value)) directly
Replace with return bool(is_ordinary_string(value))
(SIM103)
⏰ Context from checks skipped due to timeout of 90000ms (33)
- GitHub Check: tests (10, windows-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py311)
- GitHub Check: tests (9, windows-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py311)
- GitHub Check: tests (8, windows-latest, py312)
- GitHub Check: tests (8, ubuntu-latest, py312)
- GitHub Check: tests (8, ubuntu-latest, py311)
- GitHub Check: tests (7, windows-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py311)
- GitHub Check: tests (6, windows-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py311)
- GitHub Check: tests (5, windows-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py311)
- GitHub Check: tests (4, windows-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py311)
- GitHub Check: tests (3, windows-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py311)
- GitHub Check: tests (2, macos-latest, py312)
- GitHub Check: tests (2, windows-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py311)
- GitHub Check: tests (1, macos-latest, py312)
- GitHub Check: tests (1, windows-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py312)
- GitHub Check: apidocs
- GitHub Check: tests (1, ubuntu-latest, py311)
🔇 Additional comments (2)
src/snakemake/resources.py (2)
660-692: Good addition with clear documentationThe
is_ordinary_stringfunction is well-documented and addresses the need to identify strings that should be treated as literal values rather than Python expressions. This helps solve the YAML string parsing issue mentioned in the PR objectives.
542-558: Handle potential eval() exceptions properlyThe previous review highlighted a concern about exception handling when evaluating expressions that hasn't been addressed in this PR. Since you're now identifying ordinary strings that shouldn't be evaluated, this is a good opportunity to improve the error handling here.
Consider modifying the exception handling in the
eval_resource_expressionfunction:try: value = eval( val, args, ) # Triggers for string arguments like n1-standard-4 except NameError: return val except Exception as e: - if is_humanfriendly_resource(val): + try: + if is_humanfriendly_resource(val): + return val + except Exception: + pass if not is_file_not_found_error(e, kwargs["input"]): # Missing input files are handled by the caller raise WorkflowError(Let me check if there are other instances of
eval()that might need similar handling:#!/bin/bash # Check for other eval() calls in the codebase that might need similar handling rg "eval\(" src/snakemake/🧰 Tools
🪛 Ruff (0.8.2)
546-557: Within an
exceptclause, raise exceptions withraise ... from errorraise ... from Noneto distinguish them from errors in exception handling(B904)
|
@johanneskoester well, ad 2, I added a bit in a docstring, see commit a0587db However, writing a full test which will guarantee this will have no side effects? I was hoping the existing test-suite, will cover that ;-). Nevertheless, I prepared 90cb705 , which introduces a new test prior to the Any ideas for additional tests? |
|
@johanneskoester updated branch. Any ideas for additional tests? |
|
What about additionally catching SyntaxError in line 539 of resources.py? |
|
THe tests are fine here. |
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
src/snakemake/resources.py (1)
710-715: Simplify boolean return statement.The conditional return can be simplified to improve readability.
- if is_ordinary_string(value): - return True - else: - return False + return is_ordinary_string(value)🧰 Tools
🪛 Ruff (0.8.2)
712-715: Return the condition
bool(is_ordinary_string(value))directlyReplace with
return bool(is_ordinary_string(value))(SIM103)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
src/snakemake/resources.py(3 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
`**/*.py`: Do not try to improve formatting. Do not suggest type annotations for functions that are defined inside of functions or methods. Do not suggest type annotation of the `s...
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
src/snakemake/resources.py
🪛 Ruff (0.8.2)
src/snakemake/resources.py
712-715: Return the condition bool(is_ordinary_string(value)) directly
Replace with return bool(is_ordinary_string(value))
(SIM103)
⏰ Context from checks skipped due to timeout of 90000ms (34)
- GitHub Check: tests (10, windows-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py312)
- GitHub Check: tests (10, ubuntu-latest, py311)
- GitHub Check: tests (9, windows-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py312)
- GitHub Check: tests (9, ubuntu-latest, py311)
- GitHub Check: tests (8, windows-latest, py312)
- GitHub Check: tests (8, ubuntu-latest, py312)
- GitHub Check: tests (8, ubuntu-latest, py311)
- GitHub Check: tests (7, windows-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py312)
- GitHub Check: tests (7, ubuntu-latest, py311)
- GitHub Check: tests (6, windows-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py312)
- GitHub Check: tests (6, ubuntu-latest, py311)
- GitHub Check: tests (5, windows-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py312)
- GitHub Check: tests (5, ubuntu-latest, py311)
- GitHub Check: tests (4, macos-latest, py312)
- GitHub Check: tests (4, windows-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py312)
- GitHub Check: tests (4, ubuntu-latest, py311)
- GitHub Check: tests (3, macos-latest, py312)
- GitHub Check: tests (3, windows-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py312)
- GitHub Check: tests (3, ubuntu-latest, py311)
- GitHub Check: tests (2, macos-latest, py312)
- GitHub Check: tests (2, windows-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py312)
- GitHub Check: tests (2, ubuntu-latest, py311)
- GitHub Check: tests (1, macos-latest, py312)
- GitHub Check: tests (1, windows-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py312)
- GitHub Check: tests (1, ubuntu-latest, py311)
🔇 Additional comments (2)
src/snakemake/resources.py (2)
539-539: Proper exception handling syntax implemented.The code now correctly catches both
NameErrorandSyntaxErrorexceptions using the proper tuple syntax. This fixes the syntax error from the previous version.
660-692: Well-implemented new function for string validation.The new
is_ordinary_stringfunction is well-documented and provides a clear way to identify "ordinary" strings that shouldn't be evaluated as Python expressions. The regex pattern correctly identifies function calls, dictionary literals, and lambda expressions.
I'm with the AI-powered rabbit on this one! Code of the form: should be simplified. But mainly I wanted to say I'm glad to see this fix - the current syntax for defining resources is tricky and annoying. |
|
@tbooth yes, generally I agree. Just thought being more explicit would help readability. Yet, the introduced function is so small, that my latest commit just introduced a thick comment. I hope, this will do. @johanneskoester any further ideas regarding additional checks? Or are you willing to merge? |
|
@cmeesters I have added one more test case. I think we also would have to recognize such expressions as non-strings. To speed things up, I have separated the SyntaxError approach into a separate PR, because that should be always fine: #3532. |
Do we need to check expressions? Any valid expression should eval before getting checked by In any case, I've added a regex to try and catch expressions. |
|
@johanneskoester & @cademirch thank you for your input, guys. I'm afraid, I am currently on a retreat and taking a few days off after Easter. |
|
test failure is unrelated to changes here. seems like some sort of ephemeral connection issue in windows runners |
There was a problem hiding this comment.
Actionable comments posted: 1
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
💡 Knowledge Base configuration:
- MCP integration is disabled by default for public repositories
- Jira integration is disabled by default for public repositories
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (2)
tests/test_resource_quoting/test-profile-default/gurobi.logis excluded by!**/*.logtests/test_resource_quoting/test-profile/gurobi.logis excluded by!**/*.log
📒 Files selected for processing (2)
src/snakemake/resources.py(2 hunks)tests/tests.py(2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- src/snakemake/resources.py
🧰 Additional context used
📓 Path-based instructions (1)
**/*.py
⚙️ CodeRabbit configuration file
**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of theselfargument of methods.
Do not suggest type annotation of theclsargument of classmethods.
Do not suggest return type annotation if a function or method does not contain areturnstatement.
Files:
tests/tests.py
🧠 Learnings (1)
📚 Learning: 2024-10-06T14:09:54.370Z
Learnt from: johanneskoester
PR: snakemake/snakemake#3117
File: tests/test_wrapper/Snakefile:11-11
Timestamp: 2024-10-06T14:09:54.370Z
Learning: Changes made within test cases, such as in `tests/test_wrapper/Snakefile`, are for testing purposes and do not require updates to the project documentation.
Applied to files:
tests/tests.py
🧬 Code graph analysis (1)
tests/tests.py (1)
src/snakemake/resources.py (1)
is_ordinary_string(664-697)
🪛 Ruff (0.12.2)
tests/tests.py
936-936: Use of assert detected
(S101)
937-937: Use of assert detected
(S101)
938-938: Use of assert detected
(S101)
941-941: Use of assert detected
(S101)
942-942: Use of assert detected
(S101)
943-943: Use of assert detected
(S101)
944-944: Use of assert detected
(S101)
947-947: Use of assert detected
(S101)
948-948: Use of assert detected
(S101)
949-949: Use of assert detected
(S101)
🔇 Additional comments (1)
tests/tests.py (1)
16-16: LGTM: import of is_ordinary_stringImport is used by the new unit test and aligns with the module where the helper is defined.
| def test_resource_is_ordinary_string(): | ||
| # Ordinary strings | ||
| assert is_ordinary_string("hello") is True | ||
| assert is_ordinary_string("123") is True | ||
| assert is_ordinary_string("simple_string") is True | ||
|
|
||
| # Strings that are not ordinary | ||
| assert is_ordinary_string("func_name(arg1, arg2)") is False # Function call | ||
| assert is_ordinary_string("{'key': 'value'}") is False # Dictionary literal | ||
| assert is_ordinary_string("lambda x: x + 1") is False # Lambda expression | ||
| assert is_ordinary_string("2 * input.size") is False # calculation | ||
|
|
||
| # Non-string inputs | ||
| assert is_ordinary_string(123) is False # Integer | ||
| assert is_ordinary_string(["list", "of", "strings"]) is False # List | ||
| assert is_ordinary_string(None) is False # NoneType | ||
|
|
||
|
|
There was a problem hiding this comment.
💡 Verification agent
🧩 Analysis chain
Cover dashed/slashed resource strings; current regex likely misclassifies them
The regex in is_ordinary_string includes operators like '-' and '/', which will erroneously flag common resource values as non-ordinary (e.g., "--nice=150 --gres=gpu:1", "nvidia-tesla-1000", "CUDA/11.8"). That undermines the PR goal. Please expand the test to capture these cases; they should be True. If they fail, we need to relax the operator detection in is_ordinary_string (e.g., drop '-' and '/' from the operator class or constrain matches to arithmetic contexts and strip whitespace).
Apply this minimal test expansion:
def test_resource_is_ordinary_string():
# Ordinary strings
assert is_ordinary_string("hello") is True
assert is_ordinary_string("123") is True
assert is_ordinary_string("simple_string") is True
+ assert is_ordinary_string("gpu:1") is True
+ assert is_ordinary_string("--nice=150 --gres=gpu:1") is True
+ assert is_ordinary_string("nvidia-tesla-1000") is True
+ assert is_ordinary_string("CUDA/11.8") is True
@@
# Strings that are not ordinary
assert is_ordinary_string("func_name(arg1, arg2)") is False # Function call
assert is_ordinary_string("{'key': 'value'}") is False # Dictionary literal
assert is_ordinary_string("lambda x: x + 1") is False # Lambda expression
assert is_ordinary_string("2 * input.size") is False # calculationFollow-up (in src/snakemake/resources.py, not in this file): consider either stripping the value before matching and/or narrowing the arithmetic pattern so that minus and slash inside plain text (paths, flags, model names) do not trigger. I can propose a concrete regex tweak once we agree on desired edge cases.
🏁 Script executed:
#!/bin/bash
# Locate the implementation of is_ordinary_string to inspect its regex
rg -nP --type=py 'def is_ordinary_string' -C5 src/snakemake/resources.pyLength of output: 464
🏁 Script executed:
#!/bin/bash
# Show the implementation of is_ordinary_string for regex inspection
sed -n '660,740p' src/snakemake/resources.pyLength of output: 2067
Add tests for dashed/slashed resource strings and relax operator regex
- Expand
test_resource_is_ordinary_stringto assertTruefor"gpu:1","--nice=150 --gres=gpu:1","nvidia-tesla-1000", and"CUDA/11.8". - In
is_ordinary_string, remove-and/from the[\+\-\*/\%]class (or narrow it to only match arithmetic with surrounding whitespace) so common resource identifiers aren’t misclassified.
🧰 Tools
🪛 Ruff (0.12.2)
936-936: Use of assert detected
(S101)
937-937: Use of assert detected
(S101)
938-938: Use of assert detected
(S101)
941-941: Use of assert detected
(S101)
942-942: Use of assert detected
(S101)
943-943: Use of assert detected
(S101)
944-944: Use of assert detected
(S101)
947-947: Use of assert detected
(S101)
948-948: Use of assert detected
(S101)
949-949: Use of assert detected
(S101)
🤖 Prompt for AI Agents
In tests/tests.py around lines 934-951, expand the
test_resource_is_ordinary_string block to include positive assertions for
"gpu:1", "--nice=150 --gres=gpu:1", "nvidia-tesla-1000", and "CUDA/11.8" so
dashed and slashed resource tokens are considered ordinary; then in the
implementation of is_ordinary_string (where the operator regex lives) remove '-'
and '/' from the operator character class or replace the simple [\+\-\*/\%] with
a more specific pattern that only matches arithmetic operators when surrounded
by whitespace (e.g. require \s+before/after) so hyphens and slashes in resource
names aren’t treated as operators and those new tests pass.
There was a problem hiding this comment.
@cmeesters I have seen this after the merge, but could you review the suggestion for a potential follow-up PR?
There was a problem hiding this comment.
Seems like the humans are having a chat. I'll hop back into my burrow for now. If you need me again, just tag @coderabbitai in a new comment, and I'll come hopping out!
🤖 I have created a release *beep* *boop* --- ## [9.11.0](v9.10.1...v9.11.0) (2025-09-05) ### Features * add landing page to the report containing custom metadata defined with a YTE yaml template ([#3452](#3452)) ([c754d80](c754d80)) * Issue a warning when unsupported characters are used in wildcard names ([#1587](#1587)) ([fa57355](fa57355)) ### Bug Fixes * avoid checking output files in immediate-submit mode ([#3569](#3569)) ([58c42cf](58c42cf)) * clarify --keep-going flag help text to distinguish runtime vs DAG construction errors ([#3705](#3705)) ([a3a8ef4](a3a8ef4)) * enable docstring assignment in `use rule ... with:` block ([#3710](#3710)) ([2191962](2191962)) * fix missing attribute error in greedy scheduler settings when using touch, dryrun or immediate submit. ([6471004](6471004)) * only trigger script with CODE label ([#3707](#3707)) ([2d01f8c](2d01f8c)) * parser.py incorrectly parsing multiline f-strings ([#3701](#3701)) ([06ece76](06ece76)) * parsing ordinary yaml strings ([#3506](#3506)) ([ddf334e](ddf334e)) * remove temp files when using checkpoints ([#3716](#3716)) ([5ff3e20](5ff3e20)) * Restore python rules changes triggering reruns. (GH: [#3213](#3213)) ([#3485](#3485)) ([2f663be](2f663be)) * unit testing ([#3680](#3680)) ([06ba7e6](06ba7e6)) * use queue_input_wait_time when updating queue input jobs ([#3712](#3712)) ([a945a19](a945a19)) ### Documentation * output files within output directories is an error ([#2848](#2848)) ([#2913](#2913)) ([37272e5](37272e5)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
In a rule's resource section, we are able to define `gres="gpu:1"`. In a config file, however, we need to define `gres: "'gpu:1'"` or (on the command line) `--set-resources <rulename>:"'gpu=2'"`. This PR is to try, whether a common setting of simple, non-nested strings is even possible. ### QC This PR primarily is intended to check whether it will pass current QC checks. * [ ] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake). I did not cross the docs question - will be necessary, when successful. <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit * **New Features** * Improved text handling: the app now classifies ordinary/plain strings to avoid unintended evaluation. * **Bug Fixes** * Resource evaluation now safely passes through plain strings, reducing spurious errors and mis-evaluation. * **Tests** * Added test coverage validating plain-string detection across various inputs to ensure consistent behavior. <!-- end of auto-generated comment: release notes by coderabbit.ai --> --------- Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com> Co-authored-by: Johannes Koester <johannes.koester@uni-due.de> Co-authored-by: Cade Mirchandani <cmirchan@ucsc.edu> Co-authored-by: Johannes Köster <johannes.koester@tu-dortmund.de>
🤖 I have created a release *beep* *boop* --- ## [9.11.0](snakemake/snakemake@v9.10.1...v9.11.0) (2025-09-05) ### Features * add landing page to the report containing custom metadata defined with a YTE yaml template ([snakemake#3452](snakemake#3452)) ([c754d80](snakemake@c754d80)) * Issue a warning when unsupported characters are used in wildcard names ([snakemake#1587](snakemake#1587)) ([fa57355](snakemake@fa57355)) ### Bug Fixes * avoid checking output files in immediate-submit mode ([snakemake#3569](snakemake#3569)) ([58c42cf](snakemake@58c42cf)) * clarify --keep-going flag help text to distinguish runtime vs DAG construction errors ([snakemake#3705](snakemake#3705)) ([a3a8ef4](snakemake@a3a8ef4)) * enable docstring assignment in `use rule ... with:` block ([snakemake#3710](snakemake#3710)) ([2191962](snakemake@2191962)) * fix missing attribute error in greedy scheduler settings when using touch, dryrun or immediate submit. ([6471004](snakemake@6471004)) * only trigger script with CODE label ([snakemake#3707](snakemake#3707)) ([2d01f8c](snakemake@2d01f8c)) * parser.py incorrectly parsing multiline f-strings ([snakemake#3701](snakemake#3701)) ([06ece76](snakemake@06ece76)) * parsing ordinary yaml strings ([snakemake#3506](snakemake#3506)) ([ddf334e](snakemake@ddf334e)) * remove temp files when using checkpoints ([snakemake#3716](snakemake#3716)) ([5ff3e20](snakemake@5ff3e20)) * Restore python rules changes triggering reruns. (GH: [snakemake#3213](snakemake#3213)) ([snakemake#3485](snakemake#3485)) ([2f663be](snakemake@2f663be)) * unit testing ([snakemake#3680](snakemake#3680)) ([06ba7e6](snakemake@06ba7e6)) * use queue_input_wait_time when updating queue input jobs ([snakemake#3712](snakemake#3712)) ([a945a19](snakemake@a945a19)) ### Documentation * output files within output directories is an error ([snakemake#2848](snakemake#2848)) ([snakemake#2913](snakemake#2913)) ([37272e5](snakemake@37272e5)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).
In a rule's resource section, we are able to define
gres="gpu:1". In a config file, however, we need to definegres: "'gpu:1'"or (on the command line)--set-resources <rulename>:"'gpu=2'". This PR is to try, whether a common setting of simple, non-nested strings is even possible.QC
This PR primarily is intended to check whether it will pass current QC checks.
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).I did not cross the docs question - will be necessary, when successful.
Summary by CodeRabbit
New Features
Bug Fixes
Tests