feat: add helpers for deferred input/output etc. item access#2927
feat: add helpers for deferred input/output etc. item access#2927johanneskoester merged 35 commits intomainfrom
Conversation
|
Please format your code with black: |
|
|
@johanneskoester will this be merged soon? #2918 is waiting for this PR. |
| ITEM = 1 | ||
|
|
||
|
|
||
| class RuleItemProxy: |
There was a problem hiding this comment.
TODO: add a way to also obtain prefixes from such items, e.g. for passing a prefix path.
📝 WalkthroughWalkthroughThis pull request introduces significant enhancements to Snakemake's documentation, API, and utility functions. The changes span multiple components, including the addition of new modules in the Changes
Sequence DiagramsequenceDiagram
participant User
participant SnakemakeAPI
participant WorkflowAPI
participant IOUtils
participant DAG
User->>SnakemakeAPI: Initialize Workflow
SnakemakeAPI->>WorkflowAPI: Create Workflow Instance
WorkflowAPI->>IOUtils: Register Utility Functions
IOUtils-->>WorkflowAPI: Utilities Registered
User->>WorkflowAPI: Execute Workflow Methods
WorkflowAPI->>DAG: Resolve Dependencies
DAG-->>WorkflowAPI: Return Execution Plan
WorkflowAPI-->>User: Workflow Execution Results
This sequence diagram illustrates the high-level interaction between the Snakemake API components during workflow initialization and execution, highlighting the role of utility functions and dependency resolution. Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Outside diff range and nitpick comments (6)
snakemake/ioutils/exists.py (1)
9-29: Consider adding type annotations to theexistsfunction for clarity.Adding type annotations to the
existsfunction's parameter and return type can improve code readability and assist with static type checking.Example:
def exists(path: str) -> bool: """Return True if the given file or directory exists. This function considers any storage arguments given to Snakemake. """ # Function implementation remains the same...snakemake/ioutils/rule_items_proxy.py (3)
52-73: Refactor to eliminate code duplication in_deferred_getmethod.The conditional blocks handling
self.namefor "output", "input", "params", and "resources" are very similar. Consider refactoring by mappingself.nameto the corresponding parameter name to reduce code duplication.Example refactor:
def _deferred_get(self, key: Union[str, int], kind: KeyKind): if kind == KeyKind.ATTRIBUTE: def _get(item): return getattr(item, key) elif kind == KeyKind.ITEM: def _get(item): return item[key] else: raise ValueError("kind must be 'attribute' or 'item'") valid_names = {"output", "input", "params", "resources"} if self.name in valid_names: def inner(_wildcards, **kwargs): return _get(kwargs[self.name]) else: raise ValueError(f"Unknown item type: {self.name}") return inner
73-74: Enhance error message for unknown item types.When raising a
ValueErrorfor an unknownself.name, consider including the list of valid item types in the error message to aid debugging.Example:
raise ValueError(f"Unknown item type: {self.name}. Valid types are 'output', 'input', 'params', 'resources'.")
29-75: Consider adding type annotations to class methods for clarity.Adding type annotations to the methods in
RuleItemProxycan improve code readability and assist with static type checking.Example:
class RuleItemProxy: def __init__(self, name: str): self.name = name def __getattr__(self, name: str): return self._deferred_get(name, KeyKind.ATTRIBUTE) def __getitem__(self, key: Union[str, int]): return self._deferred_get(key, KeyKind.ITEM) def _deferred_get(self, key: Union[str, int], kind: KeyKind): # Method implementation...snakemake/ioutils/branch.py (1)
5-62: Consider adding type annotations to thebranchfunction for clarity.Adding type annotations to the
branchfunction's parameters and return type can improve code readability and assist with static type checking.Example:
from typing import Any def branch( condition: Union[Callable[[Any], Any], bool], then: Optional[Union[str, list[str], Callable]] = None, otherwise: Optional[Union[str, list[str], Callable]] = None, cases: Optional[Mapping[Any, Union[str, list[str], Callable]]] = None, ) -> Union[Callable, list[str]]: # Function implementation...docs/snakefiles/rules.rst (1)
218-218: Add missing word for clarity in documentationInsert "that" to improve the sentence: "Afterwards, we will further show a set of semantic helper functions that should increase readability and simplify code..."
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
📒 Files selected for processing (9)
docs/snakefiles/rules.rst(7 hunks)snakemake/ioutils/__init__.py(1 hunks)snakemake/ioutils/branch.py(1 hunks)snakemake/ioutils/collect.py(1 hunks)snakemake/ioutils/evaluate.py(1 hunks)snakemake/ioutils/exists.py(1 hunks)snakemake/ioutils/lookup.py(1 hunks)snakemake/ioutils/rule_items_proxy.py(1 hunks)tests/test_ioutils/Snakefile(2 hunks)
✅ Files skipped from review due to trivial changes (1)
- snakemake/ioutils/collect.py
🧰 Additional context used
📓 Path-based instructions (6)
snakemake/ioutils/__init__.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
snakemake/ioutils/exists.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
snakemake/ioutils/evaluate.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
snakemake/ioutils/branch.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
snakemake/ioutils/lookup.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
snakemake/ioutils/rule_items_proxy.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
🪛 Ruff (0.8.0)
snakemake/ioutils/evaluate.py
12-16: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
🔇 Additional comments (4)
tests/test_ioutils/Snakefile (1)
47-66: LGTM!
The item_access rule is correctly defined, and the assertions appropriately test access to input, output, params, and resources. The usage demonstrates deferred evaluation and item access as intended.
snakemake/ioutils/lookup.py (1)
2-2: Correct import of Mapping from collections.abc
The import of Mapping from collections.abc ensures compatibility with Python 3.3 and above, where Mapping was moved to collections.abc.
snakemake/ioutils/__init__.py (2)
1-6: LGTM!
The imports correctly organize the utility functions for global registration.
9-23: LGTM!
The register_in_globals function properly updates the global namespace with the necessary utilities, enhancing ease of access.
There was a problem hiding this comment.
Actionable comments posted: 3
🧹 Outside diff range and nitpick comments (6)
apidocs/index.rst (1)
63-63: LGTM! Consider standardizing entry format.The change improves clarity by providing a descriptive label for the internal API section. For consistency with other entries in the toctree (e.g.,
snakemake_api,snakemake_utils), consider using a similar naming pattern.- Internal API <api_reference/internal/modules> + snakemake_internal <api_reference/internal/modules>snakemake/ioutils/subpath.py (1)
27-27: Address the TODO: Implement argument type checking for function parameters.The
TODOcomment at line 27 suggests adding type checking for arguments. Implementing this will enhance error handling and ensure that users provide valid inputs to thesubpathfunction.Would you like assistance in creating a decorator or solution for argument type checking?
snakemake/api.py (1)
290-292: Consider refactoring decorators to reduce code duplication.The decorators
_no_dagand_no_execperform similar functions by settingcoresto 1 before method execution. You might consider creating a single parameterized decorator to handle both cases and reduce code duplication.Example refactored decorator:
def _set_cores_to_one(attr_path): def decorator(method): @functools.wraps(method) def wrapper(self, *args, **kwargs): setattr(self, attr_path, 1) return method(self, *args, **kwargs) return wrapper return decoratorApplying the decorator:
@_set_cores_to_one("resource_settings.cores") def lint(self, json: bool = False): # method body @_set_cores_to_one("workflow_api.resource_settings.cores") def generate_unit_tests(self, path: Path): # method bodyAlso applies to: 604-606
snakemake/ioutils/evaluate.py (1)
4-6: Enhance docstring with more detailsConsider adding:
- Parameter description for
expr- Return value description
- Example usage
- Warning about security implications of eval()
tests/test_internals.py (1)
5-21: Enhance test coverage with error and edge casesWhile the current tests cover the basic functionality well, consider adding tests for:
- Error cases:
- Invalid ancestor levels (negative or too large)
- Invalid/malformed paths
- Edge cases:
- Empty paths
- Absolute paths
- Paths with special characters
- Multiple suffix stripping
Would you like assistance in generating these additional test cases?
tests/tests.py (1)
249-251: Consider improving test documentation and assertions.The test function would benefit from:
- A docstring explaining the purpose and expected behavior of the subpath functionality being tested
- Specific assertions or validations to verify the correct behavior
-def test_subpath(): - run(dpath("test_subpath")) +def test_subpath(): + """Test the subpath functionality. + + Verifies that the subpath utility correctly processes paths according to + the specified parameters. + """ + run(dpath("test_subpath")) + # Consider adding specific assertions here
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (40)
.github/workflows/main.yml(1 hunks)apidocs/api_reference/internal/modules.rst(1 hunks)apidocs/api_reference/internal/snakemake.assets.rst(1 hunks)apidocs/api_reference/internal/snakemake.caching.rst(1 hunks)apidocs/api_reference/internal/snakemake.common.rst(1 hunks)apidocs/api_reference/internal/snakemake.common.tests.rst(1 hunks)apidocs/api_reference/internal/snakemake.common.tests.testcases.groups.rst(1 hunks)apidocs/api_reference/internal/snakemake.common.tests.testcases.rst(1 hunks)apidocs/api_reference/internal/snakemake.common.tests.testcases.simple.rst(1 hunks)apidocs/api_reference/internal/snakemake.deployment.rst(1 hunks)apidocs/api_reference/internal/snakemake.executors.rst(1 hunks)apidocs/api_reference/internal/snakemake.ioutils.rst(1 hunks)apidocs/api_reference/internal/snakemake.linting.rst(1 hunks)apidocs/api_reference/internal/snakemake.remote.rst(1 hunks)apidocs/api_reference/internal/snakemake.report.html_reporter.data.rst(1 hunks)apidocs/api_reference/internal/snakemake.report.html_reporter.rst(1 hunks)apidocs/api_reference/internal/snakemake.report.html_reporter.template.components.rst(1 hunks)apidocs/api_reference/internal/snakemake.report.html_reporter.template.rst(1 hunks)apidocs/api_reference/internal/snakemake.report.rst(1 hunks)apidocs/api_reference/internal/snakemake.rst(1 hunks)apidocs/api_reference/internal/snakemake.script.rst(1 hunks)apidocs/api_reference/internal/snakemake.settings.rst(1 hunks)apidocs/api_reference/internal/snakemake.template_rendering.rst(1 hunks)apidocs/api_reference/internal/snakemake.unit_tests.rst(1 hunks)apidocs/api_reference/internal/snakemake.unit_tests.templates.rst(1 hunks)apidocs/api_reference/snakemake_api.rst(1 hunks)apidocs/conf.py(0 hunks)apidocs/index.rst(1 hunks)snakemake/api.py(3 hunks)snakemake/common/__init__.py(2 hunks)snakemake/io.py(0 hunks)snakemake/ioutils/__init__.py(1 hunks)snakemake/ioutils/evaluate.py(1 hunks)snakemake/ioutils/rule_items_proxy.py(1 hunks)snakemake/ioutils/subpath.py(1 hunks)tests/test_internals.py(1 hunks)tests/test_subpath/Snakefile(1 hunks)tests/test_subpath/expected-results/results/foo/bar/b.txt(1 hunks)tests/testapi.py(0 hunks)tests/tests.py(1 hunks)
💤 Files with no reviewable changes (3)
- apidocs/conf.py
- snakemake/io.py
- tests/testapi.py
✅ Files skipped from review due to trivial changes (26)
- tests/test_subpath/expected-results/results/foo/bar/b.txt
- apidocs/api_reference/internal/snakemake.common.tests.testcases.groups.rst
- apidocs/api_reference/internal/snakemake.unit_tests.templates.rst
- apidocs/api_reference/internal/snakemake.assets.rst
- apidocs/api_reference/internal/snakemake.common.tests.testcases.simple.rst
- apidocs/api_reference/internal/snakemake.report.html_reporter.template.components.rst
- apidocs/api_reference/internal/snakemake.report.rst
- apidocs/api_reference/snakemake_api.rst
- apidocs/api_reference/internal/snakemake.script.rst
- apidocs/api_reference/internal/snakemake.common.tests.rst
- apidocs/api_reference/internal/snakemake.report.html_reporter.template.rst
- apidocs/api_reference/internal/snakemake.unit_tests.rst
- apidocs/api_reference/internal/snakemake.template_rendering.rst
- apidocs/api_reference/internal/snakemake.common.tests.testcases.rst
- apidocs/api_reference/internal/snakemake.settings.rst
- apidocs/api_reference/internal/modules.rst
- apidocs/api_reference/internal/snakemake.report.html_reporter.rst
- apidocs/api_reference/internal/snakemake.linting.rst
- apidocs/api_reference/internal/snakemake.deployment.rst
- apidocs/api_reference/internal/snakemake.rst
- apidocs/api_reference/internal/snakemake.caching.rst
- apidocs/api_reference/internal/snakemake.executors.rst
- apidocs/api_reference/internal/snakemake.ioutils.rst
- apidocs/api_reference/internal/snakemake.remote.rst
- apidocs/api_reference/internal/snakemake.report.html_reporter.data.rst
- apidocs/api_reference/internal/snakemake.common.rst
🚧 Files skipped from review as they are similar to previous changes (2)
- snakemake/ioutils/init.py
- snakemake/ioutils/rule_items_proxy.py
🧰 Additional context used
📓 Path-based instructions (6)
tests/tests.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
tests/test_internals.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
snakemake/common/__init__.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
snakemake/ioutils/subpath.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
snakemake/api.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
snakemake/ioutils/evaluate.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
🪛 Ruff (0.8.2)
snakemake/ioutils/evaluate.py
13-17: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
🔇 Additional comments (6)
snakemake/common/__init__.py (2)
15-15: Import statements are appropriate.
The addition of Callable and List from typing enhances type annotations and improves code clarity.
281-293: New functions correctly manage function parameters for dynamic cases.
The get_function_params and overwrite_function_params functions are implemented correctly, allowing dynamic function signature management, which is useful for cases like subpath.
snakemake/ioutils/evaluate.py (1)
13-17: Use raise ... from e to maintain exception context
When re-raising exceptions, using raise WorkflowError(...) from e preserves the original traceback and exception context, aiding in debugging.
🧰 Tools
🪛 Ruff (0.8.2)
13-17: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
.github/workflows/main.yml (1)
107-108: LGTM!
The addition of test_internals.py to the test suite is appropriate and ensures the new functionality is tested in CI.
tests/tests.py (2)
249-251: LGTM: Test structure and placement are consistent.
The test function follows the established patterns in the test suite and is placed in a logical location.
250-250: Verify test data existence and structure.
Please ensure that the test data directory test_subpath exists and contains the necessary test files.
✅ Verification successful
Test data structure is properly set up
The test data directory exists and contains all necessary files:
tests/test_subpath/Snakefilewith the required rule definitionstests/test_subpath/expected-results/results/foo/bar/b.txtmatching the expected output path
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify the existence of test data directory and its contents
# Check if the test directory exists
fd -t d "test_subpath$"
# List contents of the test directory if found
fd . -t f "test_subpath"
Length of output: 170
Script:
#!/bin/bash
# Let's check the contents of the test_subpath directory
ls -la tests/test_subpath/
# Also check if there are any references to test_subpath in the test file
rg "test_subpath" tests/tests.py -B 2 -A 2
Length of output: 370
Script:
#!/bin/bash
# Let's check the contents of the Snakefile and expected-results directory
cat tests/test_subpath/Snakefile
ls -la tests/test_subpath/expected-results/
Length of output: 473
Script:
#!/bin/bash
# Let's check the contents of the results directory in expected-results
ls -R tests/test_subpath/expected-results/results/
Length of output: 216
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Outside diff range and nitpick comments (3)
snakemake/ioutils/branch.py (3)
5-27: Consider enhancing the docstring with parameter types and return value.While the docstring is comprehensive in explaining the functionality, it could benefit from explicitly documenting:
- Parameter types and their constraints
- Return value type and possible values
Example enhancement:
def branch( condition: Union[Callable, bool], then: Optional[Union[str, list[str], Callable]] = None, otherwise: Optional[Union[str, list[str], Callable]] = None, cases: Optional[Mapping] = None, ): """Branch based on a condition that is provided as a function pointer (i.e. a Callable) or a value. If then and optionally otherwise are specified, do the following: If the condition is (or evaluates to) True, return the value of the then parameter. Otherwise, return the value of the otherwise parameter. If cases is specified, do the following: Retrieve the value of the cases mapping using the return value of the condition (if it is a function), or the condition value itself as a key. The given condition function has to take wildcards as its only parameter. Similarly, then, otherwise and the values of the cases mapping can be such functions. If any such function is given to any of those arguments, this function returns a derived input function that will be evaluated once the wildcards are known. + + Args: + condition: A boolean value or a callable that takes wildcards and returns a boolean + then: Value or callable to use when condition is True + otherwise: Value or callable to use when condition is False + cases: Mapping of condition results to values or callables + + Returns: + Union[list[str], Callable]: Either a list of strings or a callable that returns such a list + + Raises: + ValueError: If cases is used together with then or otherwise + KeyError: If the condition result is not found in cases """
29-37: Consider adding type hints to helper functions.The helper functions are well-implemented but could benefit from type hints for better maintainability and IDE support.
- def convert_none(value): + def convert_none(value: Optional[Union[str, list[str]]]) -> list[str]: return value or [] - def handle_callable(value, wildcards): + def handle_callable( + value: Optional[Union[str, list[str], Callable]], + wildcards: Any + ) -> list[str]: if isinstance(value, Callable): return convert_none(value(wildcards)) else: return convert_none(value)
52-65: Consider validating the cases parameter.When
casesis provided, it would be helpful to validate that it's not empty to prevent potential runtime errors.do_branch = do_branch_then_otherwise if cases is not None: if otherwise is not None or then is not None: raise ValueError("Cannot use cases together with then or otherwise.") + if not cases: + raise ValueError("Cases mapping cannot be empty") do_branch = do_branch_cases
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
snakemake/ioutils/branch.py(1 hunks)
🧰 Additional context used
📓 Path-based instructions (1)
snakemake/ioutils/branch.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
🪛 Ruff (0.8.2)
snakemake/ioutils/branch.py
48-48: Local variable e is assigned to but never used
Remove assignment to unused variable e
(F841)
49-49: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
🔇 Additional comments (2)
snakemake/ioutils/branch.py (2)
1-2: LGTM! Well-structured imports and type hints.
The imports are appropriate and the type hints are correctly imported from their respective modules.
5-65: Verify test coverage for the branch function.
Let's ensure that all branches and edge cases are properly tested.
🧰 Tools
🪛 Ruff (0.8.2)
48-48: Local variable e is assigned to but never used
Remove assignment to unused variable e
(F841)
49-49: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
There was a problem hiding this comment.
Actionable comments posted: 4
🧹 Nitpick comments (9)
docs/snakefiles/rules.rst (2)
551-569: Rule item access documentation needs clarification.While the section explains the concept of rule item access, it could benefit from:
- A more explicit example showing how to use the returned function
- Clarification on when to use this pattern vs. direct access
Consider adding a complete example like:
rule process: input: "data/{sample}.txt" output: "results/{sample}.txt" resources: mem_mb=lambda wildcards, input: len(input) * 100
1113-1154: Plain Python rules section needs formatting improvements.The section about shell command execution within Python rules needs better formatting:
- Code blocks should be indented consistently
- Shell command examples should show proper string formatting
Consider reformatting the code examples for better readability:
# Example of shell command in Python rule shell("somecommand {output.somename}") # Example of iterating over shell output for line in shell("somecommand {output.somename}", iterable=True): # Process line passsnakemake/ioutils/branch.py (4)
5-27: Enhance docstring with parameter descriptions and return type.The docstring provides good functional description but could be improved by adding:
- Parameter descriptions with their types
- Return type and description
- Example usage
Consider adding:
""" Parameters ---------- condition : Union[Callable, bool] The condition to evaluate, either a boolean or a callable that returns one then : Optional[Union[str, list[str], Callable]], optional Value to return if condition is True otherwise : Optional[Union[str, list[str], Callable]], optional Value to return if condition is False cases : Optional[Mapping], optional Mapping of condition results to values Returns ------- Union[Callable, list] Either a callable (if any input is callable) or the evaluated result Examples -------- >>> branch(True, "a.txt", "b.txt") ["a.txt"] >>> branch(lambda wc: wc.sample == "A", "a.txt", "b.txt") <function> """
29-37: Add docstrings to helper functions.The helper functions are well-implemented but lack documentation explaining their purpose and behavior.
Consider adding:
def convert_none(value): """Convert None to empty list, otherwise return the value. Parameters ---------- value : Any Value to convert Returns ------- Union[list, Any] Empty list if value is None, otherwise the original value """ return value or [] def handle_callable(value, wildcards): """Handle both callable and non-callable values with wildcard support. Parameters ---------- value : Union[Callable, Any] Value to handle wildcards : dict Wildcard values to pass to callable Returns ------- Any Result of callable(wildcards) if value is callable, otherwise the value """ if isinstance(value, Callable): return convert_none(value(wildcards)) else: return convert_none(value)
44-51: Improve error handling indo_branch_cases.While basic error handling exists, it could be enhanced to provide more context and better error messages.
Consider:
def do_branch_cases(wildcards): res = handle_callable(condition, wildcards) + if not cases: + raise ValueError("Cases mapping cannot be empty") try: selected_case = cases[res] except KeyError: - raise KeyError(f"Key {res} not found in given cases of branch function") + raise KeyError( + f"Result '{res}' (type: {type(res).__name__}) not found in cases. " + f"Available cases: {list(cases.keys())}" + ) return handle_callable(selected_case, wildcards)🧰 Tools
🪛 Ruff (0.8.2)
49-49: Within an
exceptclause, raise exceptions withraise ... from errorraise ... from Noneto distinguish them from errors in exception handling(B904)
52-65: Add input validation for mutually exclusive parameters.The validation of mutually exclusive parameters could be moved to the start of the function for early failure.
Consider:
def branch( condition: Union[Callable, bool], then: Optional[Union[str, list[str], Callable]] = None, otherwise: Optional[Union[str, list[str], Callable]] = None, cases: Optional[Mapping] = None, ): + # Validate mutually exclusive parameters + if cases is not None and (then is not None or otherwise is not None): + raise ValueError("Cannot use cases together with then or otherwise.") + if cases is None and then is None: + raise ValueError("Must provide either cases or then parameter.") + def convert_none(value): return value or [] # ... rest of the function ... - do_branch = do_branch_then_otherwise - if cases is not None: - if otherwise is not None or then is not None: - raise ValueError("Cannot use cases together with then or otherwise.") - do_branch = do_branch_cases + do_branch = do_branch_cases if cases is not None else do_branch_then_otherwise.github/workflows/docs.yml (3)
3-12: Consider adding path filters to optimize workflow triggers.The workflow could be more efficient by only running when documentation-related files change.
on: push: branches: - main + paths: + - 'docs/**' + - 'apidocs/**' + - '.github/workflows/docs.yml' pull_request: + paths: + - 'docs/**' + - 'apidocs/**' + - '.github/workflows/docs.yml'
23-23: Fix YAML formatting issues.There are some minor formatting issues:
- Trailing spaces on lines 23 and 38
- Missing newline at end of file
- name: Setup doc dependencies run: | - pip install -f docs/requirements.txt + pip install -f docs/requirements.txt - name: Setup doc dependencies run: | - pip install -f apidocs/requirements.txt + pip install -f apidocs/requirements.txt pip install -e . - name: Build docs run: | cd apidocs - make html \ No newline at end of file + make html +Also applies to: 38-38, 42-42
🧰 Tools
🪛 yamllint (1.35.1)
[error] 23-23: trailing spaces
(trailing-spaces)
1-42: Consider enhancing the workflow with artifacts and preview deployments.To improve the documentation workflow:
- Upload built documentation as artifacts for verification
- Consider deploying documentation previews for PRs using GitHub Pages or similar
Example implementation for artifacts:
- name: Upload documentation artifact uses: actions/upload-artifact@v3 with: name: documentation path: | docs/_build/html/ apidocs/_build/html/🧰 Tools
🪛 yamllint (1.35.1)
[error] 23-23: trailing spaces
(trailing-spaces)
[error] 38-38: trailing spaces
(trailing-spaces)
[error] 42-42: no new line character at the end of file
(new-line-at-end-of-file)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (6)
.github/workflows/docs.yml(1 hunks)docs/snakefiles/rules.rst(7 hunks)snakemake/ioutils/branch.py(1 hunks)snakemake/ioutils/lookup.py(2 hunks)snakemake/ioutils/subpath.py(1 hunks)tests/test_internals.py(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
- tests/test_internals.py
- snakemake/ioutils/subpath.py
🧰 Additional context used
📓 Path-based instructions (2)
snakemake/ioutils/lookup.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
snakemake/ioutils/branch.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
🪛 yamllint (1.35.1)
.github/workflows/docs.yml
[error] 23-23: trailing spaces
(trailing-spaces)
[error] 38-38: trailing spaces
(trailing-spaces)
[error] 42-42: no new line character at the end of file
(new-line-at-end-of-file)
🪛 Ruff (0.8.2)
snakemake/ioutils/branch.py
49-49: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling
(B904)
🔇 Additional comments (4)
docs/snakefiles/rules.rst (2)
12-22: Example rule definition has been improved.
The example has been updated to use a more concrete rule name (myrule) and structured input/output definitions. This improves clarity and follows best practices for documentation.
214-218: Clear introduction to helper functions section.
The new section provides a good overview of helper functions and their purpose in simplifying rule definitions.
snakemake/ioutils/branch.py (1)
1-2: LGTM! Well-structured imports and type hints.
The imports are appropriate and the type hints are well-defined using modern Python typing conventions.
snakemake/ioutils/lookup.py (1)
175-175: LGTM! Improved error message clarity.
The error message now clearly indicates that the query parameter is specific to pandas DataFrame or Series objects.
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (3)
docs/snakefiles/rules.rst (3)
551-569: Documentation needs examples for rule item access.While the section explains the concept of rule item access well, it would benefit from concrete examples showing:
- How to access different rule items (input, output, resources, etc.)
- Common use cases and patterns
Consider adding examples like:
# Example accessing input files rule process: input: "input.txt" output: "output.txt" params: threshold=lambda wildcards, input: get_threshold(input[0]) shell: "process {input} {output}" # Example accessing resources rule analyze: input: "data.txt" output: "results.txt" resources: mem_mb=1000 params: mem=lambda wildcards, resources: resources.mem_mb / 1000 shell: "analyze --memory {params.mem}G {input} {output}"
570-605: Documentation needs clarification on sub-path access behavior.The sub-path access section would benefit from:
- A note about path normalization behavior
- Examples of error handling when paths don't match expected patterns
- Clarification on behavior with absolute vs relative paths
3088-3115: Important note about shell command placeholders needs emphasis.The note about escaping placeholders in shell commands is crucial for preventing errors, but it could be more prominent. Consider:
- Adding a warning box or stronger visual emphasis
- Including examples of common mistakes
Add examples like:
# Wrong - will try to evaluate {input} rule wrong: shell: "echo 'Using {input} for processing' > log.txt" # Correct - escapes {input} rule correct: shell: "echo 'Using {{input}} for processing' > log.txt"
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
docs/snakefiles/rules.rst(7 hunks)
🔇 Additional comments (4)
docs/snakefiles/rules.rst (4)
12-22: LGTM: Example rule definition is clear and well-structured.
The updated example provides a clear and practical demonstration of a basic Snakemake rule with multiple inputs and outputs. The formatting and indentation are consistent.
214-218: LGTM: Clear introduction to rule helpers section.
The new section provides a good overview of helper functions and their purpose in simplifying rule definitions. The text effectively bridges between basic concepts and more advanced helper functions.
314-315: LGTM: Clear section header for semantic helpers.
The section header is appropriately formatted and provides a clear transition to the semantic helpers documentation.
1132-1173: LGTM: Plain Python rules section is well documented.
The section provides clear explanations and examples of using Python code in rules, including:
- Proper file handling with input/output
- Access to rule properties
- Integration with shell commands
There was a problem hiding this comment.
Actionable comments posted: 1
♻️ Duplicate comments (1)
.github/workflows/docs.yml (1)
33-51: 🛠️ Refactor suggestionApply the same enhancements to the apidocs job.
For consistency with the docs job, apply similar improvements:
Apply these changes:
apidocs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - uses: actions/setup-python@v4 with: - python-version: '3.x' + python-version: '3.11' # Specify exact version for consistency + cache: 'pip' - name: Setup doc dependencies run: | python -m pip install --upgrade pip pip install -r apidocs/requirements.txt pip install -e . - name: Build docs run: | cd apidocs make html SPHINXOPTS="-W --keep-going -n" + + - name: Upload API documentation + uses: actions/upload-artifact@v4 + with: + name: apidocs-html + path: apidocs/_build/html/ +🧰 Tools
🪛 actionlint (1.7.4)
38-38: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🪛 yamllint (1.35.1)
[error] 37-37: trailing spaces
(trailing-spaces)
[error] 47-47: trailing spaces
(trailing-spaces)
[error] 51-51: no new line character at the end of file
(new-line-at-end-of-file)
🧹 Nitpick comments (1)
.github/workflows/docs.yml (1)
1-51: Fix minor formatting issues.There are some formatting issues in the file:
- Remove trailing spaces on lines 19, 28, 37, and 47
- Add a newline at the end of the file
These can be automatically fixed by running:
#!/bin/bash # Remove trailing spaces and ensure newline at EOF sed -i 's/[[:space:]]*$//' .github/workflows/docs.yml echo >> .github/workflows/docs.yml🧰 Tools
🪛 actionlint (1.7.4)
20-20: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
38-38: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🪛 yamllint (1.35.1)
[error] 19-19: trailing spaces
(trailing-spaces)
[error] 28-28: trailing spaces
(trailing-spaces)
[error] 37-37: trailing spaces
(trailing-spaces)
[error] 47-47: trailing spaces
(trailing-spaces)
[error] 51-51: no new line character at the end of file
(new-line-at-end-of-file)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/docs.yml(1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.4)
.github/workflows/docs.yml
20-20: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
38-38: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🪛 yamllint (1.35.1)
.github/workflows/docs.yml
[error] 19-19: trailing spaces
(trailing-spaces)
[error] 28-28: trailing spaces
(trailing-spaces)
[error] 37-37: trailing spaces
(trailing-spaces)
[error] 47-47: trailing spaces
(trailing-spaces)
[error] 51-51: no new line character at the end of file
(new-line-at-end-of-file)
🔇 Additional comments (1)
.github/workflows/docs.yml (1)
1-13: LGTM! Well-configured workflow triggers and concurrency settings.
The workflow configuration properly handles both push events to main and pull requests, with appropriate concurrency settings to prevent build queue congestion.
| jobs: | ||
| docs: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
|
|
||
| - uses: actions/setup-python@v4 | ||
| with: | ||
| python-version: '3.x' | ||
|
|
||
| - name: Setup doc dependencies | ||
| run: | | ||
| python -m pip install --upgrade pip | ||
| pip install -r docs/requirements.txt | ||
|
|
||
| - name: Build docs | ||
| run: | | ||
| cd docs | ||
| make html SPHINXOPTS="-W --keep-going -n" |
There was a problem hiding this comment.
🛠️ Refactor suggestion
Enhance the docs job configuration with caching and specific Python version.
While the basic setup is good, consider these improvements for better efficiency and reliability:
Apply these changes:
docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
- python-version: '3.x'
+ python-version: '3.11' # Specify exact version for consistency
+ cache: 'pip'
- name: Setup doc dependencies
run: |
python -m pip install --upgrade pip
pip install -r docs/requirements.txt
- name: Build docs
run: |
cd docs
make html SPHINXOPTS="-W --keep-going -n"
+
+ - name: Upload documentation
+ uses: actions/upload-artifact@v4
+ with:
+ name: docs-html
+ path: docs/_build/html/📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| jobs: | |
| docs: | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v4 | |
| - uses: actions/setup-python@v4 | |
| with: | |
| python-version: '3.x' | |
| - name: Setup doc dependencies | |
| run: | | |
| python -m pip install --upgrade pip | |
| pip install -r docs/requirements.txt | |
| - name: Build docs | |
| run: | | |
| cd docs | |
| make html SPHINXOPTS="-W --keep-going -n" | |
| jobs: | |
| docs: | |
| runs-on: ubuntu-latest | |
| steps: | |
| - uses: actions/checkout@v4 | |
| - uses: actions/setup-python@v4 | |
| with: | |
| python-version: '3.11' # Specify exact version for consistency | |
| cache: 'pip' | |
| - name: Setup doc dependencies | |
| run: | | |
| python -m pip install --upgrade pip | |
| pip install -r docs/requirements.txt | |
| - name: Build docs | |
| run: | | |
| cd docs | |
| make html SPHINXOPTS="-W --keep-going -n" | |
| - name: Upload documentation | |
| uses: actions/upload-artifact@v4 | |
| with: | |
| name: docs-html | |
| path: docs/_build/html/ |
🧰 Tools
🪛 actionlint (1.7.4)
20-20: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🪛 yamllint (1.35.1)
[error] 19-19: trailing spaces
(trailing-spaces)
[error] 28-28: trailing spaces
(trailing-spaces)
There was a problem hiding this comment.
Actionable comments posted: 0
♻️ Duplicate comments (2)
.github/workflows/docs.yml (2)
14-32: 🛠️ Refactor suggestionEnhance the docs job configuration.
The following improvements are recommended:
Apply these changes:
docs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - + - uses: actions/setup-python@v4 with: - python-version: '3.x' + python-version: '3.11' # Specify exact version for consistency + cache: 'pip' - name: Setup doc dependencies run: | python -m pip install --upgrade pip pip install -r docs/requirements.txt - + - name: Build docs run: | cd docs make html SPHINXOPTS="-W --keep-going -n" + + - name: Upload documentation + uses: actions/upload-artifact@v4 + with: + name: docs-html + path: docs/_build/html/🧰 Tools
🪛 actionlint (1.7.4)
20-20: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🪛 yamllint (1.35.1)
[error] 19-19: trailing spaces
(trailing-spaces)
[error] 28-28: trailing spaces
(trailing-spaces)
33-51: 🛠️ Refactor suggestionApply similar improvements to the apidocs job.
The apidocs job needs the same enhancements as the docs job, plus proper file ending.
Apply these changes:
apidocs: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - + - uses: actions/setup-python@v4 with: - python-version: '3.x' + python-version: '3.11' # Specify exact version for consistency + cache: 'pip' - name: Setup doc dependencies run: | python -m pip install --upgrade pip pip install -r apidocs/requirements.txt pip install -e . - + - name: Build docs run: | cd apidocs make html SPHINXOPTS="-W --keep-going -n" + + - name: Upload API documentation + uses: actions/upload-artifact@v4 + with: + name: apidocs-html + path: apidocs/_build/html/ +🧰 Tools
🪛 actionlint (1.7.4)
38-38: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🪛 yamllint (1.35.1)
[error] 37-37: trailing spaces
(trailing-spaces)
[error] 47-47: trailing spaces
(trailing-spaces)
[error] 51-51: no new line character at the end of file
(new-line-at-end-of-file)
🧹 Nitpick comments (3)
docs/snakefiles/rules.rst (3)
12-24: Improve rule example clarity and documentation.The example rule definition could be enhanced with more descriptive variable names and additional comments explaining the purpose of each section.
- rule myrule: + rule process_data: input: - "path/to/inputfile", - "path/to/other/inputfile", + # Primary input file + "data/input/sample.txt", + # Reference data + "data/reference/ref.txt", output: - "path/to/outputfile", - "path/to/another/outputfile", + # Main result file + "results/processed.txt", + # Additional metrics + "results/metrics.txt", shell: - "somecommand {input} {output}" + # Process input data and generate results + "process_data --input {input} --output {output}"
551-569: Enhance rule item access documentation with examples.The rule item access documentation would benefit from concrete examples showing different access patterns.
Add examples like:
# Example 1: Accessing input files by name rule process: input: reads="data/{sample}.fastq", reference="ref/genome.fa" params: # Access named input file def get_reads(wildcards, input): return input.reads # Example 2: Accessing input files by index rule merge: input: "part1.txt", "part2.txt" params: # Access input file by index def get_first(wildcards, input): return input[0]
3088-3115: Improve procedural rule definition example.The procedural rule definition example could be enhanced with better error handling and validation.
for tool in ["bcftools", "freebayes"]: + # Validate tool existence and version + tool_version = { + "bcftools": "1.9", + "freebayes": "1.3.2" + } rule: name: f"call_variants_{tool}" input: f"path/to/{tool}/inputfile" output: f"path/to/{tool}/outputfile" + conda: + # Ensure consistent tool versions + f"envs/{tool}.yaml" shell: - f"{tool} {{input}} > {{output}}" + # Add version check and error handling + f"set -e; " + f"{tool} --version | grep {tool_version[tool]}; " + f"{tool} {{input}} > {{output}} || {{ echo '{tool} failed'; exit 1; }}"
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
⛔ Files ignored due to path filters (1)
CHANGELOG.mdis excluded by!CHANGELOG.md
📒 Files selected for processing (6)
.github/workflows/docs.yml(1 hunks)docs/conf.py(1 hunks)docs/getting_started/installation.rst(1 hunks)docs/index.rst(1 hunks)docs/snakefiles/rules.rst(7 hunks)docs/tutorial/short.rst(0 hunks)
💤 Files with no reviewable changes (1)
- docs/tutorial/short.rst
✅ Files skipped from review due to trivial changes (2)
- docs/getting_started/installation.rst
- docs/index.rst
🧰 Additional context used
📓 Path-based instructions (1)
docs/conf.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
🪛 actionlint (1.7.4)
.github/workflows/docs.yml
20-20: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
38-38: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🪛 yamllint (1.35.1)
.github/workflows/docs.yml
[error] 19-19: trailing spaces
(trailing-spaces)
[error] 28-28: trailing spaces
(trailing-spaces)
[error] 37-37: trailing spaces
(trailing-spaces)
[error] 47-47: trailing spaces
(trailing-spaces)
[error] 51-51: no new line character at the end of file
(new-line-at-end-of-file)
🔇 Additional comments (7)
docs/conf.py (2)
100-101: LGTM! Clean approach to handle autosectionlabel warnings.
The addition of suppress_warnings for autosectionlabel is a focused solution that reduces build noise while maintaining visibility of other important warnings.
92-98: LGTM! Verify the excluded .rst files.
The addition of specific .rst files to exclude_patterns is well-structured and follows documentation best practices.
Let's verify these files are auto-generated report files:
✅ Verification successful
LGTM! The excluded .rst files are auto-generated report files.
The files are correctly excluded as they are auto-generated report files used in the Snakemake workflow:
report/cars.rstis a caption file for the datavzrd table outputreport/horsepower_vs_mpg.rstis a caption file for plot outputsreport/workflow.rstis the main workflow report file
These files are generated during workflow execution and should not be included in the documentation build.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check if these are indeed report files that should be excluded
fd -t f "cars\.rst|horsepower_vs_mpg\.rst|workflow\.rst" -x file {}
Length of output: 179
Script:
#!/bin/bash
# Let's check if these files exist and examine their content
fd -t f "cars\.rst|horsepower_vs_mpg\.rst|workflow\.rst"
# Also check for any related generation scripts or patterns
rg -l "cars\.rst|horsepower_vs_mpg\.rst|workflow\.rst" -g "!docs/conf.py"
Length of output: 570
Script:
#!/bin/bash
# Let's check the content of the Snakefile to see if these are generated files
cat "docs/tutorial/interaction_visualization_reporting/workdir/workflow/Snakefile"
# Also check the tutorial content for context
head -n 50 "docs/tutorial/interaction_visualization_reporting/tutorial.rst"
Length of output: 4896
docs/snakefiles/rules.rst (3)
214-217: LGTM: Clear introduction to helper functions section.
The introduction effectively sets up the context for the helper functions that follow.
314-315: LGTM: Clear section header for semantic helpers.
The section header is appropriately formatted and descriptive.
Line range hint 1041-1173: LGTM: Comprehensive plain Python rules documentation.
The documentation for plain Python rules is thorough and includes good examples of both basic usage and advanced features like shell command integration.
.github/workflows/docs.yml (2)
1-13: LGTM! Well-configured workflow triggers and concurrency settings.
The workflow configuration follows best practices:
- Appropriate triggers for documentation builds
- Proper concurrency handling to prevent queue buildup
14-51: Add documentation verification steps.
Consider adding verification steps to ensure documentation quality:
- Add link checking:
- name: Build docs
run: |
cd docs
make html SPHINXOPTS="-W --keep-going -n"
+ make linkcheck # Verify external links- Run this script to verify documentation structure:
✅ Verification successful
Add linkcheck target to documentation workflow
The linkcheck target is already available in both Makefiles, but not being used. Add it to verify external links:
- name: Build docs
run: |
cd docs
make html SPHINXOPTS="-W --keep-going -n"
+ make linkcheck
- name: Build docs
run: |
cd apidocs
make html SPHINXOPTS="-W --keep-going -n"
+ make linkcheck- Both
docs/requirements.txtandapidocs/requirements.txthave the necessary Sphinx dependencies - Both Makefiles already include the
linkchecktarget - No additional verification steps needed as the documentation structure is complete and well-organized
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Verify documentation requirements and structure
# Check if requirements files exist and are not empty
if [ ! -s docs/requirements.txt ] || [ ! -s apidocs/requirements.txt ]; then
echo "Error: Documentation requirements files are missing or empty"
exit 1
fi
# Check for essential Sphinx configuration
for dir in docs apidocs; do
if [ ! -f "$dir/conf.py" ]; then
echo "Error: Missing conf.py in $dir"
exit 1
fi
done
# Check for index files
for dir in docs apidocs; do
if [ ! -f "$dir/index.rst" ] && [ ! -f "$dir/index.md" ]; then
echo "Error: Missing index file in $dir"
exit 1
fi
done
Length of output: 314
Script:
#!/bin/bash
# Let's verify if the documentation structure exists and check its contents
echo "=== Checking requirements files ==="
for dir in docs apidocs; do
echo "Contents of $dir/requirements.txt:"
if [ -f "$dir/requirements.txt" ]; then
cat "$dir/requirements.txt"
else
echo "File not found"
fi
done
echo -e "\n=== Checking Sphinx configuration ==="
for dir in docs apidocs; do
echo "Checking $dir/conf.py:"
if [ -f "$dir/conf.py" ]; then
# Look for linkcheck configuration
echo "Linkcheck settings in $dir/conf.py:"
grep -A 5 "linkcheck_" "$dir/conf.py" || echo "No linkcheck settings found"
else
echo "File not found"
fi
done
echo -e "\n=== Checking Makefile targets ==="
for dir in docs apidocs; do
echo "Available make targets in $dir/Makefile:"
if [ -f "$dir/Makefile" ]; then
grep "^[a-zA-Z].*:" "$dir/Makefile"
else
echo "Makefile not found"
fi
done
Length of output: 2473
🧰 Tools
🪛 actionlint (1.7.4)
20-20: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
38-38: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🪛 yamllint (1.35.1)
[error] 19-19: trailing spaces
(trailing-spaces)
[error] 28-28: trailing spaces
(trailing-spaces)
[error] 37-37: trailing spaces
(trailing-spaces)
[error] 47-47: trailing spaces
(trailing-spaces)
[error] 51-51: no new line character at the end of file
(new-line-at-end-of-file)
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Nitpick comments (1)
apidocs/requirements.txt (1)
12-14: Add version constraints for new dependencies.The newly added packages lack version constraints, which could lead to compatibility issues in the future. Consider adding specific version constraints:
-snakemake-interface-storage-plugins -flask -pytest +snakemake-interface-storage-plugins>=0.1.0 +flask>=2.0.0 +pytest>=7.0.0
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
apidocs/api_reference/internal/snakemake.executors.rst(1 hunks)apidocs/requirements.txt(1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
- apidocs/api_reference/internal/snakemake.executors.rst
There was a problem hiding this comment.
Actionable comments posted: 2
🧹 Nitpick comments (1)
.github/workflows/docs.yml (1)
1-63: Fix formatting issues.The YAML file has several formatting issues:
- Multiple trailing spaces (lines 19, 28, 38, 43, 53)
- Missing newline at end of file
Consider using an editor with YAML support or running
yamllintlocally to catch these issues before committing.🧰 Tools
🪛 actionlint (1.7.4)
20-20: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
44-44: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🪛 yamllint (1.35.1)
[error] 19-19: trailing spaces
(trailing-spaces)
[error] 28-28: trailing spaces
(trailing-spaces)
[error] 38-38: trailing spaces
(trailing-spaces)
[error] 43-43: trailing spaces
(trailing-spaces)
[error] 53-53: trailing spaces
(trailing-spaces)
[error] 63-63: no new line character at the end of file
(new-line-at-end-of-file)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (3)
.github/workflows/docs.yml(1 hunks)apidocs/api_reference/snakemake_api.rst(1 hunks)apidocs/api_reference/snakemake_utils.rst(1 hunks)
✅ Files skipped from review due to trivial changes (1)
- apidocs/api_reference/snakemake_utils.rst
🚧 Files skipped from review as they are similar to previous changes (1)
- apidocs/api_reference/snakemake_api.rst
🧰 Additional context used
🪛 actionlint (1.7.4)
.github/workflows/docs.yml
20-20: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
44-44: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🪛 yamllint (1.35.1)
.github/workflows/docs.yml
[error] 19-19: trailing spaces
(trailing-spaces)
[error] 28-28: trailing spaces
(trailing-spaces)
[error] 38-38: trailing spaces
(trailing-spaces)
[error] 43-43: trailing spaces
(trailing-spaces)
[error] 53-53: trailing spaces
(trailing-spaces)
[error] 63-63: no new line character at the end of file
(new-line-at-end-of-file)
🔇 Additional comments (2)
.github/workflows/docs.yml (2)
1-13: LGTM! Well-structured workflow configuration.
The workflow triggers and concurrency settings are properly configured to:
- Build docs on main branch pushes and pull requests
- Cancel redundant builds for PRs to avoid resource waste
20-22: 🛠️ Refactor suggestion
Specify exact Python version and enable pip caching.
Using a floating Python version ('3.x') could lead to inconsistent builds. Also, pip caching would improve workflow efficiency.
- uses: actions/setup-python@v4
with:
- python-version: '3.x'
+ python-version: '3.11' # Specify exact version for consistency
+ cache: 'pip'Likely invalid or redundant comment.
🧰 Tools
🪛 actionlint (1.7.4)
20-20: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
| - name: Build docs | ||
| run: | | ||
| cd docs | ||
| make html SPHINXOPTS="-W --keep-going -n" |
There was a problem hiding this comment.
Ensure consistent Sphinx options between jobs.
The docs job uses -W flag (treats warnings as errors) while apidocs job doesn't. This inconsistency could lead to different quality standards between documentation builds.
Make the Sphinx options consistent across both jobs:
# In apidocs job
make html SPHINXOPTS="--keep-going -n"
+ # Should be
+ make html SPHINXOPTS="-W --keep-going -n"Also applies to: 54-57
Co-authored-by: coderabbitai[bot] <136622811+coderabbitai[bot]@users.noreply.github.com>
…o feat/item-access
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
.github/workflows/docs.yml (1)
63-63: Add newline at end of file.Add a newline character at the end of the file to follow YAML best practices.
Apply this change:
- path: apidocs/_build/html/ + path: apidocs/_build/html/ +🧰 Tools
🪛 yamllint (1.35.1)
[error] 63-63: no new line character at the end of file
(new-line-at-end-of-file)
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
.github/workflows/docs.yml(1 hunks)
🧰 Additional context used
🪛 actionlint (1.7.4)
.github/workflows/docs.yml
20-20: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
44-44: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🪛 yamllint (1.35.1)
.github/workflows/docs.yml
[error] 19-19: trailing spaces
(trailing-spaces)
[error] 28-28: trailing spaces
(trailing-spaces)
[error] 38-38: trailing spaces
(trailing-spaces)
[error] 43-43: trailing spaces
(trailing-spaces)
[error] 53-53: trailing spaces
(trailing-spaces)
[error] 63-63: no new line character at the end of file
(new-line-at-end-of-file)
🔇 Additional comments (3)
.github/workflows/docs.yml (3)
1-13: LGTM! Well-configured workflow triggers and concurrency settings.
The workflow configuration follows best practices by:
- Triggering on relevant events (push to main and PRs)
- Using concurrency settings to cancel redundant PR builds
15-38: Enhance the docs job configuration with caching and specific Python version.
While the basic setup works, several improvements would enhance efficiency and reliability:
Apply these changes:
docs:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v4
with:
- python-version: '3.x'
+ python-version: '3.11' # Specify exact version for consistency
+ cache: 'pip'🧰 Tools
🪛 actionlint (1.7.4)
20-20: the runner of "actions/setup-python@v4" action is too old to run on GitHub Actions. update the action's version to fix this issue
(action)
🪛 yamllint (1.35.1)
[error] 19-19: trailing spaces
(trailing-spaces)
[error] 28-28: trailing spaces
(trailing-spaces)
[error] 38-38: trailing spaces
(trailing-spaces)
54-57: Ensure consistent Sphinx options between jobs.
The docs job uses -W flag while apidocs job doesn't, leading to inconsistent documentation quality standards.
Apply this change:
- make html SPHINXOPTS="--keep-going -n"
+ make html SPHINXOPTS="-W --keep-going -n"
|
🤖 I have created a release *beep* *boop* --- ## [8.26.0](v8.25.5...v8.26.0) (2024-12-23) ### Features * add helpers for deferred input/output etc. item access ([#2927](#2927)) ([2cca9bc](2cca9bc)) ### Bug Fixes * convert parameters so they can be serialized ([#2925](#2925)) ([9e653fb](9e653fb)) * correct formatting of R preamble ([#2425](#2425)) ([5380cae](5380cae)) * fix modification checks for scripts and and notebooks containing wildcards or params in their paths ([#2751](#2751)) ([773568d](773568d)) * Improved handling of missing output files in group job postprocessing, accounting for temporary files. ([#1765](#1765)) ([bac06ba](bac06ba)) * mtime of script or notebook not triggering workflow without metadata ([#3148](#3148)) ([e8a0b83](e8a0b83)) * Pass `host` attribute to `GitlabFile` instantiation within class methods ([#3155](#3155)) ([9ef52de](9ef52de)) * problem with spaces in path ([#3236](#3236)) ([2d08c63](2d08c63)) * require current yte release which contains an important bug fix for cases where numpy/pandas data is passed to templates ([#3227](#3227)) ([c3339da](c3339da)) * rerun jobs if previously failed but rule was changed afterwards (thanks to [@laf070810](https://github.com/laf070810) for bringing this up) ([#3237](#3237)) ([1dc0084](1dc0084)) * use relpath for configfiles added to the source archive (thanks to [@sposadac](https://github.com/sposadac) for the initial solution) ([#3240](#3240)) ([bff3844](bff3844)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
🤖 I have created a release *beep* *boop* --- ## [8.26.0](snakemake/snakemake@v8.25.5...v8.26.0) (2024-12-23) ### Features * add helpers for deferred input/output etc. item access ([snakemake#2927](snakemake#2927)) ([2cca9bc](snakemake@2cca9bc)) ### Bug Fixes * convert parameters so they can be serialized ([snakemake#2925](snakemake#2925)) ([9e653fb](snakemake@9e653fb)) * correct formatting of R preamble ([snakemake#2425](snakemake#2425)) ([5380cae](snakemake@5380cae)) * fix modification checks for scripts and and notebooks containing wildcards or params in their paths ([snakemake#2751](snakemake#2751)) ([773568d](snakemake@773568d)) * Improved handling of missing output files in group job postprocessing, accounting for temporary files. ([snakemake#1765](snakemake#1765)) ([bac06ba](snakemake@bac06ba)) * mtime of script or notebook not triggering workflow without metadata ([snakemake#3148](snakemake#3148)) ([e8a0b83](snakemake@e8a0b83)) * Pass `host` attribute to `GitlabFile` instantiation within class methods ([snakemake#3155](snakemake#3155)) ([9ef52de](snakemake@9ef52de)) * problem with spaces in path ([snakemake#3236](snakemake#3236)) ([2d08c63](snakemake@2d08c63)) * require current yte release which contains an important bug fix for cases where numpy/pandas data is passed to templates ([snakemake#3227](snakemake#3227)) ([c3339da](snakemake@c3339da)) * rerun jobs if previously failed but rule was changed afterwards (thanks to [@laf070810](https://github.com/laf070810) for bringing this up) ([snakemake#3237](snakemake#3237)) ([1dc0084](snakemake@1dc0084)) * use relpath for configfiles added to the source archive (thanks to [@sposadac](https://github.com/sposadac) for the initial solution) ([snakemake#3240](snakemake#3240)) ([bff3844](snakemake@bff3844)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>



QC
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).Summary by CodeRabbit
New Features
snakemake.assets,snakemake.caching, and others.subpathutility.Bug Fixes
Documentation
Chores