fix: update rust-script usage to recent version (v0.35.0) #3183#3208
fix: update rust-script usage to recent version (v0.35.0) #3183#3208johanneskoester merged 3 commits intosnakemake:mainfrom
Conversation
📝 Walkthrough📝 WalkthroughWalkthroughThe pull request introduces significant updates to the Snakemake documentation and related codebase. Key changes include an update to the minimum required version of Changes
Possibly related PRs
Suggested reviewers
Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media? 🪧 TipsChatThere are 3 ways to chat with CodeRabbit:
Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments. CodeRabbit Commands (Invoked using PR comments)
Other keywords and placeholders
CodeRabbit Configuration File (
|
There was a problem hiding this comment.
Actionable comments posted: 1
🧹 Outside diff range and nitpick comments (11)
tests/test_script_rs/envs/rust.yaml (1)
5-5: Document the breaking change.Consider adding a comment in the YAML file to document the breaking change with the
--featuresflag, as this will help future maintainers understand why this specific version requirement exists.channels: - conda-forge - bioconda dependencies: + # >= 0.35.0 required due to breaking change: --features must be placed after -- - rust-script>=0.35.0 - openssl - c-compiler - pkg-configtests/test_script_rs/scripts/test.py (2)
1-1: Improve assertion message for better debugging.Add a descriptive message to the assertion to make test failures more informative.
-assert snakemake.input.get("sort", "missing") == "missing" +assert snakemake.input.get("sort", "missing") == "missing", "Expected 'sort' input parameter to be undefined or 'missing'"🧰 Tools
🪛 Ruff
1-1: Undefined name
snakemake(F821)
3-4: Add error handling for file operations.Consider adding error handling to gracefully handle potential file operation failures.
-with open(snakemake.output[0], "w") as out: - print(1, 2, 3, file=out) +try: + with open(snakemake.output[0], "w") as out: + print(1, 2, 3, file=out) +except IOError as e: + raise RuntimeError(f"Failed to write to output file: {e}")🧰 Tools
🪛 Ruff
3-3: Undefined name
snakemake(F821)
tests/test_script/Snakefile (7)
14-14: Remove unnecessary empty lineThis empty line doesn't serve any purpose and can be removed for better code organization.
Line range hint
16-26: Add a descriptive name to the ruleConsider adding a meaningful name to this rule for better maintainability and easier referencing. For example:
-rule: +rule process_test_r:
28-33: Add rule name and consider path handling consistency
- Add a descriptive name to the rule
- Consider standardizing path handling across all rules - either use
Pathconsistently or use string literals consistently-rule: +rule generate_test_input: output: "test.in" script: - Path("scripts/test.py") + "scripts/test.py" # for consistency with other rules
Line range hint
35-44: Add rule name and document test parameter
- Add a descriptive name to the rule
- Consider adding a comment explaining the purpose of the "testparam" parameter
-rule: +rule generate_test_html: output: "test.html" params: + # Parameter used for testing Rmd parameter passing test="testparam" conda: "envs/r.yaml" script: "scripts/test.Rmd"
Line range hint
46-53: Add rule name and document purposeConsider adding a rule name and documentation to clarify the purpose of this relative source processing:
-rule: +rule process_relative_source: + """Process R script with relative source references.""" output: "rel_source.out" conda: "envs/r.yaml" script: "scripts/rel_source.R"
Line range hint
55-69: Add descriptive rule nameAdd a meaningful name to this Julia processing rule:
-rule: +rule process_julia:
Line range hint
1-93: Clean separation of test suites achievedThe removal of Rust-related rules and their relocation to a dedicated test suite (
tests/test_script_rs/) aligns well with the PR objectives. This separation of concerns improves maintainability and makes the test structure clearer.tests/test_script_rs/scripts/test-manifest.rs (1)
15-15: Consider usingserde::Deserializeinstead ofserde_deriveSince the Cargo manifest already enables the
derivefeature for serde (features = ["derive"]), it's more idiomatic to useuse serde::Deserializeinstead of importing fromserde_derivedirectly. This ensures better maintainability and consistency with serde's recommended usage patterns.-use serde_derive::Deserialize; +use serde::Deserialize;
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
⛔ Files ignored due to path filters (3)
tests/test_script_rs/expected-results/rust-manifest.outis excluded by!**/*.outtests/test_script_rs/expected-results/rust-outer-line-doc.outis excluded by!**/*.outtests/test_script_rs/expected-results/rust.outis excluded by!**/*.out
📒 Files selected for processing (11)
docs/snakefiles/rules.rst(2 hunks)snakemake/script/__init__.py(2 hunks)tests/test_script/Snakefile(4 hunks)tests/test_script_rs/Snakefile(1 hunks)tests/test_script_rs/config.yaml(1 hunks)tests/test_script_rs/envs/rust.yaml(1 hunks)tests/test_script_rs/scripts/test-manifest.rs(1 hunks)tests/test_script_rs/scripts/test-outer-line-doc.rs(1 hunks)tests/test_script_rs/scripts/test.py(1 hunks)tests/test_script_rs/test2.in(1 hunks)tests/tests.py(1 hunks)
✅ Files skipped from review due to trivial changes (2)
- tests/test_script_rs/config.yaml
- tests/test_script_rs/test2.in
🧰 Additional context used
📓 Path-based instructions (3)
snakemake/script/__init__.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
tests/test_script_rs/scripts/test.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
tests/tests.py (1)
Pattern **/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.
🪛 Ruff
tests/test_script_rs/scripts/test.py
1-1: Undefined name snakemake
(F821)
3-3: Undefined name snakemake
(F821)
🔇 Additional comments (19)
tests/test_script_rs/envs/rust.yaml (1)
5-5: LGTM! Verify package availability.
The version update to >=0.35.0 matches the PR objectives and addresses the CLI breaking change.
Let's verify the package availability in conda-forge:
tests/test_script_rs/scripts/test.py (1)
1-4: Verify test coverage for rust-script version update.
This test script appears to be a basic functionality test. Given that the PR objectives mention updating rust-script to v0.35.0 due to CLI breaking changes (specifically the --features option placement), we should verify if there are adequate test cases covering this specific change.
✅ Verification successful
Test coverage appears adequate for rust-script functionality
After examining the test files, I can see that the test suite includes comprehensive testing of rust-script functionality:
- Basic script execution and output verification (
test.rs) - Cargo manifest handling with dependencies and features (
test-manifest.rs) - Outer line documentation testing (
test-outer-line-doc.rs) - Various input/output assertions and parameter handling
While the --features flag placement isn't explicitly tested, the test suite includes Rust files that use feature-enabled dependencies (e.g., serde = { version = "1.0", features = ["derive"] }), which would implicitly test the feature flag handling through the manifest-based configuration.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check for test cases related to rust-script features flag usage
# Look for Snakefiles or test files that might contain rust-script feature flag tests
echo "Searching for rust-script feature flag usage in test files..."
rg -l --type python "rust-script.*--features" tests/
# Look for related test cases in the test suite
echo "Searching for related test cases..."
rg -l --type python "test.*rust.*script.*features" tests/
Length of output: 390
Script:
#!/bin/bash
# Let's try a different approach to search for rust-script related tests
# Search for rust-script usage in test files
echo "Searching for rust-script usage in tests..."
rg "rust-script" tests/
# Search for Snakefiles that might contain rust-script feature flags
echo -e "\nSearching in Snakefiles..."
fd Snakefile tests/ --exec rg "rust-script" {}
# Look for test files containing 'features' flag
echo -e "\nSearching for feature flag usage..."
rg "features" tests/
Length of output: 346
Script:
#!/bin/bash
# Let's check the PR changes and test directory structure
# First, let's see what files are in the tests directory
echo "Listing test directory structure:"
fd . tests/ --type f
# Check the specific test script directory mentioned in the review
echo -e "\nChecking test_script_rs directory content:"
ls -la tests/test_script_rs/
# Look for any Rust-related test files
echo -e "\nSearching for Rust-related test files:"
fd . tests/ | grep -i rust
Length of output: 835
Script:
#!/bin/bash
# Let's examine the Snakefile and scripts in the test_script_rs directory
# Check the Snakefile content
echo "Content of tests/test_script_rs/Snakefile:"
cat tests/test_script_rs/Snakefile
# Check all files in the scripts directory
echo -e "\nContent of scripts directory:"
ls -la tests/test_script_rs/scripts/
# Check content of any Rust files in scripts directory
echo -e "\nContent of Rust files in scripts directory:"
cat tests/test_script_rs/scripts/*.rs 2>/dev/null || echo "No .rs files found"
Length of output: 5438
🧰 Tools
🪛 Ruff
1-1: Undefined name snakemake
(F821)
3-3: Undefined name snakemake
(F821)
tests/test_script_rs/Snakefile (7)
1-4: LGTM: Clean imports and config setup
The import and configuration setup follow Snakemake best practices.
36-47: LGTM: Well-structured rule with proper logging
The rule follows best practices by including a log file.
#!/bin/bash
# Verify the manifest test script exists
if [ -f "tests/test_script_rs/scripts/test-manifest.rs" ]; then
echo "Manifest test script exists"
# Look for any feature flags that might need updating
rg "features?" "tests/test_script_rs/scripts/test-manifest.rs"
fi14-19: Verify the existence of test.py script
The rule is well-structured, but let's ensure the Python script exists.
#!/bin/bash
# Check if the Python script exists
if [ -f "tests/test_script_rs/scripts/test.py" ]; then
echo "Python script exists"
# Show script content for context
cat "tests/test_script_rs/scripts/test.py"
else
echo "Warning: Python script not found at expected location"
fi7-12: LGTM: Rule dependencies are properly defined
The all rule correctly aggregates outputs from all Rust-related rules.
✅ Verification successful
Rule dependencies are correctly defined and verified
The all rule's dependencies are properly defined. The Snakefile shows that:
rust.outis generated by thesimplerulerust-manifest.outis generated by themanifest_docrulerust-outer-line-doc.outis generated by themanifest_outer_line_docrule
All output files listed in the all rule have corresponding rules that generate them, confirming the original review comment's assessment.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Verify that all output files in the 'all' rule have corresponding rules that generate them
for file in "rust.out" "rust-manifest.out" "rust-outer-line-doc.out"; do
echo "Checking rule for $file:"
rg "output:.*$file" --context 2
done
Length of output: 470
Script:
#!/bin/bash
# Search for rules in Snakefile that generate the output files
echo "Searching in Snakefile:"
cat tests/test_script_rs/Snakefile
# Also check for any other Snakefiles in the repository
echo -e "\nSearching for other Snakefiles:"
fd Snakefile
Length of output: 1470
21-35: Verify rust-script version in conda environment
The rule structure is good, but we need to ensure the conda environment uses rust-script v0.35.0 as mentioned in the PR objectives.
✅ Verification successful
✓ Rust-script version requirement is correctly specified
The conda environment file rust.yaml already specifies rust-script>=0.35.0 as required, which aligns with the PR objectives. The environment configuration also includes necessary dependencies like openssl, c-compiler, and pkg-config.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Check rust-script version in conda environment file
echo "Checking rust-script version in conda environment:"
cat "tests/test_script_rs/envs/rust.yaml"
# Verify the test.rs script exists and check for any feature flags
echo -e "\nChecking Rust script and its feature usage:"
if [ -f "tests/test_script_rs/scripts/test.rs" ]; then
# Look for feature flags that might need updating based on PR objectives
rg "features?" "tests/test_script_rs/scripts/test.rs"
fi
Length of output: 470
Script:
#!/bin/bash
# Check if the conda environment file exists and its contents
echo "Checking for rust.yaml:"
fd "rust.yaml" tests/test_script_rs/envs/
echo -e "\nChecking contents of rust.yaml if found:"
if [ -f "tests/test_script_rs/envs/rust.yaml" ]; then
cat "tests/test_script_rs/envs/rust.yaml"
fi
# Check if the Rust script exists
echo -e "\nChecking for test.rs:"
fd "test.rs" tests/test_script_rs/scripts/
echo -e "\nChecking contents of test.rs if found:"
if [ -f "tests/test_script_rs/scripts/test.rs" ]; then
cat "tests/test_script_rs/scripts/test.rs"
fi
Length of output: 1303
48-56: Consider adding a log file for consistency
The rule is similar to manifest_doc but doesn't specify a log file. Consider adding one for consistency with the other rules.
rule manifest_outer_line_doc:
output:
"rust-outer-line-doc.out",
params:
keep="-",
conda:
"envs/rust.yaml"
+ log:
+ "rust-outer-line-doc.log"
script:
"scripts/test-outer-line-doc.rs"1-56: Verify feature flag usage in all Rust scripts
Given the PR objectives mention a breaking change in rust-script v0.35.0 regarding --features placement (now needs to be after --), we should verify all Rust scripts in this test suite for compatibility.
tests/test_script/Snakefile (1)
Line range hint 72-93: LGTM! Well-structured bash rule
This rule demonstrates good practices:
- Meaningful rule name
- Named inputs
- Comprehensive parameter structure
- Resource specification
- Log file configuration
tests/test_script_rs/scripts/test-outer-line-doc.rs (3)
15-15: LGTM: Import change follows best practices
The change to use serde_derive::Deserialize instead of serde::Deserialize aligns with modern Rust practices where derive macros are separated into their own crate.
Line range hint 1-11: Verify dependency versions for rust-script v0.35.0 compatibility
Given the update to rust-script v0.35.0, we should ensure the dependency versions in the manifest are compatible:
- csv = "1.1"
- serde = { version = "1.0", features = ["derive"] }
Let's check for any version compatibility issues:
#!/bin/bash
# Description: Search for dependency versions across rust scripts
# Expect: Consistent versions that work with rust-script v0.35.0
# Find all rust scripts with cargo manifests
rg -U "```cargo[\s\S]*?```" --type rustLine range hint 23-31: Well-structured BED format parsing implementation
The BedRecord struct is well-designed with:
- Appropriate use of
Option<T>for optional BED fields (name, score, strand) - Correct field types (u64 for positions, String for chromosome)
- Proper derive macros for debugging and deserialization
Let's verify the BED format field types match the specification:
✅ Verification successful
The previous script had a syntax error in the quoting. I've fixed that and added a ripgrep search to catch any BED-related structs that might have slightly different naming patterns. This will help us verify the consistency of field types across BED format implementations.
BED format field types are correctly implemented
The BedRecord struct implementation follows the standard BED format specification:
- Required fields (chrom, start, end) use appropriate types
- Optional fields (name, score, strand) are correctly wrapped in Option
- Field types match common usage: String for text, u64 for coordinates, u16 for score
Since no other BED format implementations were found in the codebase, this appears to be the sole implementation, making consistency concerns irrelevant.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Search for other BED format structs to ensure consistency
# Expect: Similar field types across other BED parsing implementations
ast-grep --pattern 'struct $name {
chrom: String,
start: u64,
end: u64,
$$$
}'
Length of output: 93
Script:
#!/bin/bash
# Search for BED format struct definitions with correct quoting
ast-grep --pattern $'struct $name {
chrom: String,
start: u64,
end: u64,
$$$
}'
# Also search for any other BED-related structs
rg "struct.*Bed.*\{" -A 6
Length of output: 116
tests/test_script_rs/scripts/test-manifest.rs (1)
18-22: LGTM: Good use of const for compile-time constant data
The change from static to const is a good improvement. Using const for immutable compile-time values is more idiomatic in Rust and can enable better optimizations.
snakemake/script/__init__.py (2)
1297-1297: LGTM: Command format updated to match rust-script v0.35.0 requirements.
The command format has been correctly updated to place --features after --, which aligns with rust-script's new CLI requirements where feature flags are forwarded to cargo.
1316-1318: LGTM: More precise version specification for serde dependencies.
The version numbers for serde dependencies have been updated to use more explicit version specifications ("1.0" instead of "1"), which follows semantic versioning conventions better.
Let's verify these are the latest stable versions:
✅ Verification successful
Version "1.0" is compatible with latest serde crates
The specified version requirement "1.0" in the code is appropriate as it:
- Follows semantic versioning and allows compatibility with all 1.0.x releases
- Matches the major version of current stable releases (serde 1.0.215, serde_json 1.0.132, serde_derive 1.0.215)
- Provides flexibility for minor version updates while maintaining compatibility
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Description: Check latest versions of serde dependencies
# Test: Expect version numbers to be compatible with our requirements
for pkg in serde serde_json serde_derive; do
echo "Checking $pkg..."
curl -s "https://crates.io/api/v1/crates/$pkg" | jq -r '.crate.max_stable_version'
done
Length of output: 569
tests/tests.py (1)
416-424: LGTM! Well-structured test function for rust-script.
The new test function is properly implemented with appropriate decorators and follows the established pattern for script-related tests in the codebase. The implementation aligns well with the PR's objective of updating rust-script usage.
docs/snakefiles/rules.rst (2)
1383-1385: LGTM! Dependencies are properly documented.
The addition of explicit serde dependencies required by json_typegen is well documented and appropriate.
1298-1306: LGTM! Version requirement update is well documented.
The update to rust-script version 0.35.0 and addition of required dependencies is correct and aligns with the PR objectives to fix test suite issues.
Let's verify no other files still reference older versions:
✅ Verification successful
Version requirement update is consistent across the codebase
The verification confirms that all references to rust-script version in the codebase consistently specify version 0.35.0, and no older versions were found. The documentation update is accurate and complete.
🏁 Scripts executed
The following scripts were executed for the analysis:
Script:
#!/bin/bash
# Search for any references to older rust-script versions
rg -i "rust-script.*0\.[0-2][0-9]"
Length of output: 36
Script:
#!/bin/bash
# Let's try a broader search for rust-script version references
rg -i "rust-script.*[0-9]+\.[0-9]+"
# Also search for any rust-script related configuration files
fd -e yaml -e yml -x grep -l "rust-script" {}
Length of output: 298
|
tedil
left a comment
There was a problem hiding this comment.
LGTM, thank you very much!
🤖 I have created a release *beep* *boop* --- ## [8.25.4](v8.25.3...v8.25.4) (2024-11-27) ### Bug Fixes * clean env vars in apptainer ([#3199](#3199)) ([76d5329](76d5329)) * ensure that intermediate files in job groups do not cause spurious mtime errors when checking for consistency with output files ([#3220](#3220)) ([4ba2bdf](4ba2bdf)) * Remove incomplete marker also when drop-metadata is active ([#3215](#3215)) ([a4f2e5c](a4f2e5c)) * Remove incomplete marker for job finished only after metadata is written ([#3197](#3197)) ([6567e5f](6567e5f)) * Support versioned URLs in Asset class and fix missing versions in Snakemake report ([#3203](#3203)) ([f086f6c](f086f6c)) * update rust-script usage to recent version (v0.35.0) [#3183](#3183) ([#3208](#3208)) ([43885d7](43885d7)) ### Documentation * clarify continuously updated input section ([#3219](#3219)) ([72a6994](72a6994)) * Fix typo in CHANGELOG.md ([#3198](#3198)) ([0e445ed](0e445ed)) * refer to Merkle trees instead of "blockchain" in caching.rst ([#3216](#3216)) ([282e5d9](282e5d9)) * remove twitter in favor of bluesky and mastodon ([#3217](#3217)) ([231c6df](231c6df)) * use "dictionary" not "array" wording in config docs ([#3156](#3156)) ([17aed41](17aed41)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please). Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>



closes #3183
rust-script's conda recipe was very out of date. As such, the test suite didn't pick up a breaking CLI change (which isn't even in their changelog annoyingly).
--featuresnow needs to be passed after--as it is passed on tocargo.I also moved the rust script tests into their own dedicated test suite.
QC
docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).Summary by CodeRabbit
New Features
Bug Fixes
Documentation
Tests
test_script_rsfunctionality.