Skip to content

registry: add s5cmd#7376

Merged
jdx merged 11 commits intomainfrom
registry/s5cmd
Dec 18, 2025
Merged

registry: add s5cmd#7376
jdx merged 11 commits intomainfrom
registry/s5cmd

Conversation

@jdx
Copy link
Owner

@jdx jdx commented Dec 18, 2025

Summary

  • Add s5cmd to the registry using the aqua backend (peak/s5cmd)
  • s5cmd is a parallel S3 and local filesystem execution tool

Test plan

  • Verify mise install s5cmd works
  • Verify s5cmd version outputs version correctly

🤖 Generated with Claude Code


Note

Adds s5cmd to registry.toml with aqua backend (peak/s5cmd) and a version test.

Written by Cursor Bugbot for commit 32ab489. This will update automatically on new commits. Configure here.

claude and others added 10 commits December 17, 2025 21:59
Add a new `AssetMatcher` module that provides a high-level, builder-style
API for selecting the best asset from a list of available downloads.

Features:
- **Auto-detection**: Uses platform heuristics (OS, arch, libc) to score
  and rank assets, leveraging the existing AssetPicker
- **Pattern matching**: Support for explicit patterns with placeholders
  like `{os}`, `{arch}`, `{version}`, `{ext}`
- **Filtering**: Custom predicates and exclusion lists for fine-grained
  asset selection
- **Related assets**: Methods to find checksum and signature files for
  a given asset

Example usage:
```rust
// Auto-detect best asset for current platform
let asset = AssetMatcher::new()
    .for_current_platform()
    .pick_from(&assets)?;

// Match using explicit pattern
let asset = AssetMatcher::new()
    .with_pattern("tool-{version}-{os}-{arch}.tar.gz")
    .with_version("1.0.0")
    .pick_from(&assets)?;

// Find checksum file for an asset
let checksum = AssetMatcher::new()
    .find_checksum_for("tool-1.0.0-linux-x64.tar.gz", &assets);
```

This provides a unified interface that backends can use instead of
implementing their own asset matching logic. The existing AssetPicker
in asset_detector.rs remains unchanged and is used internally.

Includes 11 comprehensive tests covering:
- Auto-detection for various platforms
- Pattern matching with version substitution
- Checksum and signature file discovery
- Custom filtering and exclusion
- Real-world assets (ripgrep)

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Update the GitHub/GitLab backend to use the new AssetMatcher module
instead of the lower-level asset_detector::detect_asset_for_target.

This demonstrates the adoption of the unified AssetMatcher API and
validates that it's a drop-in replacement for the existing functionality.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ab backends

Add ChecksumFetcher to asset_matcher.rs that can:
- Find checksum files (SHA256SUMS, *.sha256, etc.) from release assets
- Fetch and parse checksum file contents
- Extract checksums for specific asset files
- Support both SHASUMS format and single-file checksum formats

Integrate checksum fetching in UnifiedGitBackend:
- GitHub: Falls back to fetching from checksum files when API digest unavailable
- GitLab: Always tries to fetch checksums (GitLab API doesn't provide digests)

This provides real value by automatically discovering and using checksums
from release assets, improving security verification without manual config.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Consolidate platform detection and asset matching into a single module:
- Move AssetPicker, platform detection types (AssetOs, AssetArch, AssetLibc),
  and detect_platform_from_url into asset_matcher.rs
- Update tool_stub.rs to import from asset_matcher
- Delete asset_detector.rs
- Add tests from asset_detector to asset_matcher (now 25 tests total)

This simplifies the module structure by having all asset-related
functionality in one place.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
…ettings

Bug fixes for AssetMatcher:

1. target_libc setting was ignored: create_picker() only passed os/arch
   to AssetPicker::new(), which re-derived libc from compile-time flags.
   Now AssetPicker::with_libc() accepts optional libc parameter.

2. prefer_archive and allow_binary had no effect: These fields were stored
   but never used in matching logic. Now filter_assets() respects them:
   - allow_binary=false filters out non-archive files
   - prefer_archive=true returns only archives when archives exist

Added 4 tests verifying these settings work correctly.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
1. Global checksum search used flawed OR condition:
   The contains("checksum") check was OR'd with pattern match, causing
   any file with "checksum" in name to be tried on every iteration.
   Now: exact pattern match first, then "checksum" files as last resort.

2. Checksum parsing returned wrong hash for missing files:
   When target file wasn't found in SHASUMS, code fell through and
   returned first hash (wrong file). Now returns None instead.

3. min_score setting was ignored by pick_from():
   match_by_auto_detection() never checked min_score threshold.
   Now rejects assets scoring below the minimum.

Added 3 tests and fixed existing test to verify correct behavior.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings December 18, 2025 00:56
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR refactors the asset detection and matching logic by renaming asset_detector to asset_matcher and consolidating functionality. The change also adds the s5cmd tool to the registry using the aqua backend.

Key changes:

  • Renamed asset_detector module to asset_matcher with expanded functionality including checksum fetching
  • Updated imports across affected files to use the new module name
  • Added s5cmd tool entry to the registry with appropriate test configuration

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
src/cli/generate/tool_stub.rs Updated import to reference renamed asset_matcher module
src/backend/mod.rs Changed module declaration from asset_detector to asset_matcher
src/backend/github.rs Updated imports and added checksum fetching logic for GitHub/GitLab releases
src/backend/asset_matcher.rs New consolidated module combining asset detection, matching, and checksum fetching
src/backend/asset_detector.rs Removed file (renamed to asset_matcher)
registry.toml Added s5cmd tool configuration with aqua backend

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

pub struct DetectedPlatform {
pub os: AssetOs,
pub arch: AssetArch,
#[allow(unused)]
Copy link

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The libc field in DetectedPlatform is marked as unused with #[allow(unused)], but it is actually used in the code (e.g., in tests at line 1505, 1527). This annotation should be removed since the field has legitimate uses.

Suggested change
#[allow(unused)]

Copilot uses AI. Check for mistakes.
Comment on lines +1070 to +1071
// Strip leading * or . from filename if present (some formats use this)
let clean_filename = filename.trim_start_matches(['*', '.']);
Copy link

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The trim_start_matches call trims leading '' and '.' characters from filenames. However, this could incorrectly match files that start with dots (like .hidden_file). Consider only trimming '' which is the standard checksum file prefix, or add validation to ensure meaningful filename matching.

Suggested change
// Strip leading * or . from filename if present (some formats use this)
let clean_filename = filename.trim_start_matches(['*', '.']);
// Strip leading '*' from filename if present (standard checksum prefix)
let clean_filename = filename.trim_start_matches('*');

Copilot uses AI. Check for mistakes.
registry.toml Outdated
[tools.s5cmd]
backends = ["aqua:peak/s5cmd"]
description = "Parallel S3 and local filesystem execution tool"
test = ["s5cmd version", "s5cmd version {{version}}"]
Copy link

Copilot AI Dec 18, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test command appears to run 's5cmd version' twice with different expected outputs. The first element should likely be the command to execute, and the second should be the expected output pattern. Based on the pattern in other entries, this should probably be test = [\"s5cmd version\", \"{{version}}\"] to match just the version number.

Suggested change
test = ["s5cmd version", "s5cmd version {{version}}"]
test = ["s5cmd version", "{{version}}"]

Copilot uses AI. Check for mistakes.
@github-actions
Copy link

github-actions bot commented Dec 18, 2025

Hyperfine Performance

mise x -- echo

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2025.12.10 x -- echo 20.1 ± 0.6 19.5 26.5 1.00
mise x -- echo 20.4 ± 0.9 19.5 38.7 1.01 ± 0.05

mise env

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2025.12.10 env 19.5 ± 0.2 18.8 20.5 1.00
mise env 19.9 ± 0.7 18.9 25.8 1.02 ± 0.04

mise hook-env

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2025.12.10 hook-env 19.8 ± 0.3 19.0 21.7 1.00
mise hook-env 20.0 ± 0.4 19.1 23.1 1.01 ± 0.03

mise ls

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2025.12.10 ls 17.2 ± 0.3 16.5 21.1 1.00
mise ls 17.6 ± 0.7 17.0 29.7 1.02 ± 0.05

xtasks/test/perf

Command mise-2025.12.10 mise Variance
install (cached) 109ms 109ms +0%
ls (cached) 66ms 67ms -1%
bin-paths (cached) 73ms 73ms +0%
task-ls (cached) 2261ms ✅ 280ms +707%

✅ Performance improvement: task-ls cached is 707%

# Conflicts:
#	src/backend/asset_matcher.rs
[tools.s5cmd]
backends = ["aqua:peak/s5cmd"]
description = "Parallel S3 and local filesystem execution tool"
test = ["s5cmd version", "v{{version}}"]
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Tool entry placed in wrong alphabetical position

The s5cmd tool entry is placed between scorecard and sd, but in the lexicographical ordering used by this file, digits come before letters. Since '5' (ASCII 53) precedes 'a' (ASCII 97), s5cmd should be positioned after rye and before saml2aws, not in its current location. Other tools like 1password, k0sctl, k3d, and e1s follow this digits-before-letters ordering convention throughout the file.

Fix in Cursor Fix in Web

@jdx jdx merged commit ba919aa into main Dec 18, 2025
32 of 35 checks passed
@jdx jdx deleted the registry/s5cmd branch December 18, 2025 02:55
jdx pushed a commit that referenced this pull request Dec 18, 2025
### 🚀 Features

- **(alias)** rename alias to tool-alias, add shell-alias command by
@jdx in [#7357](#7357)
- **(upgrade)** display summary of upgraded tools by @jdx in
[#7372](#7372)
- **(vfox)** embed vfox plugin Lua code in binary by @jdx in
[#7369](#7369)

### 🐛 Bug Fixes

- **(aqua)** add start_operations for progress reporting by @jdx in
[#7354](#7354)
- **(github)** improve asset detection for distro-specific and Swift
artifacts by @jdx in [#7347](#7347)
- **(github)** clean up static_helpers.rs and fix archive bin= option by
@jdx in [#7366](#7366)
- **(http)** add start_operations for progress reporting by @jdx in
[#7355](#7355)
- **(lockfile)** place lockfile alongside config file by @jdx in
[#7360](#7360)
- **(progress)** add start_operations to core plugins by @jdx in
[#7351](#7351)
- **(ruby-install)** Use ruby_install_bin to update by @calebhearth in
[#7350](#7350)
- **(rust)** add release_url for rust versions by @jdx in
[#7373](#7373)
- **(schema)** add `tool_alias`, mark `alias` as deprecated by @SKalt in
[#7358](#7358)
- **(toolset)** filter tools by OS in list_current_versions by @jdx in
[#7356](#7356)
- **(ubi)** only show deprecation warning during installation by @jdx in
[#7380](#7380)
- **(ui)** remove noisy "record size" message during install by @jdx in
[#7381](#7381)
- update mise-versions URL to use /tools/ prefix by @jdx in
[#7378](#7378)

### 🚜 Refactor

- **(backend)** unified AssetMatcher with checksum fetching by @jdx in
[#7370](#7370)
- **(backend)** deprecate ubi backend in favor of github by @jdx in
[#7374](#7374)
- **(toolset)** decompose mod.rs into smaller modules by @jdx in
[#7371](#7371)

### 🧪 Testing

- **(e2e)** fix and rename ubi and vfox_embedded_override tests by @jdx
in
[052ea40](052ea40)

### 📦 Registry

- add vfox-gcloud backend for gcloud by @jdx in
[#7349](#7349)
- convert amplify to use github backend by @jdx in
[#7365](#7365)
- add github backend for djinni tool by @jdx in
[#7363](#7363)
- switch glab to native gitlab backend by @jdx in
[#7364](#7364)
- add s5cmd by @jdx in [#7376](#7376)

### Chore

- **(registry)** disable flaky tests for gitu and ktlint by @jdx in
[64151cb](64151cb)
- resolve clippy warnings and add stricter CI check by @jdx in
[#7367](#7367)
- suppress dead_code warnings in asset_matcher module by @jdx in
[#7377](#7377)

### New Contributors

- @calebhearth made their first contribution in
[#7350](#7350)
jekis913 added a commit to jekis913/mise that referenced this pull request Dec 18, 2025
* upstream/main:
  fix(ui): remove noisy "record size" message during install (jdx#7381)
  test(e2e): fix and rename ubi and vfox_embedded_override tests
  fix: update mise-versions URL to use /tools/ prefix (jdx#7378)
  fix(ubi): only show deprecation warning during installation (jdx#7380)
  registry: add s5cmd (jdx#7376)
  chore: suppress dead_code warnings in asset_matcher module (jdx#7377)
  refactor(backend): deprecate ubi backend in favor of github (jdx#7374)
  fix(rust): add release_url for rust versions (jdx#7373)
  feat(vfox): embed vfox plugin Lua code in binary (jdx#7369)
  refactor(backend): unified AssetMatcher with checksum fetching (jdx#7370)
  feat(upgrade): display summary of upgraded tools (jdx#7372)
  fix(github): clean up static_helpers.rs and fix archive bin= option (jdx#7366)
  refactor(toolset): decompose mod.rs into smaller modules (jdx#7371)
  chore: resolve clippy warnings and add stricter CI check (jdx#7367)
  registry: switch glab to native gitlab backend (jdx#7364)
  fix(ruby-install): Use ruby_install_bin to update (jdx#7350)
  registry: add github backend for djinni tool (jdx#7363)
  registry: convert amplify to use github backend (jdx#7365)
  chore(registry): disable flaky tests for gitu and ktlint
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants