feat(conda): add dependency resolution for conda packages by jdx · Pull Request #7280 · jdx/mise

jdx · 2025-12-13T20:34:16Z

Summary

Add recursive dependency resolution from anaconda.org API
Download all packages in parallel using parallel::parallel
Use shared download cache at ~/.local/share/mise/downloads/conda-packages/
Add file locking for parallel installation safety
Skip virtual packages (__osx, __glibc) and runtime deps (python, ruby)
Make conda the default backend for clang in registry.toml
Add e2e test for jq (has oniguruma dependency)
Add Windows e2e test for clang

How it works

When installing a conda package:

Fetch main package info from anaconda.org API
Parse attrs.depends to get dependency list
Recursively resolve all dependencies (skip virtual packages and runtime deps)
Download all packages in parallel to shared cache
Extract dependencies first, then main package (so main takes precedence)

Parallel installations of different tools with shared deps work correctly via file locking.

Test plan

Local test: mise x conda:jq@1.7.1 -- jq --version (jq depends on oniguruma)
CI e2e test added for jq with dependencies
Windows e2e test for clang

🤖 Generated with Claude Code

Note

Implements recursive dependency resolution and parallel downloads for the conda backend with a shared cache, streamlines error output, gates experimental backends, updates registry (clang via conda), and adds e2e tests.

Conda backend:
- Recursive dependency resolution via anaconda API (attrs.depends), skipping virtual/runtime/system deps.
- Parallel package downloads with parallel::parallel to shared cache at data/conda-packages/ with checksum verification.
- Improved version matching (wildcards, ranges) and platform selection; prefers .conda over .tar.bz2.
- Robust extraction using temp dirs; extract deps first, then main; sets executable bits; Windows Library/bin handling (list_bin_paths).
- Lockfile URLs/checksums now point to main package.
Experimental backends:
- Mark conda, spm, dotnet as experimental and filter them when experimental mode is disabled (BackendType::is_experimental, registry.rs).
Error output:
- Always print full error chains using {:#}; remove RUST_BACKTRACE gating; update failure summaries.
Parallel runtime:
- Safer task cancellation handling in parallel::parallel (drain remaining tasks).
Registry:
- Add conda:clang backend in registry.toml.
E2E tests:
- Add e2e/backend/test_conda (incl. jq with dependency) and Windows ripgrep test.
- Update error display expectations for friendly/detailed modes.

^{Written by Cursor Bugbot for commit 04190f5. This will update automatically on new commits. Configure here.}

- Add recursive dependency resolution from anaconda.org API - Download all packages in parallel using parallel::parallel - Use shared download cache at ~/.local/share/mise/downloads/conda-packages/ - Add file locking for parallel installation safety - Skip virtual packages (__osx, __glibc) and runtime deps (python, ruby) - Make conda the default backend for clang in registry.toml - Add e2e test for jq (has oniguruma dependency) - Add Windows e2e test for clang 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Copilot

Pull request overview

This PR adds recursive dependency resolution for conda packages, enabling automatic installation of transitive dependencies. The main changes enhance the conda backend to resolve and download package dependencies in parallel, use a shared download cache, and implement file locking for parallel installation safety. Additionally, the PR modifies task argument handling to ensure flags after task names are passed to tasks rather than being parsed by mise.

Key changes:

Recursive dependency resolution for conda packages with parallel downloads
Shared download cache at ~/.local/share/mise/downloads/conda-packages/ with file locking
Task flag routing improvements to pass flags after task names to tasks

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
src/backend/conda.rs	Implements recursive dependency resolution, parallel package downloads, and shared cache management
src/task/mod.rs	Re-exports `has_any_args_defined` for use in other modules
src/cli/run.rs	Updates help flag handling to check if tasks have usage specs before showing mise help
src/cli/mod.rs	Adds flag escaping logic to pass flags after task names to tasks instead of mise
registry.toml	Makes conda the default backend for clang
e2e/tasks/test_task_skip_deps	Updates test commands to place `--skip-deps` before task name
e2e/tasks/test_task_remote_git_https	Updates test commands to place `--no-cache` before task name
e2e/tasks/test_task_help_with_cd	Updates test to verify `--help` is passed through to tasks without usage specs
e2e/tasks/test_task_help	Updates test expectations for `--help` behavior on tasks without usage specs
e2e/tasks/test_task_double_dash_behavior	Updates test expectations to reflect new flag routing behavior
e2e/tasks/test_task_args_position	New test file validating flag routing behavior
e2e/backend/test_conda	Adds test for jq installation with dependencies
e2e-win/clang.Tests.ps1	New Windows test for clang installation via conda

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-13T20:35:06Z

+const SKIP_PACKAGES: &[&str] = &[
+    "python",
+    "python_abi",
+    "ruby",
+    "perl",
+    "r-base",
+    "libgcc-ng",    // Linux-specific, provided by system
+    "libstdcxx-ng", // Linux-specific, provided by system
+];


The constant SKIP_PACKAGES lacks documentation explaining why these specific packages are skipped. Consider adding a doc comment explaining the rationale (e.g., 'Runtime interpreters and system libraries that are typically not needed for standalone tools or are provided by the host system').

Copilot · 2025-12-13T20:35:06Z

+/// Parse a conda dependency specification
+/// Returns (package_name, optional_version_spec) or None if should be skipped


The function documentation doesn't mention that it filters out virtual packages (those starting with '__'). Update the doc comment to include: 'Filters out virtual packages (starting with __) and packages in SKIP_PACKAGES'.

Suggested change

/// Parse a conda dependency specification

/// Returns (package_name, optional_version_spec) or None if should be skipped

/// Parse a conda dependency specification.

/// Returns (package_name, optional_version_spec) or None if should be skipped.

/// Filters out virtual packages (starting with `__`) and packages in `SKIP_PACKAGES`.

Copilot · 2025-12-13T20:35:06Z

        Ok(())
    }
+
+    /// Download a conda package to shared cache (standalone for parallel::parallel)


The comment mentions '(standalone for parallel::parallel)' but doesn't explain what this means or why it matters. Consider clarifying: 'Standalone function (not using &self) so it can be called in parallel via parallel::parallel'.

Suggested change

/// Download a conda package to shared cache (standalone for parallel::parallel)

/// Standalone function (not using &self) so it can be called in parallel via parallel::parallel.

/// Downloads a conda package to shared cache.

cursor · 2025-12-13T20:42:47Z

+                    .copied()
+                    .copied();
+            }
+        }


Bug: Version constraint parsing ignores upper bound constraints

The parse_dependency function extracts version specs like >=1.0,<2.0 as raw strings, but find_matching_package only checks for exact equality or prefix matches against package versions. Since "1.25.0" == ">=1.0,<2.0" and "1.25.0".starts_with(">=1.0,<2.0") are both false, the code falls back to selecting the latest available version. For dependencies with upper bound constraints (e.g., <2.0), this could install incompatible versions like 3.0, potentially causing runtime failures.

Additional Locations (1)

src/backend/conda.rs#L584-L586

cursor

Bug: Shared temp directory causes race condition during extraction

The extract_conda_format function creates a temp directory using a fixed path conda_path.parent().unwrap().join("conda_extract_temp"). Since packages are now stored in a shared cache directory, this temp directory is shared by all concurrent extractions. When multiple mise processes install different conda tools simultaneously with .conda format dependencies, they would all use the same temp directory. This can cause extraction failures (when one process removes the temp directory while another is using it) or potentially extract the wrong package contents (when different packages' inner tarballs get mixed up in the shared temp directory).

src/backend/conda.rs#L159-L160

mise/src/backend/conda.rs

Lines 159 to 160 in 48bd078

    
           // Create a temp directory for extraction 
        
           let temp_dir = conda_path.parent().unwrap().join("conda_extract_temp");

- Add platform_to_conda_subdir() helper to eliminate duplicate platform mapping - Unify fetch_package_files to delegate to fetch_package_files_for - Unify find_package_file to handle both exact version and latest version cases - Add version_matches() to support conda version specs (wildcards, comparisons) - Add to_resolved_package() method to CondaPackageFile for cleaner code - Change package storage from dirs::DOWNLOADS to dirs::DATA - Rename conda_cache_dir -> conda_data_dir, package_cache_path -> package_path - Fix version matching for wildcard patterns like "6.9.*" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

github-actions · 2025-12-13T21:15:56Z

Hyperfine Performance

`mise x -- echo`

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`mise-2025.12.7 x -- echo`	20.4 ± 0.4	19.7	23.5	1.01 ± 0.04
`mise x -- echo`	20.1 ± 0.8	19.2	23.3	1.00

`mise env`

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`mise-2025.12.7 env`	20.0 ± 0.7	19.3	26.2	1.04 ± 0.04
`mise env`	19.3 ± 0.3	18.6	21.0	1.00

`mise hook-env`

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`mise-2025.12.7 hook-env`	20.1 ± 0.3	19.4	21.5	1.01 ± 0.03
`mise hook-env`	19.9 ± 0.5	18.9	22.0	1.00

`mise ls`

Command	Mean [ms]	Min [ms]	Max [ms]	Relative
`mise-2025.12.7 ls`	17.1 ± 0.3	16.5	18.4	1.00
`mise ls`	17.4 ± 0.4	16.6	18.8	1.02 ± 0.03

`xtasks/test/perf`

Command	mise-2025.12.7	mise	Variance
install (cached)	108ms	109ms	+0%
ls (cached)	65ms	66ms	-1%
bin-paths (cached)	72ms	72ms	+0%
task-ls (cached)	439ms	⚠️ 2260ms	-80%

⚠️ Warning: task-ls cached performance variance is -80%

- Add is_experimental() to BackendType for conda, spm, dotnet - Filter experimental backends when experimental mode is disabled This allows tools like clang to fall back to asdf when not using experimental mode - Remove unused version field from ResolvedPackage - Fix failed dependency resolution to throw errors instead of silently skipping - Fix shared temp directory race condition in extract_conda_format Uses tempfile::tempdir_in() for unique temp dirs per extraction - Fix version constraint parsing to properly handle comparison operators Now supports >=, <=, >, <, ==, != and compound specs like ">=1.0,<2.0" 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Each experimental backend (conda, spm, dotnet) now defines its own `pub const EXPERIMENTAL: bool = true;` constant. BackendType::is_experimental() references these module constants instead of maintaining a centralized list. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Split find_package_file into find_package_file + find_package_file_for_subdir to properly prefer platform-specific packages before falling back to noarch - Remove redundant or_else fallbacks since find_package_file now handles noarch fallback internally - Add Windows system packages to SKIP_PACKAGES (ucrt, vc, vc14_runtime, vs2015_runtime) to fix Windows e2e test failures 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Some conda packages have platform-specific dependencies that aren't available on all platforms (e.g., compiler-rt21 isn't available on win-64). Instead of failing when a dependency isn't found, skip it with a debug message. This fixes the Windows e2e test for clang via conda backend. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

…lution The main package name was not added to the visited set before calling resolve_dependencies. If a dependency circularly depended back on the main package, it would be added to resolved, causing it to appear twice in all_packages (once from resolved.values() and once from the explicit push). 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Windows conda packages put native binaries in Library/bin instead of bin. Override list_bin_paths to return the correct path on each platform. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The clang conda package on Windows has issues with missing compiler-rt21 dependency and produces incomplete version output. Switch to ripgrep which is a simpler tool with better Windows support in conda-forge. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Add better error context to download_package to show which package and URL failed. Also improve parallel module to provide clearer error messages when tasks are cancelled or panic. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Wrap LockFile::lock() in spawn_blocking to avoid blocking the async runtime, which was causing task cancellation errors on Windows - Use Library/bin for binaries on Windows in install_version_ to match the path logic in list_bin_paths 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

join_all() panics if any task was cancelled. After calling abort_all(), all remaining tasks are cancelled, so join_all() would panic. Replace with a join_next() loop that properly drains cancelled tasks. This was causing Windows e2e test failures with "task N was cancelled" panic messages. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

If a file exists but checksum verification fails (e.g., from a previous incomplete/interrupted download), delete the corrupted file and re-download instead of failing immediately. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

- Download to a unique temp file (with PID suffix) - Verify checksum of temp file before renaming to final path - Remove the spawn_blocking lock which wasn't working on Windows - Show full error chain in install failures using {:#} format - Include file size in checksum error message for debugging This ensures the final path never contains a corrupted file and makes debugging checksum failures easier by showing expected vs actual hashes. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

The anaconda.org API sometimes returns "" instead of null for sha256, causing checksum verification to fail with an empty expected value. Filter out empty strings when converting to ResolvedPackage. 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

Updated the test expectations to match the new error formatting that uses {:#} to show full error chains: - Friendly errors: single-line format with : separators - Detailed errors: eyre format with 0: prefix but includes full chain

cursor · 2025-12-15T16:56:39Z

+            && let Some(s) = Versioning::new(spec_ver)
+        {
+            return v < s;
+        }


Bug: Comparison operators with wildcards fail silently in version matching

Version specs combining comparison operators with wildcards like >=1.*, <2.*, or >=1.0,<2.* fail to match any versions because Versioning::new() cannot parse version strings containing wildcards (e.g., "2.*" returns None). When check_version_constraint is called with such a spec, the constraint check falls through to return false, causing no versions to match. This silently skips the dependency via the debug log at line 381, potentially leading to missing runtime dependencies for packages that use these spec formats.

@jdx

### 🚀 Features - **(conda)** add dependency resolution for conda packages by @jdx in [#7280](#7280) - **(go)** add created_at support to ls-remote --json by @jdx in [#7305](#7305) - **(hook-env)** add hook_env.cache_ttl and hook_env.chpwd_only settings for NFS optimization by @jdx in [#7312](#7312) - **(hooks)** add MISE_TOOL_NAME and MISE_TOOL_VERSION to preinstall/postinstall hooks by @jdx in [#7311](#7311) - **(shell_alias)** add shell_alias support for cross-shell aliases by @jdx in [#7316](#7316) - **(tool)** add security field to mise tool --json by @jdx in [#7303](#7303) - add --before flag for date-based version filtering by @jdx in [#7298](#7298) ### 🐛 Bug Fixes - **(aqua)** support cosign v3 bundle verification by @jdx in [#7314](#7314) - **(config)** use correct config_root in tera context for hooks by @jdx in [#7309](#7309) - **(nu)** fix nushell deactivation script on Windows by @fu050409 in [#7213](#7213) - **(python)** apply uv_venv_create_args in auto-venv code path by @jdx in [#7310](#7310) - **(shell)** escape exe path in activation scripts for paths with spaces by @jdx in [#7315](#7315) - **(task)** parallelize exec_env loading to fix parallel task execution by @jdx in [#7313](#7313) - track downloads for python and java by @jdx in [#7304](#7304) - include full tool ID in download track by @jdx in [#7320](#7320) ### 📚 Documentation - Switch `postinstall` code to be shell-agnostic by @thejcannon in [#7317](#7317) ### 🧪 Testing - **(e2e)** disable debug mode by default for windows-e2e by @jdx in [#7318](#7318) ### New Contributors - @fu050409 made their first contribution in [#7213](#7213)

Copilot AI review requested due to automatic review settings December 13, 2025 20:34

jdx force-pushed the feat/conda-dependency-resolution branch from 9650b2d to 65553da Compare December 13, 2025 20:34

Copilot AI reviewed Dec 13, 2025

View reviewed changes

[autofix.ci] apply automated fixes

48bd078

cursor Bot reviewed Dec 13, 2025

View reviewed changes

jdx and others added 3 commits December 13, 2025 14:45

[autofix.ci] apply automated fixes

fc0d64e

[autofix.ci] apply automated fixes (attempt 2/3)

adb4d97

jdx and others added 3 commits December 13, 2025 15:27

[autofix.ci] apply automated fixes

0b11f0d

cursor Bot reviewed Dec 13, 2025

View reviewed changes

Comment thread src/backend/conda.rs Outdated

jdx and others added 2 commits December 13, 2025 15:56

cursor Bot reviewed Dec 15, 2025

View reviewed changes

Comment thread src/backend/conda.rs

jdx enabled auto-merge (squash) December 15, 2025 13:15

jdx and others added 3 commits December 15, 2025 07:29

cursor Bot reviewed Dec 15, 2025

View reviewed changes

Comment thread src/backend/conda.rs Outdated

jdx and others added 6 commits December 15, 2025 09:19

jdx added 2 commits December 15, 2025 10:43

Merge origin/main into feat/conda-dependency-resolution

dd5e62a

chore: remove unused RUST_BACKTRACE static

04190f5

cursor Bot reviewed Dec 15, 2025

View reviewed changes

jdx merged commit 78587ad into main Dec 15, 2025
49 of 50 checks passed

jdx deleted the feat/conda-dependency-resolution branch December 15, 2025 17:26

mise-en-dev mentioned this pull request Dec 15, 2025

chore: release 2025.12.8 #7301

Merged

BrewTestBot mentioned this pull request Dec 15, 2025

mise 2025.12.8 Homebrew/homebrew-core#258752

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(conda): add dependency resolution for conda packages#7280

feat(conda): add dependency resolution for conda packages#7280
jdx merged 22 commits intomainfrom
feat/conda-dependency-resolution

jdx commented Dec 13, 2025 •

edited by cursor Bot

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 13, 2025

Uh oh!

Copilot AI Dec 13, 2025

Uh oh!

Copilot AI Dec 13, 2025

Uh oh!

cursor Bot Dec 13, 2025

Uh oh!

cursor Bot left a comment

Uh oh!

github-actions Bot commented Dec 13, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot Dec 15, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

		/// Parse a conda dependency specification
		/// Returns (package_name, optional_version_spec) or None if should be skipped

-/// Parse a conda dependency specification
-/// Returns (package_name, optional_version_spec) or None if should be skipped
+/// Parse a conda dependency specification.
+/// Returns (package_name, optional_version_spec) or None if should be skipped.
+/// Filters out virtual packages (starting with `__`) and packages in `SKIP_PACKAGES`.

	/// Download a conda package to shared cache (standalone for parallel::parallel)
	/// Standalone function (not using &self) so it can be called in parallel via parallel::parallel.
	/// Downloads a conda package to shared cache.

	// Create a temp directory for extraction
	let temp_dir = conda_path.parent().unwrap().join("conda_extract_temp");

Uh oh!

Conversation

jdx commented Dec 13, 2025 • edited by cursor Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

How it works

Test plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 13, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 13, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 13, 2025

Choose a reason for hiding this comment

Uh oh!

cursor Bot Dec 13, 2025

Choose a reason for hiding this comment

Bug: Version constraint parsing ignores upper bound constraints

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Bug: Shared temp directory causes race condition during extraction

Uh oh!

github-actions Bot commented Dec 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Hyperfine Performance

mise x -- echo

mise env

mise hook-env

mise ls

xtasks/test/perf

Uh oh!

Uh oh!

Uh oh!

Uh oh!

cursor Bot Dec 15, 2025

Choose a reason for hiding this comment

Bug: Comparison operators with wildcards fail silently in version matching

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

jdx commented Dec 13, 2025 •

edited by cursor Bot

Loading

github-actions Bot commented Dec 13, 2025 •

edited

Loading

`mise x -- echo`

`mise env`

`mise hook-env`

`mise ls`

`xtasks/test/perf`