fix(conda): deduplicate repodata records to fix solver error on Linux#8337
fix(conda): deduplicate repodata records to fix solver error on Linux#8337
Conversation
The conda solver (resolvo) rejects duplicate records with the same identifier and archive type. On Linux, the gateway can return overlapping records across platform-specific and noarch subdir queries, causing "encountered duplicate records for adwaita-icon-theme-..." errors that prevent installing tools like imagemagick. Fix by deduplicating records by URL in flatten_repodata before passing them to the solver. Also fix test-tool-retry.py to treat non-github/aqua backends (like conda) as soft failures instead of hard errors in the grace period check, since the grace period logic only applies to GitHub releases. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Summary of ChangesHello @jdx, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed! This pull request addresses critical issues related to package management and testing infrastructure. It ensures the robustness of the conda integration by preventing solver failures due to duplicate repodata entries and refines the test retry mechanism to correctly handle non-GitHub/Aqua backends, improving the reliability of automated checks. Highlights
Changelog
Activity
Using Gemini Code AssistThe full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips. Invoking Gemini You can request assistance from Gemini at any point by creating a comment using either
Customization To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a Limitations & Feedback Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here. You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension. Footnotes
|
There was a problem hiding this comment.
Code Review
The pull request addresses a critical issue in the conda backend where duplicate records from repodata queries could cause solver errors on Linux. The fix involves introducing a deduplication step using a HashSet based on the package URL. Additionally, it refines the test-tool-retry.py script to handle non-GitHub/Aqua backends more gracefully during grace period checks, treating them as warnings instead of hard errors. The changes are well-implemented and directly address the reported problems, improving the robustness of the conda integration and the CI tooling.
| use rattler_virtual_packages::{VirtualPackageOverrides, VirtualPackages}; | ||
| use serde::{Deserialize, Serialize}; | ||
| use std::collections::BTreeMap; | ||
| use std::collections::{BTreeMap, HashSet}; |
| /// Flatten gateway RepoData into owned records for the solver, deduplicating | ||
| /// by URL to avoid DuplicateRecords errors when the same package appears in | ||
| /// multiple subdir queries (e.g. platform + noarch). |
There was a problem hiding this comment.
Pull request overview
This PR fixes two issues that were causing test failures in the release PR #8304:
-
Conda solver duplicate records: The rattler conda repodata gateway can return duplicate package records when querying both platform-specific (e.g., Linux64) and noarch subdirs, causing the resolvo solver to reject them with duplicate record errors. The fix deduplicates records by URL before passing them to the solver.
-
Test-tool-retry grace period logic: The grace period check was treating non-GitHub/aqua backends (like conda) as hard errors, when it should only apply to tools backed by GitHub releases.
Changes:
- Added URL-based deduplication in
flatten_repodatato prevent duplicate package records from being passed to the conda solver - Changed test-tool-retry.py to treat non-GitHub/aqua backends as warnings instead of errors during grace period checks
Reviewed changes
Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
| src/backend/conda.rs | Adds HashSet-based deduplication by URL in flatten_repodata to filter out duplicate package records from multiple subdir queries |
| xtasks/test-tool-retry.py | Changes handling of tools without github/aqua backends from hard error to warning, since grace period logic only applies to GitHub releases |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
Hey @jdx this is amazing! Happy to support your efforts here! This is a relatively minimal implementation in rattler itself: https://github.com/conda/rattler/blob/main/crates/rattler-bin/src/commands/create.rs to just create an environment from some specs. We recommend |
Hyperfine Performance
|
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2026.2.19 x -- echo |
24.3 ± 0.5 | 23.4 | 27.3 | 1.00 |
mise x -- echo |
26.0 ± 0.9 | 24.6 | 33.6 | 1.07 ± 0.04 |
mise env
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2026.2.19 env |
23.6 ± 0.6 | 22.5 | 27.2 | 1.00 |
mise env |
24.8 ± 1.4 | 23.2 | 42.3 | 1.05 ± 0.06 |
mise hook-env
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2026.2.19 hook-env |
23.9 ± 0.6 | 22.7 | 27.1 | 1.00 |
mise hook-env |
25.5 ± 0.8 | 23.6 | 27.2 | 1.07 ± 0.04 |
mise ls
| Command | Mean [ms] | Min [ms] | Max [ms] | Relative |
|---|---|---|---|---|
mise-2026.2.19 ls |
20.8 ± 0.5 | 19.9 | 22.5 | 1.00 |
mise ls |
21.7 ± 0.5 | 20.8 | 24.2 | 1.04 ± 0.04 |
xtasks/test/perf
| Command | mise-2026.2.19 | mise | Variance |
|---|---|---|---|
| install (cached) | 125ms | -16% | |
| ls (cached) | 77ms | 84ms | -8% |
| bin-paths (cached) | 82ms | 89ms | -7% |
| task-ls (cached) | 808ms | 824ms | -1% |
### 🚀 Features - **(conda)** replace custom backend with rattler crates by @jdx in [#8325](#8325) - **(task)** enforce per-task timeout configuration by @tvararu in [#8250](#8250) - **(vsix)** added vsix archives to http backend by @sosumappu in [#8306](#8306) - add core dotnet plugin for .NET SDK management by @jdx in [#8326](#8326) ### 🐛 Bug Fixes - **(conda)** preserve conda_packages on locked install and fix temp file race by @jdx in [#8335](#8335) - **(conda)** deduplicate repodata records to fix solver error on Linux by @jdx in [#8337](#8337) - **(env)** include watch_files in fast-path early exit check by @jdx in [#8317](#8317) - **(env)** clear fish completions when setting/unsetting shell aliases by @jdx in [#8324](#8324) - **(lockfile)** prevent lockfile writes when --locked is set by @jdx in [#8308](#8308) - **(lockfile)** prune orphan tool entries on mise lock by @mackwic in [#8265](#8265) - **(lockfile)** error on contradictory locked=true + lockfile=false config by @jdx in [#8329](#8329) - **(regal)** Update package location by @charlieegan3 in [#8315](#8315) - **(release)** strip markdown heading prefix from communique release title by @jdx in [#8303](#8303) - **(schema)** enforce additionalProperties constraint for env by @adamliang0 in [#8328](#8328) ### 📚 Documentation - Remove incorrect oh-my-zsh plugin ordering comment by @bvosk in [#8323](#8323) - require AI disclosure on GitHub comments by @jdx in [#8330](#8330) ### 📦 Registry - add `oxfmt` by @taoufik07 in [#8316](#8316) ### New Contributors - @adamliang0 made their first contribution in [#8328](#8328) - @tvararu made their first contribution in [#8250](#8250) - @bvosk made their first contribution in [#8323](#8323) - @taoufik07 made their first contribution in [#8316](#8316) - @charlieegan3 made their first contribution in [#8315](#8315) - @sosumappu made their first contribution in [#8306](#8306) ## 📦 Aqua Registry Updates #### New Packages (3) - [`Tyrrrz/FFmpegBin`](https://github.com/Tyrrrz/FFmpegBin) - [`elixir-lang/expert`](https://github.com/elixir-lang/expert) - [`erikjuhani/basalt`](https://github.com/erikjuhani/basalt) #### Updated Packages (5) - [`caarlos0/fork-cleaner`](https://github.com/caarlos0/fork-cleaner) - [`firecow/gitlab-ci-local`](https://github.com/firecow/gitlab-ci-local) - [`jackchuka/mdschema`](https://github.com/jackchuka/mdschema) - [`kunobi-ninja/kunobi-releases`](https://github.com/kunobi-ninja/kunobi-releases) - [`peco/peco`](https://github.com/peco/peco)
…jdx#8337) ## Summary - **Fix conda solver duplicate records error**: On Linux, the conda repodata gateway can return overlapping records across platform-specific and noarch subdir queries. The resolvo solver rejects these with `"encountered duplicate records for adwaita-icon-theme-40.1.1-ha770c72_1.tar.bz2"`, preventing tools like imagemagick from installing. Fix by deduplicating records by URL in `flatten_repodata` before passing to the solver. - **Fix test-tool-retry for non-github/aqua backends**: The `test-tool-retry.py` grace period check treated non-github/aqua backends (like conda) as hard errors. Now they're treated as warnings since the grace period logic only applies to GitHub releases. Fixes the `test-tool-5` failure on the release PR (jdx#8304). ## Test plan - [x] `mise run build` — compiles - [x] `mise run lint` — all checks pass 🤖 Generated with [Claude Code](https://claude.com/claude-code) <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **Medium Risk** > Changes alter the conda solver input set and could affect dependency resolution order/selection, though the intent is limited to removing duplicate URLs. The xtask change only affects CI/tooling behavior and is low risk. > > **Overview** > Fixes conda solves failing with duplicate record errors by deduplicating flattened `RepoDataRecord`s by URL (using a `HashSet`) before passing them to the solver when querying both platform and `noarch` repodata. > > Updates `xtasks/test-tool-retry.py` so tools without a `github:`/`aqua:` backend are no longer treated as hard errors during `--grace-period` checks; they’re now logged as warnings and skipped. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit 2e8a97d. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup> <!-- /CURSOR_SUMMARY --> Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
### 🚀 Features - **(conda)** replace custom backend with rattler crates by @jdx in [jdx#8325](jdx#8325) - **(task)** enforce per-task timeout configuration by @tvararu in [jdx#8250](jdx#8250) - **(vsix)** added vsix archives to http backend by @sosumappu in [jdx#8306](jdx#8306) - add core dotnet plugin for .NET SDK management by @jdx in [jdx#8326](jdx#8326) ### 🐛 Bug Fixes - **(conda)** preserve conda_packages on locked install and fix temp file race by @jdx in [jdx#8335](jdx#8335) - **(conda)** deduplicate repodata records to fix solver error on Linux by @jdx in [jdx#8337](jdx#8337) - **(env)** include watch_files in fast-path early exit check by @jdx in [jdx#8317](jdx#8317) - **(env)** clear fish completions when setting/unsetting shell aliases by @jdx in [jdx#8324](jdx#8324) - **(lockfile)** prevent lockfile writes when --locked is set by @jdx in [jdx#8308](jdx#8308) - **(lockfile)** prune orphan tool entries on mise lock by @mackwic in [jdx#8265](jdx#8265) - **(lockfile)** error on contradictory locked=true + lockfile=false config by @jdx in [jdx#8329](jdx#8329) - **(regal)** Update package location by @charlieegan3 in [jdx#8315](jdx#8315) - **(release)** strip markdown heading prefix from communique release title by @jdx in [jdx#8303](jdx#8303) - **(schema)** enforce additionalProperties constraint for env by @adamliang0 in [jdx#8328](jdx#8328) ### 📚 Documentation - Remove incorrect oh-my-zsh plugin ordering comment by @bvosk in [jdx#8323](jdx#8323) - require AI disclosure on GitHub comments by @jdx in [jdx#8330](jdx#8330) ### 📦 Registry - add `oxfmt` by @taoufik07 in [jdx#8316](jdx#8316) ### New Contributors - @adamliang0 made their first contribution in [jdx#8328](jdx#8328) - @tvararu made their first contribution in [jdx#8250](jdx#8250) - @bvosk made their first contribution in [jdx#8323](jdx#8323) - @taoufik07 made their first contribution in [jdx#8316](jdx#8316) - @charlieegan3 made their first contribution in [jdx#8315](jdx#8315) - @sosumappu made their first contribution in [jdx#8306](jdx#8306) ## 📦 Aqua Registry Updates #### New Packages (3) - [`Tyrrrz/FFmpegBin`](https://github.com/Tyrrrz/FFmpegBin) - [`elixir-lang/expert`](https://github.com/elixir-lang/expert) - [`erikjuhani/basalt`](https://github.com/erikjuhani/basalt) #### Updated Packages (5) - [`caarlos0/fork-cleaner`](https://github.com/caarlos0/fork-cleaner) - [`firecow/gitlab-ci-local`](https://github.com/firecow/gitlab-ci-local) - [`jackchuka/mdschema`](https://github.com/jackchuka/mdschema) - [`kunobi-ninja/kunobi-releases`](https://github.com/kunobi-ninja/kunobi-releases) - [`peco/peco`](https://github.com/peco/peco)
Summary
"encountered duplicate records for adwaita-icon-theme-40.1.1-ha770c72_1.tar.bz2", preventing tools like imagemagick from installing. Fix by deduplicating records by URL inflatten_repodatabefore passing to the solver.test-tool-retry.pygrace period check treated non-github/aqua backends (like conda) as hard errors. Now they're treated as warnings since the grace period logic only applies to GitHub releases.Fixes the
test-tool-5failure on the release PR (#8304).Test plan
mise run build— compilesmise run lint— all checks pass🤖 Generated with Claude Code
Note
Medium Risk
Changes alter the conda solver input set and could affect dependency resolution order/selection, though the intent is limited to removing duplicate URLs. The xtask change only affects CI/tooling behavior and is low risk.
Overview
Fixes conda solves failing with duplicate record errors by deduplicating flattened
RepoDataRecords by URL (using aHashSet) before passing them to the solver when querying both platform andnoarchrepodata.Updates
xtasks/test-tool-retry.pyso tools without agithub:/aqua:backend are no longer treated as hard errors during--grace-periodchecks; they’re now logged as warnings and skipped.Written by Cursor Bugbot for commit 2e8a97d. This will update automatically on new commits. Configure here.