Skip to content

fix(conda): deduplicate repodata records to fix solver error on Linux#8337

Merged
jdx merged 1 commit intomainfrom
fix/conda-solver-duplicate-records
Feb 24, 2026
Merged

fix(conda): deduplicate repodata records to fix solver error on Linux#8337
jdx merged 1 commit intomainfrom
fix/conda-solver-duplicate-records

Conversation

@jdx
Copy link
Owner

@jdx jdx commented Feb 24, 2026

Summary

  • Fix conda solver duplicate records error: On Linux, the conda repodata gateway can return overlapping records across platform-specific and noarch subdir queries. The resolvo solver rejects these with "encountered duplicate records for adwaita-icon-theme-40.1.1-ha770c72_1.tar.bz2", preventing tools like imagemagick from installing. Fix by deduplicating records by URL in flatten_repodata before passing to the solver.
  • Fix test-tool-retry for non-github/aqua backends: The test-tool-retry.py grace period check treated non-github/aqua backends (like conda) as hard errors. Now they're treated as warnings since the grace period logic only applies to GitHub releases.

Fixes the test-tool-5 failure on the release PR (#8304).

Test plan

  • mise run build — compiles
  • mise run lint — all checks pass

🤖 Generated with Claude Code


Note

Medium Risk
Changes alter the conda solver input set and could affect dependency resolution order/selection, though the intent is limited to removing duplicate URLs. The xtask change only affects CI/tooling behavior and is low risk.

Overview
Fixes conda solves failing with duplicate record errors by deduplicating flattened RepoDataRecords by URL (using a HashSet) before passing them to the solver when querying both platform and noarch repodata.

Updates xtasks/test-tool-retry.py so tools without a github:/aqua: backend are no longer treated as hard errors during --grace-period checks; they’re now logged as warnings and skipped.

Written by Cursor Bugbot for commit 2e8a97d. This will update automatically on new commits. Configure here.

The conda solver (resolvo) rejects duplicate records with the same
identifier and archive type. On Linux, the gateway can return overlapping
records across platform-specific and noarch subdir queries, causing
"encountered duplicate records for adwaita-icon-theme-..." errors that
prevent installing tools like imagemagick.

Fix by deduplicating records by URL in flatten_repodata before passing
them to the solver.

Also fix test-tool-retry.py to treat non-github/aqua backends (like
conda) as soft failures instead of hard errors in the grace period check,
since the grace period logic only applies to GitHub releases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings February 24, 2026 17:28
@gemini-code-assist
Copy link
Contributor

Summary of Changes

Hello @jdx, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses critical issues related to package management and testing infrastructure. It ensures the robustness of the conda integration by preventing solver failures due to duplicate repodata entries and refines the test retry mechanism to correctly handle non-GitHub/Aqua backends, improving the reliability of automated checks.

Highlights

  • Conda Solver Error Fix: Resolved an issue where the conda repodata gateway could return duplicate records, causing the resolvo solver to fail on Linux. Records are now deduplicated by URL in flatten_repodata.
  • Test Tool Retry Logic Update: Modified test-tool-retry.py to treat missing GitHub/Aqua backends as warnings instead of hard errors for the grace period check, as this logic is only relevant for GitHub releases.
Changelog
  • src/backend/conda.rs
    • Imported HashSet for efficient deduplication.
    • Modified flatten_repodata to deduplicate RepoDataRecords based on their URL, preventing solver errors caused by overlapping records.
    • Updated the documentation for flatten_repodata to reflect the new deduplication logic.
  • xtasks/test-tool-retry.py
    • Changed an error message to a warning when check_grace_period encounters a tool without a GitHub/Aqua backend, as the grace period logic does not apply in such cases.
    • Removed the addition of such tools to the hard_failures list, aligning the behavior with the intended purpose of the grace period check.
Activity
  • No human activity has occurred on this pull request yet.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

The pull request addresses a critical issue in the conda backend where duplicate records from repodata queries could cause solver errors on Linux. The fix involves introducing a deduplication step using a HashSet based on the package URL. Additionally, it refines the test-tool-retry.py script to handle non-GitHub/Aqua backends more gracefully during grace period checks, treating them as warnings instead of hard errors. The changes are well-implemented and directly address the reported problems, improving the robustness of the conda integration and the CI tooling.

use rattler_virtual_packages::{VirtualPackageOverrides, VirtualPackages};
use serde::{Deserialize, Serialize};
use std::collections::BTreeMap;
use std::collections::{BTreeMap, HashSet};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The addition of HashSet is necessary for the deduplication logic introduced in flatten_repodata. This is a good change to prevent solver errors caused by duplicate records.

Comment on lines +95 to +97
/// Flatten gateway RepoData into owned records for the solver, deduplicating
/// by URL to avoid DuplicateRecords errors when the same package appears in
/// multiple subdir queries (e.g. platform + noarch).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The updated documentation for flatten_repodata clearly explains the purpose of the deduplication, which is crucial for understanding why this change was made. This improves code clarity and maintainability.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR fixes two issues that were causing test failures in the release PR #8304:

  1. Conda solver duplicate records: The rattler conda repodata gateway can return duplicate package records when querying both platform-specific (e.g., Linux64) and noarch subdirs, causing the resolvo solver to reject them with duplicate record errors. The fix deduplicates records by URL before passing them to the solver.

  2. Test-tool-retry grace period logic: The grace period check was treating non-GitHub/aqua backends (like conda) as hard errors, when it should only apply to tools backed by GitHub releases.

Changes:

  • Added URL-based deduplication in flatten_repodata to prevent duplicate package records from being passed to the conda solver
  • Changed test-tool-retry.py to treat non-GitHub/aqua backends as warnings instead of errors during grace period checks

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/backend/conda.rs Adds HashSet-based deduplication by URL in flatten_repodata to filter out duplicate package records from multiple subdir queries
xtasks/test-tool-retry.py Changes handling of tools without github/aqua backends from hard error to warning, since grace period logic only applies to GitHub releases

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@wolfv
Copy link
Contributor

wolfv commented Feb 24, 2026

Hey @jdx this is amazing! Happy to support your efforts here!

This is a relatively minimal implementation in rattler itself: https://github.com/conda/rattler/blob/main/crates/rattler-bin/src/commands/create.rs to just create an environment from some specs.

We recommend conda-forge as the default channel for what it's worth.

@github-actions
Copy link

Hyperfine Performance

mise x -- echo

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2026.2.19 x -- echo 24.3 ± 0.5 23.4 27.3 1.00
mise x -- echo 26.0 ± 0.9 24.6 33.6 1.07 ± 0.04

mise env

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2026.2.19 env 23.6 ± 0.6 22.5 27.2 1.00
mise env 24.8 ± 1.4 23.2 42.3 1.05 ± 0.06

mise hook-env

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2026.2.19 hook-env 23.9 ± 0.6 22.7 27.1 1.00
mise hook-env 25.5 ± 0.8 23.6 27.2 1.07 ± 0.04

mise ls

Command Mean [ms] Min [ms] Max [ms] Relative
mise-2026.2.19 ls 20.8 ± 0.5 19.9 22.5 1.00
mise ls 21.7 ± 0.5 20.8 24.2 1.04 ± 0.04

xtasks/test/perf

Command mise-2026.2.19 mise Variance
install (cached) 125ms ⚠️ 150ms -16%
ls (cached) 77ms 84ms -8%
bin-paths (cached) 82ms 89ms -7%
task-ls (cached) 808ms 824ms -1%

⚠️ Warning: install cached performance variance is -16%

@jdx jdx merged commit 71e29e8 into main Feb 24, 2026
55 of 57 checks passed
@jdx jdx deleted the fix/conda-solver-duplicate-records branch February 24, 2026 21:54
mise-en-dev added a commit that referenced this pull request Feb 25, 2026
### 🚀 Features

- **(conda)** replace custom backend with rattler crates by @jdx in
[#8325](#8325)
- **(task)** enforce per-task timeout configuration by @tvararu in
[#8250](#8250)
- **(vsix)** added vsix archives to http backend by @sosumappu in
[#8306](#8306)
- add core dotnet plugin for .NET SDK management by @jdx in
[#8326](#8326)

### 🐛 Bug Fixes

- **(conda)** preserve conda_packages on locked install and fix temp
file race by @jdx in [#8335](#8335)
- **(conda)** deduplicate repodata records to fix solver error on Linux
by @jdx in [#8337](#8337)
- **(env)** include watch_files in fast-path early exit check by @jdx in
[#8317](#8317)
- **(env)** clear fish completions when setting/unsetting shell aliases
by @jdx in [#8324](#8324)
- **(lockfile)** prevent lockfile writes when --locked is set by @jdx in
[#8308](#8308)
- **(lockfile)** prune orphan tool entries on mise lock by @mackwic in
[#8265](#8265)
- **(lockfile)** error on contradictory locked=true + lockfile=false
config by @jdx in [#8329](#8329)
- **(regal)** Update package location by @charlieegan3 in
[#8315](#8315)
- **(release)** strip markdown heading prefix from communique release
title by @jdx in [#8303](#8303)
- **(schema)** enforce additionalProperties constraint for env by
@adamliang0 in [#8328](#8328)

### 📚 Documentation

- Remove incorrect oh-my-zsh plugin ordering comment by @bvosk in
[#8323](#8323)
- require AI disclosure on GitHub comments by @jdx in
[#8330](#8330)

### 📦 Registry

- add `oxfmt` by @taoufik07 in
[#8316](#8316)

### New Contributors

- @adamliang0 made their first contribution in
[#8328](#8328)
- @tvararu made their first contribution in
[#8250](#8250)
- @bvosk made their first contribution in
[#8323](#8323)
- @taoufik07 made their first contribution in
[#8316](#8316)
- @charlieegan3 made their first contribution in
[#8315](#8315)
- @sosumappu made their first contribution in
[#8306](#8306)

## 📦 Aqua Registry Updates

#### New Packages (3)

- [`Tyrrrz/FFmpegBin`](https://github.com/Tyrrrz/FFmpegBin)
- [`elixir-lang/expert`](https://github.com/elixir-lang/expert)
- [`erikjuhani/basalt`](https://github.com/erikjuhani/basalt)

#### Updated Packages (5)

- [`caarlos0/fork-cleaner`](https://github.com/caarlos0/fork-cleaner)
-
[`firecow/gitlab-ci-local`](https://github.com/firecow/gitlab-ci-local)
- [`jackchuka/mdschema`](https://github.com/jackchuka/mdschema)
-
[`kunobi-ninja/kunobi-releases`](https://github.com/kunobi-ninja/kunobi-releases)
- [`peco/peco`](https://github.com/peco/peco)
risu729 pushed a commit to risu729/mise that referenced this pull request Feb 27, 2026
…jdx#8337)

## Summary
- **Fix conda solver duplicate records error**: On Linux, the conda
repodata gateway can return overlapping records across platform-specific
and noarch subdir queries. The resolvo solver rejects these with
`"encountered duplicate records for
adwaita-icon-theme-40.1.1-ha770c72_1.tar.bz2"`, preventing tools like
imagemagick from installing. Fix by deduplicating records by URL in
`flatten_repodata` before passing to the solver.
- **Fix test-tool-retry for non-github/aqua backends**: The
`test-tool-retry.py` grace period check treated non-github/aqua backends
(like conda) as hard errors. Now they're treated as warnings since the
grace period logic only applies to GitHub releases.

Fixes the `test-tool-5` failure on the release PR (jdx#8304).

## Test plan
- [x] `mise run build` — compiles
- [x] `mise run lint` — all checks pass

🤖 Generated with [Claude Code](https://claude.com/claude-code)

<!-- CURSOR_SUMMARY -->
---

> [!NOTE]
> **Medium Risk**
> Changes alter the conda solver input set and could affect dependency
resolution order/selection, though the intent is limited to removing
duplicate URLs. The xtask change only affects CI/tooling behavior and is
low risk.
> 
> **Overview**
> Fixes conda solves failing with duplicate record errors by
deduplicating flattened `RepoDataRecord`s by URL (using a `HashSet`)
before passing them to the solver when querying both platform and
`noarch` repodata.
> 
> Updates `xtasks/test-tool-retry.py` so tools without a
`github:`/`aqua:` backend are no longer treated as hard errors during
`--grace-period` checks; they’re now logged as warnings and skipped.
> 
> <sup>Written by [Cursor
Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit
2e8a97d. This will update automatically
on new commits. Configure
[here](https://cursor.com/dashboard?tab=bugbot).</sup>
<!-- /CURSOR_SUMMARY -->

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
risu729 pushed a commit to risu729/mise that referenced this pull request Feb 27, 2026
### 🚀 Features

- **(conda)** replace custom backend with rattler crates by @jdx in
[jdx#8325](jdx#8325)
- **(task)** enforce per-task timeout configuration by @tvararu in
[jdx#8250](jdx#8250)
- **(vsix)** added vsix archives to http backend by @sosumappu in
[jdx#8306](jdx#8306)
- add core dotnet plugin for .NET SDK management by @jdx in
[jdx#8326](jdx#8326)

### 🐛 Bug Fixes

- **(conda)** preserve conda_packages on locked install and fix temp
file race by @jdx in [jdx#8335](jdx#8335)
- **(conda)** deduplicate repodata records to fix solver error on Linux
by @jdx in [jdx#8337](jdx#8337)
- **(env)** include watch_files in fast-path early exit check by @jdx in
[jdx#8317](jdx#8317)
- **(env)** clear fish completions when setting/unsetting shell aliases
by @jdx in [jdx#8324](jdx#8324)
- **(lockfile)** prevent lockfile writes when --locked is set by @jdx in
[jdx#8308](jdx#8308)
- **(lockfile)** prune orphan tool entries on mise lock by @mackwic in
[jdx#8265](jdx#8265)
- **(lockfile)** error on contradictory locked=true + lockfile=false
config by @jdx in [jdx#8329](jdx#8329)
- **(regal)** Update package location by @charlieegan3 in
[jdx#8315](jdx#8315)
- **(release)** strip markdown heading prefix from communique release
title by @jdx in [jdx#8303](jdx#8303)
- **(schema)** enforce additionalProperties constraint for env by
@adamliang0 in [jdx#8328](jdx#8328)

### 📚 Documentation

- Remove incorrect oh-my-zsh plugin ordering comment by @bvosk in
[jdx#8323](jdx#8323)
- require AI disclosure on GitHub comments by @jdx in
[jdx#8330](jdx#8330)

### 📦 Registry

- add `oxfmt` by @taoufik07 in
[jdx#8316](jdx#8316)

### New Contributors

- @adamliang0 made their first contribution in
[jdx#8328](jdx#8328)
- @tvararu made their first contribution in
[jdx#8250](jdx#8250)
- @bvosk made their first contribution in
[jdx#8323](jdx#8323)
- @taoufik07 made their first contribution in
[jdx#8316](jdx#8316)
- @charlieegan3 made their first contribution in
[jdx#8315](jdx#8315)
- @sosumappu made their first contribution in
[jdx#8306](jdx#8306)

## 📦 Aqua Registry Updates

#### New Packages (3)

- [`Tyrrrz/FFmpegBin`](https://github.com/Tyrrrz/FFmpegBin)
- [`elixir-lang/expert`](https://github.com/elixir-lang/expert)
- [`erikjuhani/basalt`](https://github.com/erikjuhani/basalt)

#### Updated Packages (5)

- [`caarlos0/fork-cleaner`](https://github.com/caarlos0/fork-cleaner)
-
[`firecow/gitlab-ci-local`](https://github.com/firecow/gitlab-ci-local)
- [`jackchuka/mdschema`](https://github.com/jackchuka/mdschema)
-
[`kunobi-ninja/kunobi-releases`](https://github.com/kunobi-ninja/kunobi-releases)
- [`peco/peco`](https://github.com/peco/peco)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants