refactor(resolver): move yank policy to resolver layer #17083
Conversation
Error message is not helpful here because `source::query` has filtered out yanked versions. Later install didn't know how to recover from that. Only exact requirement has informative error message. This will be improved when moving yanked version handling to resolver
|
r? @epage rustbot has assigned @epage. Use Why was this reviewer chosen?The reviewer was selected based on:
|
e171161 to
f4a63f0
Compare
| false | ||
| }; | ||
| if is_yanked { | ||
| // Let's see if there is any yanked version so can give a more concrete error. |
There was a problem hiding this comment.
This showcases where this refactor could be useful.
| let summaries = crate::util::block_on(source.query_vec(&query, QueryKind::Exact))?; | ||
| let available = summaries.iter().any(|s| !s.as_summary().is_yanked()); | ||
| Ok(available) | ||
| Ok(!summaries.is_empty()) |
There was a problem hiding this comment.
This showcases where this refactor could be useful.
| /// Available for consideration | ||
| Candidate(Summary), | ||
| /// Yanked within its registry | ||
| Yanked(Summary), |
There was a problem hiding this comment.
I'm hesitant about dropping this. Having it as an enum ensures explicit acknowledgement rather than hoping callers know to check this.
There was a problem hiding this comment.
We can keep this, though I wonder how we envision on IndexSummary evolution. Do we want to have variant like IndexSummary::TooNew for min-publish-age? It was done in #17012. Or we treat yanked a bit special but not others.
Actually having IndexSummary::TooNew may help as we can flag those at registry level, and apply the policy at resolver level. What do you think?
explicit acknowledgement rather than hoping callers know to check this.
Caller still need to manually check because we have IndexSummary::{as_summary, into_summary} providing a error-prone API bypassing it. And they are all over the places we call Summary::is_yanked in this PR.
There was a problem hiding this comment.
Is there anything we can do to help in this situation?
What is your read on how worth it this has been in the end?
There was a problem hiding this comment.
I experimented a branch removing as_summary and into_summary. It was noisy. Some call sites only need something like s.as_summary().version() and maybe for that we should expose IndexSummary::version. Or maybe Invalid/Unsupported/Offline should be folded into one Unavailable and have their sub-enum.
Actually having
IndexSummary::TooNewmay help as we can flag those at registry level, and apply the policy at resolver level. What do you think?
This may look weird, as IndexSummary::parse currently doesn't have any registry specific logic. If we construct IndexSummary::TooNew there we need to thread registry config into. If we don't, we need to remap IndexSummary during query just like how IndexSummary::Offline is doing now.
On the other hand, the current change (removing IndexSummary:Yanked) is also weird because it pushes the yanked field down to Summary which is shared with git/path packages as well, and they don't have the concept of yanking.
I don't have a good answer to it though, maybe we can start with not removing IndexSummary::Yanked and see what we can do next?
There was a problem hiding this comment.
Done the refactor. I plan to remove into_summary and as_summary in follow-up.
f4a63f0 to
0a51a03
Compare
Since this is a refactor, we preserve the old `Source::query` behavior that filter out yanked by default. Some queries actually can live without filter to get better diagnostics, though those will be in follow-up fixes. Here is the list of all call site I audit: | Call site | Yanked versions handling | | --------- | ------------------------ | | [`core/resolver/dep_cache.rs` `query`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/core/resolver/dep_cache.rs#L64) | Filter | | [`core/resolver/dep_cache.rs` `[replace]` override](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/core/resolver/dep_cache.rs#L91) | Filter | | [`core/registry.rs` `patch` collection](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/core/registry.rs#L383) | Filter | | [`core/registry.rs` `query_overrides`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/core/registry.rs#L553) | Filter | | [`core/registry.rs` `summary_for_patch` unlocked-retry](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/core/registry.rs#L965) | Filter | | [`core/registry.rs` `summary_for_patch` name-only diagnostic](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/core/registry.rs#L998) | Filter | | [`core/resolver/errors.rs` `alt_versions`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/core/resolver/errors.rs#L432) | Filter | | [`core/resolver/errors.rs` `rejected_versions`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/core/resolver/errors.rs#L450) | Keep | | [`core/resolver/errors.rs` `alt_names`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/core/resolver/errors.rs#L469) | Keep | | [`ops/common_for_install_and_uninstall.rs` `select_dep_pkg` (x2)](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/ops/common_for_install_and_uninstall.rs#L611) | Filter | | [`ops/cargo_update.rs` `upgrade_dependency`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/ops/cargo_update.rs#L373) | Filter | | [`ops/cargo_update.rs` `report_latest` (x4)](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/ops/cargo_update.rs#L791) | Filter | | [`ops/cargo_add/mod.rs` `get_latest_dependency`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/ops/cargo_add/mod.rs#L823) | Filter | | [`ops/cargo_add/mod.rs` `select_package`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/ops/cargo_add/mod.rs#L952) | Filter | | [`ops/cargo_add/mod.rs` `populate_available_features`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/ops/cargo_add/mod.rs#L1180) | Filter | | [`ops/registry/info/mod.rs` `query_summaries`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/ops/registry/info/mod.rs#L215) | Filter | | [`ops/registry/publish.rs` `poll_one_package`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/ops/registry/publish.rs#L439) | Filter | | [`ops/registry/publish.rs` `verify_unpublished`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/ops/registry/publish.rs#L451) | Filter | | [`core/compiler/future_incompat.rs` `get_updates`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/src/cargo/core/compiler/future_incompat.rs#L334) | Filter | | [`crates/xtask-bump-check/src/xtask.rs` `check`](https://github.com/rust-lang/cargo/blob/71b70c095bb15e278ab9f0f808397c8033079888/crates/xtask-bump-check/src/xtask.rs#L448) | Filter | This doesn't go into `VersionPreferences` or `sort_summaries` yet because * We want to minimize the refactor change. Changing the order of query and filter may change lots of diagnostics * `[replace]` resolution should not see yanked candidates either * A future `VersionPreferences` predicate may still need to be invoked at this same point.
0a51a03 to
9041c48
Compare
| // Prefer yanked only when | ||
| // | ||
| // * it is recorded in lock file or a `[patch]` entry | ||
| // * it is specified in `cargo update --precise` | ||
| IndexSummary::Yanked(summary) => { | ||
| let pkg_id = summary.package_id(); | ||
| let allow_precise = pkg_id | ||
| .source_id() | ||
| .precise_registry_version(pkg_id.name().as_str()) | ||
| .is_some_and(|(_, to)| to == pkg_id.version()); | ||
| if allow_precise || self.version_prefs.should_prefer(&pkg_id) { |
There was a problem hiding this comment.
Isn't this beyond what the current code does for this to be just a refactor?
There was a problem hiding this comment.
Oh, its because this logic got moved up into this layer
There was a problem hiding this comment.
If so, why isn't this needed at other call sites?
There was a problem hiding this comment.
Oh, is it because this code is only active within this branch?
There was a problem hiding this comment.
Yes, I think you answered yourself :)
### What does this PR try to resolve? `cargo_package::check_yanked` is the only external caller. Replace it with a direct `Source::query` instead since we want `Source` to be yanked-naive. This is an follow-up of rust-lang#17083 ### How to test and review this PR?
…17092) ### What does this PR try to resolve? `Summary::{as_summary,into_summary}` are easy to misuse. * Removed `into_summary` as it has only one caller. * Changed the visibility of `as_summary` and renamed it to `as_summary_unchecked`. This is a follow-up of rust-lang#17083 ### How to test and review this PR?
Update cargo submodule
11 commits in 0b1123a48825309b697312b44fdb64b3df00c958..fe63976b245b8a649c3f2949bf89fdc307bfbae4
2026-06-01 21:20:28 +0000 to 2026-06-11 09:17:57 +0000
- refactor: reduce use `Summary::{as_summary,into_summary}` (rust-lang/cargo#17092)
- docs(diag): Provide jumping off points for writing diagnostics and passes (rust-lang/cargo#17090)
- refactor(source): drop `Source::is_yanked` (rust-lang/cargo#17091)
- refactor(resolver): move yank policy to resolver layer (rust-lang/cargo#17083)
- fix(publish): avoid false deadlock when to_confirm is non-empty (rust-lang/cargo#17071)
- doc(guide): use fresh actions/checkout version in GH actions examples (rust-lang/cargo#17087)
- fix: strip CR from `cargo:token-from-stdout` (rust-lang/cargo#17081)
- test: show some odd `--precise` cases (rust-lang/cargo#17082)
- chore(deps): update rust crate gix to 0.84.0 (rust-lang/cargo#17063)
- refactor: drop `yanked_whitelist` from source loading (rust-lang/cargo#17014)
- chore(deps): update msrv to v1.96 (rust-lang/cargo#17041)
Update cargo submodule
11 commits in 0b1123a48825309b697312b44fdb64b3df00c958..fe63976b245b8a649c3f2949bf89fdc307bfbae4
2026-06-01 21:20:28 +0000 to 2026-06-11 09:17:57 +0000
- refactor: reduce use `Summary::{as_summary,into_summary}` (rust-lang/cargo#17092)
- docs(diag): Provide jumping off points for writing diagnostics and passes (rust-lang/cargo#17090)
- refactor(source): drop `Source::is_yanked` (rust-lang/cargo#17091)
- refactor(resolver): move yank policy to resolver layer (rust-lang/cargo#17083)
- fix(publish): avoid false deadlock when to_confirm is non-empty (rust-lang/cargo#17071)
- doc(guide): use fresh actions/checkout version in GH actions examples (rust-lang/cargo#17087)
- fix: strip CR from `cargo:token-from-stdout` (rust-lang/cargo#17081)
- test: show some odd `--precise` cases (rust-lang/cargo#17082)
- chore(deps): update rust crate gix to 0.84.0 (rust-lang/cargo#17063)
- refactor: drop `yanked_whitelist` from source loading (rust-lang/cargo#17014)
- chore(deps): update msrv to v1.96 (rust-lang/cargo#17041)
Update cargo submodule
11 commits in 0b1123a48825309b697312b44fdb64b3df00c958..fe63976b245b8a649c3f2949bf89fdc307bfbae4
2026-06-01 21:20:28 +0000 to 2026-06-11 09:17:57 +0000
- refactor: reduce use `Summary::{as_summary,into_summary}` (rust-lang/cargo#17092)
- docs(diag): Provide jumping off points for writing diagnostics and passes (rust-lang/cargo#17090)
- refactor(source): drop `Source::is_yanked` (rust-lang/cargo#17091)
- refactor(resolver): move yank policy to resolver layer (rust-lang/cargo#17083)
- fix(publish): avoid false deadlock when to_confirm is non-empty (rust-lang/cargo#17071)
- doc(guide): use fresh actions/checkout version in GH actions examples (rust-lang/cargo#17087)
- fix: strip CR from `cargo:token-from-stdout` (rust-lang/cargo#17081)
- test: show some odd `--precise` cases (rust-lang/cargo#17082)
- chore(deps): update rust crate gix to 0.84.0 (rust-lang/cargo#17063)
- refactor: drop `yanked_whitelist` from source loading (rust-lang/cargo#17014)
- chore(deps): update msrv to v1.96 (rust-lang/cargo#17041)
Update cargo submodule
11 commits in 0b1123a48825309b697312b44fdb64b3df00c958..fe63976b245b8a649c3f2949bf89fdc307bfbae4
2026-06-01 21:20:28 +0000 to 2026-06-11 09:17:57 +0000
- refactor: reduce use `Summary::{as_summary,into_summary}` (rust-lang/cargo#17092)
- docs(diag): Provide jumping off points for writing diagnostics and passes (rust-lang/cargo#17090)
- refactor(source): drop `Source::is_yanked` (rust-lang/cargo#17091)
- refactor(resolver): move yank policy to resolver layer (rust-lang/cargo#17083)
- fix(publish): avoid false deadlock when to_confirm is non-empty (rust-lang/cargo#17071)
- doc(guide): use fresh actions/checkout version in GH actions examples (rust-lang/cargo#17087)
- fix: strip CR from `cargo:token-from-stdout` (rust-lang/cargo#17081)
- test: show some odd `--precise` cases (rust-lang/cargo#17082)
- chore(deps): update rust crate gix to 0.84.0 (rust-lang/cargo#17063)
- refactor: drop `yanked_whitelist` from source loading (rust-lang/cargo#17014)
- chore(deps): update msrv to v1.96 (rust-lang/cargo#17041)
What does this PR try to resolve?
This PR moves move yank policy to resolver layer from source layer.
We mostly preserve the old
Source::querybehavior that filter out yanked by default.The yanking policy logic doesn't go into
VersionPreferencesorsort_summariesyet because[replace]resolution should not see yanked candidates eitherVersionPreferencespredicate may still need to be invoked at this same point.cc #17012, #17014 (comment)
How to test and review this PR?
Three tests are added to pin down more existing tests, along with #17082
Here is the list of all call site I audit:
core/resolver/dep_cache.rsquerycore/resolver/dep_cache.rs[replace]overridecore/registry.rspatchcollectioncore/registry.rsquery_overridescore/registry.rssummary_for_patchunlocked-retrycore/registry.rssummary_for_patchname-only diagnosticcore/resolver/errors.rsalt_versionscore/resolver/errors.rsrejected_versionscore/resolver/errors.rsalt_namesops/common_for_install_and_uninstall.rsselect_dep_pkg(x2)ops/cargo_update.rsupgrade_dependencyops/cargo_update.rsreport_latest(x4)ops/cargo_add/mod.rsget_latest_dependencyops/cargo_add/mod.rsselect_packageops/cargo_add/mod.rspopulate_available_featuresops/registry/info/mod.rsquery_summariesops/registry/publish.rspoll_one_packageops/registry/publish.rsverify_unpublishedcore/compiler/future_incompat.rsget_updatescrates/xtask-bump-check/src/xtask.rscheck