feat: -Zmin-publish-age (RFC 3923)#17012
Conversation
|
r? @epage rustbot has assigned @epage. Use Why was this reviewer chosen?The reviewer was selected based on:
|
|
Found something wrong. Wait a sec. |
2477e6c to
bc3e9c0
Compare
| IndexSummary::TooNew(summary, age) => { | ||
| let opts = jiff::SpanRound::new() | ||
| .largest(jiff::Unit::Day) | ||
| .smallest(jiff::Unit::Minute) | ||
| .relative(jiff::SpanRelativeTo::days_are_24_hours()); |
There was a problem hiding this comment.
The version 1.99.0 is too new (published 2d ago) message currenly show the time span in always down to minute, and up to day precise. It is annoying that you would get something like 2d 8h 23m long. We might want to round it more for better UX. I personally would leave it for follow-up.
There was a problem hiding this comment.
@clouatre commented in #17012 (review):
The
largest=Day, smallest=Minuterounding produces output likepublished 2d 8h 23m ago. Suggest simplifying to a single unit based on magnitude:>= 2 daysrounds to nearest day (published 3 days ago),< 2 daysrounds to nearest hour (published 11 hours ago). The sub-day precision is noise at the day scale and the user cannot act on the minutes component.For context: we currently enforce this in CI with a bash script that diffs
Cargo.lockagainst the base branch and calls the crates.io API to check publish timestamps on net-new crates. This feature would replace that entirely. The error message is the main user-facing surface, so getting the formatting right matters.
There was a problem hiding this comment.
Moved it here so it is better to discuss in a thread.
There was a problem hiding this comment.
Plan to leave it in a follow-up since the PR is large in size already
There was a problem hiding this comment.
since the PR is large in size already
imo this would be much faster to review if it was broken up even further.
| } | ||
| } | ||
|
|
||
| fn warn_unused_min_publish_age(gctx: &GlobalContext) -> CargoResult<()> { |
There was a problem hiding this comment.
We warn all unused min-publish-age all at once, regardless of the precedence rule.
There was a problem hiding this comment.
I didn't do an extensive testing for different age span. If it is needed let me know
1ea1f55 to
ddba374
Compare
There was a problem hiding this comment.
The largest=Day, smallest=Minute rounding produces output like published 2d 8h 23m ago. Suggest simplifying to a single unit based on magnitude: >= 2 days rounds to nearest day (published 3 days ago), < 2 days rounds to nearest hour (published 11 hours ago). The sub-day precision is noise at the day scale and the user cannot act on the minutes component.
For context: we currently enforce this in CI with a bash script that diffs Cargo.lock against the base branch and calls the crates.io API to check publish timestamps on net-new crates. This feature would replace that entirely. The error message is the main user-facing surface, so getting the formatting right matters.
| // `min-publish-age` policy must be applied here too. Per RFC 3923 it uses | ||
| // `for_install`: `resolver.incompatible-publish-age` does not apply to | ||
| // `cargo install`, so `allow` cannot disable the check here. | ||
| let publish_age = PublishAgePolicy::for_install(gctx)?; |
There was a problem hiding this comment.
I thought cargo install was to be untouched?
There was a problem hiding this comment.
I think there was some ambiguity in the last minute edit: rust-lang/rfcs@6dca5a1.
We can remove this install check if we don't want for now, and discuss before stabilization.
There was a problem hiding this comment.
My intention when I signed off on the RFC was that cargo install would not change its behavior which would match what we did with MSRV aware resolver. [registry] sets values but not what to do with them. That is defined in the [resolver] table which does not apply to cargo install.
There was a problem hiding this comment.
I somehow missed that edit and had been pushing through the RFC process for the exact opposite.
There was a problem hiding this comment.
That edit runs counter to the threads at
- https://triagebot.infra.rust-lang.org/gh-comments/rust-lang/rfcs/pull/3923#discussion_r2841930230
- https://triagebot.infra.rust-lang.org/gh-comments/rust-lang/rfcs/pull/3923#discussion_r3162275161
Looks like that edit was made in response to https://triagebot.infra.rust-lang.org/gh-comments/rust-lang/rfcs/pull/3923#discussion_r3191124681 after I had checked my box.
There was a problem hiding this comment.
The version before the edit:
cargo installalso skips pubtime-incompatible versions
as the[resolver]table is documented as not applying tocargo install.
My interpretation of "skip" was the same as the edited version.
It feels more nature. The rationale:
- min-publish-age is a registry level config. It also seems nature to apply by default
[resolver]is a global resolver behavior control over other the default registry config.
For example, if in the future we have [[registry.denylist]] then it would also apply to cargo install by default, while resolver.allow-denylist = "deny" (what a bad name) would override registry config. That said, we have globally unique pkgid so we probably won't have denylist under registry.
Anyway, I'll remove that part.
There was a problem hiding this comment.
Wow, we had been talking past each other a lot there. For example, I think I had edited in a word and thought you were saying that any of the logic in this RFC would be skipped for cargo install.
| /// See the [`invocation_time`] field doc for details. | ||
| /// | ||
| /// [`invocation_time`]: GlobalContext::invocation_time | ||
| pub fn invocation_time(&self) -> jiff::Timestamp { |
There was a problem hiding this comment.
Does gc have a "now" that can also use this?
There was a problem hiding this comment.
Let me see if we can split this off.
There was a problem hiding this comment.
Looked into this. gc does have its own now from global_cache_tracker::now() While it is currently pretty close to startup time, GC in the future can be run multiple times other than that. Let's keep them separate for now.
| /// Mocked "now" for deterministic publish-age comparisons. | ||
| const NOW: &str = "2006-08-08T00:00:00Z"; | ||
|
|
||
| fn setup_packages() { |
There was a problem hiding this comment.
publish_packages to better clarify intent?
| .env("__CARGO_TEST_INVOCATION_TIME", NOW) | ||
| .run(); | ||
|
|
||
| p.cargo("update -Zunstable-options -Zmin-publish-age --breaking bar") |
There was a problem hiding this comment.
Personally, I wouldn't consider it worth including --breaking in this
There was a problem hiding this comment.
Fair enough. Whoever using --breaking may not afraid of breaks. Will abandon the change and document it in the big table in the PR description.
There was a problem hiding this comment.
imo --breaking is in too broken of a state.
There was a problem hiding this comment.
I kept the tests there just in case we won't forget they are related in the future. Snapshot updates are cheap thanks to snapbox :)
There was a problem hiding this comment.
I would rather dump the tests personally and just note this in the tracking issue.
If nothing else, what do we name these tests? Currently, they describe the intended behavior but the test bodies won't match that.
Also warn when `-Zmin-publish-age` is absent
Implement the `-Zmin-publish-age` filter at the resolver layer. The resolver filters out versions that are too new, unless pinned by a lock file or a `[patch]` entry.
When resolution fails because every candidate is newer than the configured `min-publish-age`, report "version X is too new (published N ago)" instead of the generic "version X is unavailable".
Since policy is now in resolver layer, any query doesn't use resolver should enforce the policy if needed. `cargo add` is one of these places.
a97b933 to
ec4f3b6
Compare
Report `published 3 days ago` / `published 11 hours ago` instead of a multi-unit span like `2d 8h 23m ago`. Sub-day precision is noise.
| fn setup_packages() { | ||
| fn publish_packages() { |
There was a problem hiding this comment.
Fixed this in the wrong commit?
| #[cargo_test] | ||
| fn cargo_install_filters_too_new() { |
There was a problem hiding this comment.
If we are removing cargo install support, then these test names don't reflect expected behavior: that cargo install is unaffected
| let global = parse(gctx.get::<Option<String>>("registry.global-min-publish-age")?); | ||
| let crates_io = parse(gctx.get::<Option<String>>("registry.min-publish-age")?); |
There was a problem hiding this comment.
I take it [registry] is one of those tables that we don't load as a struct but per-field? Or is this related to being unstable?
| let parse = |raw: Option<String>| -> Option<Duration> { | ||
| let raw = raw?; | ||
| // A configured value of `"0"` disables the threshold. | ||
| if raw == "0" { | ||
| return None; | ||
| } | ||
| maybe_parse_time_span(&raw) | ||
| }; | ||
|
|
||
| let global = parse(gctx.get::<Option<String>>("registry.global-min-publish-age")?); | ||
| let crates_io = parse(gctx.get::<Option<String>>("registry.min-publish-age")?); |
There was a problem hiding this comment.
maybe_parse_time_span returns None on error but here we are treating None as "unset". See also invalid_publish_age_value which shows no errors which I would expect it would.
| /// 1. `registries.<name>.min-publish-age` | ||
| /// 2. `registry.min-publish-age` (default registry) | ||
| /// 3. `registry.global-min-publish-age` | ||
| fn min_age(&self, source_id: SourceId) -> Option<Duration> { | ||
| let specific = if let Some(name) = source_id.alt_registry_key() { | ||
| self.per_registry.get(name).copied() | ||
| } else if source_id.is_crates_io() { | ||
| self.crates_io | ||
| } else { | ||
| None | ||
| }; |
There was a problem hiding this comment.
Can you do registries.crates-io?
| } else { | ||
| None | ||
| }; | ||
| specific.or(self.global) |
There was a problem hiding this comment.
"0" means "use latest" but is translated to None above and None is treated here as "unset" and falls back to another source
| /// 1. `registries.<name>.min-publish-age` | ||
| /// 2. `registry.min-publish-age` (default registry) | ||
| /// 3. `registry.global-min-publish-age` | ||
| fn min_age(&self, source_id: SourceId) -> Option<Duration> { |
There was a problem hiding this comment.
nit: As this is a helper for too_new, personally, i would put too_new first
See also https://epage.github.io/dev/rust-style/#m-caller-callee
| [UPGRADING] bar ^1 -> ^2 | ||
| [ERROR] failed to select a version for the requirement `bar = "^2.1.0"` | ||
| version 2.1.0 is unavailable | ||
| version 2.1.0 is too new (published 1d ago) |
There was a problem hiding this comment.
In the RFC, we suggested
$ cargo update
error: failed to select a version for the requirement `some-package = "^1.3"`
version 1.3.0 is too new (published 2 days ago, minimum age 14 days)
help: to preserve the min-publish-age, downgrade the version requirement to `"1.1"`
help: to use `"1.3"` anyways, re-resolve with `CARGO_RESOLVER_INCOMPATIBLE_PUBLISH_AGE=allow`
- listing the publish age helps to know which value applied
- give next steps both to respect min publish age and to override it
| use crate::core::Workspace; | ||
| use crate::core::dependency::DepKind; | ||
| use crate::core::registry::PackageRegistry; | ||
| use crate::core::resolver::PublishAgePolicy; |
There was a problem hiding this comment.
Should the resolver be home to different policies like this? At first, it was surprising to me. Now, I somewhat get it.
| possibilities.retain(|s| publish_age.too_new(s).is_none()); | ||
| if possibilities.is_empty() && has_candidates { | ||
| anyhow::bail!( | ||
| "all versions of crate `{dependency}` are too new per `min-publish-age`" |
There was a problem hiding this comment.
Should we provide more context, like we do with the resolver?
| required by package `foo v0.0.0 ([ROOT]/foo)` | ||
| [ADDING] bar v1.0.0 to dependencies | ||
| [LOCKING] 1 package to latest compatible version | ||
| [ADDING] bar v1.0.0 (available: v1.1.0) |
There was a problem hiding this comment.
The RFC called for these to provide more context, like
Adding some-package v1.2.3 (available: v1.3.0, published 2 days ago)
similar to what we do with msrv, to give users context as to why an old version may be used.
|
Finally was able to sit down and go through the commits. Might also be good to go through the final test state after things get updated. |
| /// Creation time of this config, used to output the total build time | ||
| creation_time: Instant, | ||
| /// Wall-clock time of this cargo invocation. | ||
| /// | ||
| /// Currently used as the reference time for `min-publish-age` and `-Zbuild-analysis`. | ||
| invocation_time: jiff::Timestamp, |
There was a problem hiding this comment.
How different are creation_time and invocation_time?
View all comments
What does this PR try to resolve?
This implements RFC 3923 min-publish-age (rust-lang/rfcs#3923, #17009).
The policy lives entirely at the resolver layer. A
PublishAgePolicyis built before resolution and applied wherever a version filer needs to be appliedWhere the filter is applied
Because the policy is enforced in the resolver, every code path that selects a version outside the resolver must apply it explicitly. Full audit:
dep_cache::query)cargo add(get_latest_dependency)Cargo.toml)cargo addpath/git (select_package)cargo addfeatures (populate_available_features)cargo install(select_dep_pkg)cargo update --breaking(upgrade_dependency)Cargo.toml)cargo updatereport (report_latest)get_updates)cargo info(query_summaries)How to test and review this PR?
29 tests in
tests/testsuite/min_publish_age.rscover:"0"disabling[patch]preservation of too-new versionscargo install/cargo add/cargo update --breakingselection pathsOpen questions
version 1.99.0 is too new (published 2d ago)message currenly show the time span in always down to minute, and up to day precise. It is annoying that you would get something like2d 8h 23mlong. We might want to round it more for better UX. I personally would leave it for follow-up.registry.min-publish-age/registries.min-publish-ageprecedence rule basically follows howregistry.tokenis handle. Forregistry.min-publish-agewe only check if the default registyr is crates.io for consistency withtoken, though it is open to change.