Always include Cargo.lock in published crates by sdroege · Pull Request #14815 · rust-lang/cargo

sdroege · 2024-11-13T10:20:55Z

What does this PR try to resolve?

Originally it was only included for packages that have executables or examples for cargo install, however this causes inconsistencies and is kind of unexpected nowadays, e.g. with cdylib crates.

Including it always only slightly increases the crate size and allows for all crates to know a set of dependency versions that were working, which can make regression tracking easier.

Fixes #13447

How should we test and review this PR?

The existing tests are covering this change in all kinds of various already, and one test that previously asserted that there is no Cargo.lock for library crates was changed to explicitly check for the new behaviour.

rustbot · 2024-11-13T10:21:01Z

r? @epage

rustbot has assigned @epage.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

tests/testsuite/registry_auth.rs

weihanglo · 2024-11-13T20:32:11Z

tests/testsuite/package.rs

-[VERIFYING] level1 v0.0.1 ([ROOT]/foo/level1)
 [UPDATING] crates.io index
-[ERROR] failed to verify package tarball
+[ERROR] failed to prepare local package for uploading


This is nice! It failed earlier before the actual packaging happened, so users don't need to pay unnecessary stuff.

Originally it was only included for packages that have executables or examples for `cargo install`, however this causes inconsistencies and is kind of unexpected nowadays, e.g. with cdylib crates. Including it always only slightly increases the crate size and allows for all crates to know a set of dependency versions that were working, which can make regression tracking easier. Fixes rust-lang#13447

weihanglo · 2024-11-14T14:39:50Z

fatal: unable to access 'https://github.com/rust-lang/crates.io-index-archive/': Failed to connect to github.com port 443 after 134663 ms: Connection timed out

Network failure. Re-queue.

Update cargo 15 commits in 4a2d8dc636445b276288543882e076f254b3ae95..69e595908e2c420e7f0d1be34e6c5b984c8cfb84 2024-11-09 19:10:33 +0000 to 2024-11-16 01:26:11 +0000 - refactor(fingerprint): Track the intent for each use of `UnitHash` (rust-lang/cargo#14826) - fix(toml): Update frontmatter parser for RFC 3503 (rust-lang/cargo#14792) - docs(unstable): Move -Zwarnings from stable to unstable section (rust-lang/cargo#14827) - Simplify English used in guide (rust-lang/cargo#14825) - feat(resolver): Stabilize resolver v3 (rust-lang/cargo#14754) - docs: Clean up doc comments (rust-lang/cargo#14823) - fix(remove): On error, suggest other dependencies (rust-lang/cargo#14818) - Always include Cargo.lock in published crates (rust-lang/cargo#14815) - fix(build-rs)!: Updates from an audit (rust-lang/cargo#14817) - feat(rustdoc): diplay env vars in extra verbose mode (rust-lang/cargo#14812) - Migrate build-rs to the Cargo repo (rust-lang/cargo#14786) - chore(ci): Check for clippy `correctness` (rust-lang/cargo#14796) - git: do not validate submodules of fresh checkouts (rust-lang/cargo#14605) - refactor: clone-on-write when needed for InternedString (rust-lang/cargo#14808) - fix(docs): typo in cargo-fmt.md (rust-lang/cargo#14805)

glandium · 2025-01-09T23:04:27Z

FWIW, this had the side effect of adding Cargo.lock to vendored crates too, irrespective of whether the crate published on crates.io has one.

weihanglo · 2025-01-13T21:23:20Z

The other side effect / regression of this change: #15059

This was overlooked in rust-lang#14815.

### What does this PR try to resolve? This was overlooked in #14815. ### How should we test and review this PR? ``` cargo build target/debug/cargo help package # and read the manpage ```

### What does this PR try to resolve? This was changed in <#14815> since 1.84 but we missed some doc updates.

@NoisyCoil

### What does this PR try to resolve? Fixes #15059 Fixes #15159 This provides an escape hatch `--exclude-lockfile`for uncommon workflows that don't verify (`--no-verify` is passed) the build with their unpublished packages In effect, this takes the heuristic removed in #14815 and replaces it with a flag When `--exclude-lockfile` is enabled, `cargo package` will not verify the lock file if present, nor will it generate a new one if absent. Cargo.lock will not be included in the resulting tarball. Together with `--no-verify`, this flag decouples packaging from checking the registry index. While this is useful for some non-normal workflows that requires to assemble packages having unpublished dependencies. It is recommended to use `-Zpackage-workspace` to package the entire workspace, instead of opting out lockfile. ### How should we test and review this PR? The first commit was stolen from <NoisyCoil@1a104b5> (credit to @NoisyCoil!) The second added two failing cases we observed in #15059. ### Additional information

Starting from rust 1.84.0 (cargo 1.84.0), published crates now always include a Cargo.lock file. Originally it was only included for packages that have executables or examples for use with cargo install. see [1] This behaviour change alters the contents of the .tar.gz archives, which causes SHA256 hash mistmatches when trying to build Rust packages. Example build failure with bat-0.24.0: ERROR: while checking hashes from package/bat/bat.hash ERROR: bat-0.24.0-cargo2.tar.gz has wrong sha256 hash: ERROR: expected: 45fcdd6076dc1b45698a7b6c0f4d1f5d9ae676f3ca3b155402ad24680d5b4df6 ERROR: got : 28b302b1aa325221796d4ebb25bacab19a8927ef32f4d56a965b32a7b1c102fc After using the ne hash to download the new archive tar.gz, we have the difference between the old archive and the new one using diffoscope: │ │ --rw-r--r-- 0 0 0 1529 2023-10-11 17:14:12.000000 bat-0.24.0/VENDOR/bincode/.cargo-checksum.json │ │ +-rw-r--r-- 0 0 0 1609 2023-10-11 17:14:12.000000 bat-0.24.0/VENDOR/bincode/.cargo-checksum.json │ │ +-rw-r--r-- 0 0 0 1766 2023-10-11 17:14:12.000000 bat-0.24.0/VENDOR/bincode/Cargo.lock │ │ -rw-r--r-- 0 0 0 1388 2023-10-11 17:14:12.000000 bat-0.24.0/VENDOR/bincode/Cargo.toml We can see that Cargo.lock has been added. To avoid hash mismatch issues and to clearly mark archives generated with the new Cargo behavior, we migrate the naming from 'cargo2.tar.gz' to 'cargo3.tar.gz'. We did not find any alternative to disable this new cargo-publish behavior, so this change is necessary to allow updating the hashes of Cargo-fetched packages. [1] rust-lang/cargo#14815 https://doc.rust-lang.org/nightly/cargo/CHANGELOG.html Signed-off-by: El Mehdi YOUNES <elmehdi.younes@smile.fr> Signed-off-by: Thomas Petazzoni <thomas.petazzoni@bootlin.com>

rustbot assigned epage Nov 13, 2024

rustbot added Command-package S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Nov 13, 2024

sdroege commented Nov 13, 2024

View reviewed changes

tests/testsuite/registry_auth.rs Show resolved Hide resolved

sdroege mentioned this pull request Nov 13, 2024

Heuristics to include Cargo.lock or not in a package are suboptimal #13447

Closed

weihanglo approved these changes Nov 13, 2024

View reviewed changes

sdroege force-pushed the always-include-cargo-lock branch from b806e0a to 5db2605 Compare November 13, 2024 16:24

weihanglo approved these changes Nov 13, 2024

View reviewed changes

sdroege force-pushed the always-include-cargo-lock branch from 5db2605 to dd698ff Compare November 14, 2024 07:06

weihanglo added this pull request to the merge queue Nov 14, 2024

github-merge-queue bot removed this pull request from the merge queue due to failed status checks Nov 14, 2024

weihanglo added this pull request to the merge queue Nov 14, 2024

Merged via the queue into rust-lang:master with commit cfea065 Nov 14, 2024

sdroege deleted the always-include-cargo-lock branch November 14, 2024 15:18

weihanglo mentioned this pull request Nov 16, 2024

Update cargo rust-lang/rust#133094

Merged

FabijanC mentioned this pull request Nov 27, 2024

v3.7.0 is a breaking version paritytech/parity-scale-codec#656

Closed

weihanglo added this to the 1.84.0 milestone Nov 28, 2024

This was referenced Dec 13, 2024

cargo publish of a [lib] fails when dependencies exist newer than my toolchain version #13306

Closed

packaging: omit generated Cargo.lock if binaries and examples are package-excluded #11557

Closed

weihanglo added a commit to weihanglo/cargo that referenced this pull request Jan 15, 2025

docs(cargo-package): alwasy include the lockfile

7fecabc

This was overlooked in rust-lang#14815.

weihanglo mentioned this pull request Jan 15, 2025

docs(cargo-package): alwasy include the lockfile #15067

Merged

weihanglo added a commit to weihanglo/cargo that referenced this pull request Jan 15, 2025

docs(cargo-package): alwasy include the lockfile

71ec5d8

This was overlooked in rust-lang#14815.

weihanglo added a commit to weihanglo/cargo that referenced this pull request Jan 16, 2025

docs(cargo-package): alwasy include the lockfile

662394e

This was overlooked in rust-lang#14815.

epage mentioned this pull request Feb 26, 2025

feat(package): add --exclude-lockfile flag #15234

Merged

weihanglo mentioned this pull request Mar 3, 2025

docs: lockfile is always included since 1.84 #15257

Merged

github-merge-queue bot pushed a commit that referenced this pull request Mar 3, 2025

docs: lockfile is always included since 1.84 (#15257)

abe461c

### What does this PR try to resolve? This was changed in <#14815> since 1.84 but we missed some doc updates.

epage mentioned this pull request Mar 4, 2025

Cargo publish --locked doesn't use the lock file #15264

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Always include Cargo.lock in published crates#14815

Always include Cargo.lock in published crates#14815
weihanglo merged 1 commit intorust-lang:masterfrom
sdroege:always-include-cargo-lock

sdroege commented Nov 13, 2024

Uh oh!

rustbot commented Nov 13, 2024

Uh oh!

Uh oh!

Uh oh!

weihanglo Nov 13, 2024

Uh oh!

Uh oh!

weihanglo commented Nov 14, 2024

Uh oh!

glandium commented Jan 9, 2025

Uh oh!

weihanglo commented Jan 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

sdroege commented Nov 13, 2024

What does this PR try to resolve?

How should we test and review this PR?

Uh oh!

rustbot commented Nov 13, 2024

Uh oh!

Uh oh!

Uh oh!

weihanglo Nov 13, 2024

Choose a reason for hiding this comment

Uh oh!

Uh oh!

weihanglo commented Nov 14, 2024

Uh oh!

glandium commented Jan 9, 2025

Uh oh!

weihanglo commented Jan 13, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants