uv-distribution: include all wheels in distribution types by BurntSushi · Pull Request #3595 · astral-sh/uv

BurntSushi · 2024-05-14T22:45:10Z

Our current flow of data from "simple registry package" to "final
resolved distribution" goes through a number of types:

SimpleMetadata is the API response from a registry that includes all
published versions for a package. Each version has an assortment of metadata
associated with it.
VersionFiles is the aforementioned metadata. It is split in two: a
group of files for source distributions and a group of files for wheels.
PrioritizedDist collects a subset of the files from VersionFiles
to form a selection of the "best" sdist and the "best" wheel for the
current environment.
CompatibleDist is created from a borrowed PrioritizedDist that,
perhaps among other things, encapsulates the decision of whether to pick
an sdist or a wheel. (This decision depends both on compatibility and
the action being performed. e.g., When doing installation, a
CompatibleDist will sometimes select an sdist over a wheel.)
ResolvedDistRef is like a ResolvedDist, but borrows a Dist.
ResolvedDist is the almost-final-form of a distribution in a
resolution and is created from a ResolvedDistRef.
AnnotatedResolvedDist is a new data type that is the actual final
form of a distribution that a universal lock file cares about. It
bundles a ResolvedDist with some metadata needed to generate a lock
file.

One of the requirements of a universal lock file is that we include all
wheels (and maybe all source distributions? but at least one if it's
present) associated with a distribution. But the above flow of data (in
the step from VersionFiles to PrioritizedDist) drops all wheels
except for the best one.

To remedy this, in this PR, we rejigger PrioritizedDist,
CompatibleDist and ResolvedDistRef so that all wheel data is
preserved. And when a ResolvedDistRef is finally turned into a
ResolvedDist, we copy all of the wheel data. And finally, we adjust
the Lock constructor to read this new data and include it in the lock
file. To make this work, we also modify RegistryBuiltDist so that it
can contain one or more wheels instead of just one.

One shortcoming here (called out in the code as a FIXME) is that if a
source distribution is selected as the "best" thing to use (perhaps
there are no compatible wheels), then the wheels won't end up in the
lock file. I plan to fix this in a follow-up PR.

We also aren't totally consistent on source distribution naming.
Sometimes we use sdist. Sometimes source. Sometimes source_dist.
I think it'd be nice to just use sdist everywhere, but I do prefer
the type names to be SourceDist. And sometimes you want function
names to match the type names (i.e., from_source_dist), which in turn
leads to an appearance of inconsistency. I'm open to ideas.

Closes #3351

codspeed-hq · 2024-05-15T15:29:39Z

CodSpeed Performance Report

Merging #3595 will degrade performances by 10.28%

_{Comparing ag/propagate-wheels (7308f17) with main (8d68d45)}

Summary

❌ 1 regressions
✅ 11 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

	Benchmark	`main`	`ag/propagate-wheels`	Change
❌	`resolve_warm_jupyter`	371 ms	413.5 ms	-10.28%

charliermarsh · 2024-05-15T15:32:50Z

Do you think that's a real regression?

BurntSushi · 2024-05-15T15:46:20Z

Do you think that's a real regression?

It's possible. The regression is highlighting the drop impl for ResolvedDist, which is definitely an area that has been touched here.

The various distribution data is cloned occasionally in full. I don't have a complete picture yet on how to eliminate those clones, but we probably should, especially since we're carrying around more data now. It's possible that fixing this is a gnarly refactor though.

I'll take a look and see if there's a quicker fix.

BurntSushi · 2024-05-15T15:55:38Z

I can't measure a regression locally. This PR actually seems to be consistently a hair faster:

$ hyperfine -w10 -r1000 'uv-main pip compile ~/astral/tmp/reqs/jupyter.in' 'uv pip compile ~/astral/tmp/reqs/jupyter.in'
Benchmark 1: uv-main pip compile ~/astral/tmp/reqs/jupyter.in
  Time (mean ± σ):      33.9 ms ±   8.8 ms    [User: 28.5 ms, System: 44.0 ms]
  Range (min … max):    19.2 ms …  75.0 ms    1000 runs

Benchmark 2: uv pip compile ~/astral/tmp/reqs/jupyter.in
  Time (mean ± σ):      32.8 ms ±   8.6 ms    [User: 27.4 ms, System: 42.6 ms]
  Range (min … max):    19.9 ms …  73.2 ms    1000 runs

Summary
  uv pip compile ~/astral/tmp/reqs/jupyter.in ran
    1.04 ± 0.38 times faster than uv-main pip compile ~/astral/tmp/reqs/jupyter.in

charliermarsh · 2024-05-15T15:58:21Z

Okay, that's fine with me then.

charliermarsh · 2024-05-15T15:59:26Z

Thanks for investigating.

BurntSushi · 2024-05-15T16:05:11Z

Aye. And looking more at our distribution types to see if we can avoid copying them, it looks a little tricky. I think the main problem is that VersionMap is the owner of PrioritizedDist and PrioritizedDist is ultimately the owner of each RegistryBuiltDist (which is now a list of wheels and an optional sdist). When we do the PrioritizedDist -> CandidateDist -> ResolvedDistRef -> ResolvedDist sequence, it's the ResolvedDistRef -> ResolvedDist conversion that does the copy. The VersionMap data structure isn't really designed to give up ownership, and I don't see an obvious way to do it. That in turn probably means that the key to avoiding a copy is reference counting. But that means injecting an Arc into types like RegistryBuiltDist. I'm not quite sure that's worth doing.

Anyway, I agree. I also saw mention on Discord that resolve_warm_jupyter was noisy in particular. So I think I'm also okay with plowing ahead for now.

…to it Following from #3595, we'd like wheels to make their way into the lock file even if the current environment selects an sdist. With #3595, this didn't happen: $ cargo run -p uv -- pip compile -p3.10 <(echo psycopg2) --unstable-uv-lock-file $ cat uv.lock version = 1 [[distribution]] name = "psycopg2" version = "2.9.9" source = "registry+https://pypi.org/simple" [distribution.sdist] url = "https://files.pythonhosted.org/packages/c9/5e/dc6acaf46d78979d6b03458b7a1618a68e152a6776fce95daac5e0f0301b/psycopg2-2.9.9.tar.gz" hash = "sha256:d1454bde93fb1e224166811694d600e746430c006fbb031ea06ecc2ea41bf156" The above example uses `psycopg2`, which has an sdist and wheels only on Windows. Since I ran the above on Linux, an sdist was selected. But no wheels appeared in the lock file. With this PR, wheels are now correctly plumbed through: $ cargo run -p uv -- pip compile -p3.10 <(echo psycopg2) --unstable-uv-lock-file $ cat uv.lock version = 1 [[distribution]] name = "psycopg2" version = "2.9.9" source = "registry+https://pypi.org/simple" [distribution.sdist] url = "https://files.pythonhosted.org/packages/c9/5e/dc6acaf46d78979d6b03458b7a1618a68e152a6776fce95daac5e0f0301b/psycopg2-2.9.9.tar.gz" hash = "sha256:d1454bde93fb1e224166811694d600e746430c006fbb031ea06ecc2ea41bf156" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/a2/14/2767d963915f957c07f5d4c3d9c5c9a407415289f5cde90b82cb3e8c2a12/psycopg2-2.9.9-cp310-cp310-win32.whl" hash = "sha256:38a8dcc6856f569068b47de286b472b7c473ac7977243593a288ebce0dc89516" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/bc/bc/6572dec6834e779668421e25f8812a872d978e241f85491a5e4dda606a98/psycopg2-2.9.9-cp310-cp310-win_amd64.whl" hash = "sha256:426f9f29bde126913a20a96ff8ce7d73fd8a216cfb323b1f04da402d452853c3" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/91/2c/1fc5b9d33cd248c548ba19f2cef8e89cabaafab9858a602868a592cdc1b0/psycopg2-2.9.9-cp311-cp311-win32.whl" hash = "sha256:ade01303ccf7ae12c356a5e10911c9e1c51136003a9a1d92f7aa9d010fb98372" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/37/2c/5133dd3183a3bd82371569f0dd783e6927672de7e671b278ce248810b7f7/psycopg2-2.9.9-cp311-cp311-win_amd64.whl" hash = "sha256:121081ea2e76729acfb0673ff33755e8703d45e926e416cb59bae3a86c6a4981" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/13/13/f74ffe6b6f58822e807c70391dc5679a53feb92ce119ccb8a6546c3fb893/psycopg2-2.9.9-cp312-cp312-win32.whl" hash = "sha256:d735786acc7dd25815e89cc4ad529a43af779db2e25aa7c626de864127e5a024" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/58/4b/c4a26e191882b60150bfcb639e416524ae7f8249ab7ee854fb5247f16c40/psycopg2-2.9.9-cp312-cp312-win_amd64.whl" hash = "sha256:a7653d00b732afb6fc597e29c50ad28087dcb4fbfb28e86092277a559ae4e693" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/2b/77/ffeb9ac356b3d99d97ca681bf0d0aa74f6d1d8c2ce0d6c4f2f34e396dbc0/psycopg2-2.9.9-cp37-cp37m-win32.whl" hash = "sha256:5e0d98cade4f0e0304d7d6f25bbfbc5bd186e07b38eac65379309c4ca3193efa" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/be/a7/0a39176d369a8289191f3d327139cfb4923dcedcfd7105774e57996f63cd/psycopg2-2.9.9-cp37-cp37m-win_amd64.whl" hash = "sha256:7e2dacf8b009a1c1e843b5213a87f7c544b2b042476ed7755be813eaf4e8347a" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/1f/78/86b90d30c4e02e88379184ade34c2fd4883a4d3e420cc3c0f6da2b8f3a9a/psycopg2-2.9.9-cp38-cp38-win32.whl" hash = "sha256:ff432630e510709564c01dafdbe996cb552e0b9f3f065eb89bdce5bd31fabf4c" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/8e/e8/c439b378efc9f2d0fd1fd5f66b03cb9ed41423f179997a935f10374f3c0d/psycopg2-2.9.9-cp38-cp38-win_amd64.whl" hash = "sha256:bac58c024c9922c23550af2a581998624d6e02350f4ae9c5f0bc642c633a2d5e" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/6b/a8/5080c0e61a3b393a379ea2fa93402135c73baffcd5f08b9503e508aac116/psycopg2-2.9.9-cp39-cp39-win32.whl" hash = "sha256:c92811b2d4c9b6ea0285942b2e7cac98a59e166d59c588fe5cfe1eda58e72d59" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/f8/ec/ec73fe66d4317db006a38ebafbde02cb7e1d727ed65f5bbe54efb191d9e6/psycopg2-2.9.9-cp39-cp39-win_amd64.whl" hash = "sha256:de80739447af31525feddeb8effd640782cf5998e1a4e9192ebdf829717e3913" Ref #3351

BurntSushi · 2024-05-15T16:30:07Z

One shortcoming here (called out in the code as a FIXME) is that if a
source distribution is selected as the "best" thing to use (perhaps
there are no compatible wheels), then the wheels won't end up in the
lock file. I plan to fix this in a follow-up PR.

I did this in #3610.

(Also, this PR and #3610 are ready for review.)

charliermarsh · 2024-05-15T16:55:22Z

You got it, I will review today.

crates/distribution-types/src/prioritized_distribution.rs

crates/distribution-types/src/resolved.rs

crates/uv-resolver/src/lock.rs

crates/distribution-types/src/lib.rs

This is still a WIP.

Our current flow of data from "simple registry package" to "final resolved distribution" goes through a number of types: * `SimpleMetadata` is the API response from a registry that includes all published versions for a package. Each version has an assortment of metadata associated with it. * `VersionFiles` is the aforementioned metadata. It is split in two: a group of files for source distributions and a group of files for wheels. * `PrioritizedDist` collects a subset of the files from `VersionFiles` to form a selection of the "best" sdist and the "best" wheel for the current environment. * `CompatibleDist` is created from a borrowed `PrioritizedDist` that, perhaps among other things, encapsulates the decision of whether to pick an sdist or a wheel. (This decision depends both on compatibility and the action being performed. e.g., When doing installation, a `CompatibleDist` will sometimes select an sdist over a wheel.) * `ResolvedDistRef` is like a `ResolvedDist`, but borrows a `Dist`. * `ResolvedDist` is the almost-final-form of a distribution in a resolution and is created from a `ResolvedDistRef`. * `AnnotatedResolvedDist` is a new data type that is the actual final form of a distribution that a universal lock file cares about. It bundles a `ResolvedDist` with some metadata needed to generate a lock file. One of the requirements of a lock file is that we include all wheels (and maybe all source distributions? but at least one if it's present) associated with a distribution. But the above flow of data (in the step from `VersionFiles` to `PrioritizedDist`) drops all wheels except for the best one. To remedy this, in this commit, we rejigger `PrioritizedDist`, `CompatibleDist` and `ResolvedDistRef` so that all wheel data is preserved. And when a `ResolvedDistRef` is finally turned into a `ResolvedDist`, we copy all of the wheel data. And finally, we adjust the `Lock` constructor to read this new data and include it in the lock file. One shortcoming here (called out in the code as a FIXME) is that if a source distribution is selected as the "best" thing to use (perhaps there are no compatible wheels), then the wheels won't end up in the lock file. I plan to fix this in a follow-up PR. Closes #3351

…to it Following from #3595, we'd like wheels to make their way into the lock file even if the current environment selects an sdist. With #3595, this didn't happen: $ cargo run -p uv -- pip compile -p3.10 <(echo psycopg2) --unstable-uv-lock-file $ cat uv.lock version = 1 [[distribution]] name = "psycopg2" version = "2.9.9" source = "registry+https://pypi.org/simple" [distribution.sdist] url = "https://files.pythonhosted.org/packages/c9/5e/dc6acaf46d78979d6b03458b7a1618a68e152a6776fce95daac5e0f0301b/psycopg2-2.9.9.tar.gz" hash = "sha256:d1454bde93fb1e224166811694d600e746430c006fbb031ea06ecc2ea41bf156" The above example uses `psycopg2`, which has an sdist and wheels only on Windows. Since I ran the above on Linux, an sdist was selected. But no wheels appeared in the lock file. With this PR, wheels are now correctly plumbed through: $ cargo run -p uv -- pip compile -p3.10 <(echo psycopg2) --unstable-uv-lock-file $ cat uv.lock version = 1 [[distribution]] name = "psycopg2" version = "2.9.9" source = "registry+https://pypi.org/simple" [distribution.sdist] url = "https://files.pythonhosted.org/packages/c9/5e/dc6acaf46d78979d6b03458b7a1618a68e152a6776fce95daac5e0f0301b/psycopg2-2.9.9.tar.gz" hash = "sha256:d1454bde93fb1e224166811694d600e746430c006fbb031ea06ecc2ea41bf156" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/a2/14/2767d963915f957c07f5d4c3d9c5c9a407415289f5cde90b82cb3e8c2a12/psycopg2-2.9.9-cp310-cp310-win32.whl" hash = "sha256:38a8dcc6856f569068b47de286b472b7c473ac7977243593a288ebce0dc89516" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/bc/bc/6572dec6834e779668421e25f8812a872d978e241f85491a5e4dda606a98/psycopg2-2.9.9-cp310-cp310-win_amd64.whl" hash = "sha256:426f9f29bde126913a20a96ff8ce7d73fd8a216cfb323b1f04da402d452853c3" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/91/2c/1fc5b9d33cd248c548ba19f2cef8e89cabaafab9858a602868a592cdc1b0/psycopg2-2.9.9-cp311-cp311-win32.whl" hash = "sha256:ade01303ccf7ae12c356a5e10911c9e1c51136003a9a1d92f7aa9d010fb98372" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/37/2c/5133dd3183a3bd82371569f0dd783e6927672de7e671b278ce248810b7f7/psycopg2-2.9.9-cp311-cp311-win_amd64.whl" hash = "sha256:121081ea2e76729acfb0673ff33755e8703d45e926e416cb59bae3a86c6a4981" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/13/13/f74ffe6b6f58822e807c70391dc5679a53feb92ce119ccb8a6546c3fb893/psycopg2-2.9.9-cp312-cp312-win32.whl" hash = "sha256:d735786acc7dd25815e89cc4ad529a43af779db2e25aa7c626de864127e5a024" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/58/4b/c4a26e191882b60150bfcb639e416524ae7f8249ab7ee854fb5247f16c40/psycopg2-2.9.9-cp312-cp312-win_amd64.whl" hash = "sha256:a7653d00b732afb6fc597e29c50ad28087dcb4fbfb28e86092277a559ae4e693" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/2b/77/ffeb9ac356b3d99d97ca681bf0d0aa74f6d1d8c2ce0d6c4f2f34e396dbc0/psycopg2-2.9.9-cp37-cp37m-win32.whl" hash = "sha256:5e0d98cade4f0e0304d7d6f25bbfbc5bd186e07b38eac65379309c4ca3193efa" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/be/a7/0a39176d369a8289191f3d327139cfb4923dcedcfd7105774e57996f63cd/psycopg2-2.9.9-cp37-cp37m-win_amd64.whl" hash = "sha256:7e2dacf8b009a1c1e843b5213a87f7c544b2b042476ed7755be813eaf4e8347a" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/1f/78/86b90d30c4e02e88379184ade34c2fd4883a4d3e420cc3c0f6da2b8f3a9a/psycopg2-2.9.9-cp38-cp38-win32.whl" hash = "sha256:ff432630e510709564c01dafdbe996cb552e0b9f3f065eb89bdce5bd31fabf4c" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/8e/e8/c439b378efc9f2d0fd1fd5f66b03cb9ed41423f179997a935f10374f3c0d/psycopg2-2.9.9-cp38-cp38-win_amd64.whl" hash = "sha256:bac58c024c9922c23550af2a581998624d6e02350f4ae9c5f0bc642c633a2d5e" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/6b/a8/5080c0e61a3b393a379ea2fa93402135c73baffcd5f08b9503e508aac116/psycopg2-2.9.9-cp39-cp39-win32.whl" hash = "sha256:c92811b2d4c9b6ea0285942b2e7cac98a59e166d59c588fe5cfe1eda58e72d59" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/f8/ec/ec73fe66d4317db006a38ebafbde02cb7e1d727ed65f5bbe54efb191d9e6/psycopg2-2.9.9-cp39-cp39-win_amd64.whl" hash = "sha256:de80739447af31525feddeb8effd640782cf5998e1a4e9192ebdf829717e3913" Ref #3351

BurntSushi · 2024-05-15T19:07:14Z

The perf regression remains (and actually got bigger), but #3610, which builds on this one, has no regression. And there's nothing in #3610 that ought to have impacted things here. I re-ran my hyperfine benchmark from above to double check:

[andrew@duff uv]$ hyperfine -w10 -r1000 'uv-main pip compile ~/astral/tmp/reqs/jupyter.in' 'uv pip compile ~/astral/tmp/reqs/jupyter.in' ; A bart
Benchmark 1: uv-main pip compile ~/astral/tmp/reqs/jupyter.in
  Time (mean ± σ):      32.4 ms ±   8.7 ms    [User: 26.9 ms, System: 43.4 ms]
  Range (min … max):    19.1 ms …  66.8 ms    1000 runs

Benchmark 2: uv pip compile ~/astral/tmp/reqs/jupyter.in
  Time (mean ± σ):      32.5 ms ±   8.1 ms    [User: 27.7 ms, System: 42.6 ms]
  Range (min … max):    19.8 ms …  66.4 ms    1000 runs

Summary
  uv-main pip compile ~/astral/tmp/reqs/jupyter.in ran
    1.00 ± 0.37 times faster than uv pip compile ~/astral/tmp/reqs/jupyter.in

So I'm going to go ahead and merge this. It looks like we might have a noisy benchmark.

…to it Following from #3595, we'd like wheels to make their way into the lock file even if the current environment selects an sdist. With #3595, this didn't happen: $ cargo run -p uv -- pip compile -p3.10 <(echo psycopg2) --unstable-uv-lock-file $ cat uv.lock version = 1 [[distribution]] name = "psycopg2" version = "2.9.9" source = "registry+https://pypi.org/simple" [distribution.sdist] url = "https://files.pythonhosted.org/packages/c9/5e/dc6acaf46d78979d6b03458b7a1618a68e152a6776fce95daac5e0f0301b/psycopg2-2.9.9.tar.gz" hash = "sha256:d1454bde93fb1e224166811694d600e746430c006fbb031ea06ecc2ea41bf156" The above example uses `psycopg2`, which has an sdist and wheels only on Windows. Since I ran the above on Linux, an sdist was selected. But no wheels appeared in the lock file. With this PR, wheels are now correctly plumbed through: $ cargo run -p uv -- pip compile -p3.10 <(echo psycopg2) --unstable-uv-lock-file $ cat uv.lock version = 1 [[distribution]] name = "psycopg2" version = "2.9.9" source = "registry+https://pypi.org/simple" [distribution.sdist] url = "https://files.pythonhosted.org/packages/c9/5e/dc6acaf46d78979d6b03458b7a1618a68e152a6776fce95daac5e0f0301b/psycopg2-2.9.9.tar.gz" hash = "sha256:d1454bde93fb1e224166811694d600e746430c006fbb031ea06ecc2ea41bf156" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/a2/14/2767d963915f957c07f5d4c3d9c5c9a407415289f5cde90b82cb3e8c2a12/psycopg2-2.9.9-cp310-cp310-win32.whl" hash = "sha256:38a8dcc6856f569068b47de286b472b7c473ac7977243593a288ebce0dc89516" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/bc/bc/6572dec6834e779668421e25f8812a872d978e241f85491a5e4dda606a98/psycopg2-2.9.9-cp310-cp310-win_amd64.whl" hash = "sha256:426f9f29bde126913a20a96ff8ce7d73fd8a216cfb323b1f04da402d452853c3" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/91/2c/1fc5b9d33cd248c548ba19f2cef8e89cabaafab9858a602868a592cdc1b0/psycopg2-2.9.9-cp311-cp311-win32.whl" hash = "sha256:ade01303ccf7ae12c356a5e10911c9e1c51136003a9a1d92f7aa9d010fb98372" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/37/2c/5133dd3183a3bd82371569f0dd783e6927672de7e671b278ce248810b7f7/psycopg2-2.9.9-cp311-cp311-win_amd64.whl" hash = "sha256:121081ea2e76729acfb0673ff33755e8703d45e926e416cb59bae3a86c6a4981" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/13/13/f74ffe6b6f58822e807c70391dc5679a53feb92ce119ccb8a6546c3fb893/psycopg2-2.9.9-cp312-cp312-win32.whl" hash = "sha256:d735786acc7dd25815e89cc4ad529a43af779db2e25aa7c626de864127e5a024" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/58/4b/c4a26e191882b60150bfcb639e416524ae7f8249ab7ee854fb5247f16c40/psycopg2-2.9.9-cp312-cp312-win_amd64.whl" hash = "sha256:a7653d00b732afb6fc597e29c50ad28087dcb4fbfb28e86092277a559ae4e693" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/2b/77/ffeb9ac356b3d99d97ca681bf0d0aa74f6d1d8c2ce0d6c4f2f34e396dbc0/psycopg2-2.9.9-cp37-cp37m-win32.whl" hash = "sha256:5e0d98cade4f0e0304d7d6f25bbfbc5bd186e07b38eac65379309c4ca3193efa" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/be/a7/0a39176d369a8289191f3d327139cfb4923dcedcfd7105774e57996f63cd/psycopg2-2.9.9-cp37-cp37m-win_amd64.whl" hash = "sha256:7e2dacf8b009a1c1e843b5213a87f7c544b2b042476ed7755be813eaf4e8347a" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/1f/78/86b90d30c4e02e88379184ade34c2fd4883a4d3e420cc3c0f6da2b8f3a9a/psycopg2-2.9.9-cp38-cp38-win32.whl" hash = "sha256:ff432630e510709564c01dafdbe996cb552e0b9f3f065eb89bdce5bd31fabf4c" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/8e/e8/c439b378efc9f2d0fd1fd5f66b03cb9ed41423f179997a935f10374f3c0d/psycopg2-2.9.9-cp38-cp38-win_amd64.whl" hash = "sha256:bac58c024c9922c23550af2a581998624d6e02350f4ae9c5f0bc642c633a2d5e" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/6b/a8/5080c0e61a3b393a379ea2fa93402135c73baffcd5f08b9503e508aac116/psycopg2-2.9.9-cp39-cp39-win32.whl" hash = "sha256:c92811b2d4c9b6ea0285942b2e7cac98a59e166d59c588fe5cfe1eda58e72d59" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/f8/ec/ec73fe66d4317db006a38ebafbde02cb7e1d727ed65f5bbe54efb191d9e6/psycopg2-2.9.9-cp39-cp39-win_amd64.whl" hash = "sha256:de80739447af31525feddeb8effd640782cf5998e1a4e9192ebdf829717e3913" Ref #3351

…to it (#3610) Following from #3595, we'd like wheels to make their way into the lock file even if the current environment selects an sdist. With #3595, this didn't happen: $ cargo run -p uv -- pip compile -p3.10 <(echo psycopg2) --unstable-uv-lock-file $ cat uv.lock version = 1 [[distribution]] name = "psycopg2" version = "2.9.9" source = "registry+https://pypi.org/simple" [distribution.sdist] url = "https://files.pythonhosted.org/packages/c9/5e/dc6acaf46d78979d6b03458b7a1618a68e152a6776fce95daac5e0f0301b/psycopg2-2.9.9.tar.gz" hash = "sha256:d1454bde93fb1e224166811694d600e746430c006fbb031ea06ecc2ea41bf156" The above example uses `psycopg2`, which has an sdist and wheels only on Windows. Since I ran the above on Linux, an sdist was selected. But no wheels appeared in the lock file. With this PR, wheels are now correctly plumbed through: $ cargo run -p uv -- pip compile -p3.10 <(echo psycopg2) --unstable-uv-lock-file $ cat uv.lock version = 1 [[distribution]] name = "psycopg2" version = "2.9.9" source = "registry+https://pypi.org/simple" [distribution.sdist] url = "https://files.pythonhosted.org/packages/c9/5e/dc6acaf46d78979d6b03458b7a1618a68e152a6776fce95daac5e0f0301b/psycopg2-2.9.9.tar.gz" hash = "sha256:d1454bde93fb1e224166811694d600e746430c006fbb031ea06ecc2ea41bf156" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/a2/14/2767d963915f957c07f5d4c3d9c5c9a407415289f5cde90b82cb3e8c2a12/psycopg2-2.9.9-cp310-cp310-win32.whl" hash = "sha256:38a8dcc6856f569068b47de286b472b7c473ac7977243593a288ebce0dc89516" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/bc/bc/6572dec6834e779668421e25f8812a872d978e241f85491a5e4dda606a98/psycopg2-2.9.9-cp310-cp310-win_amd64.whl" hash = "sha256:426f9f29bde126913a20a96ff8ce7d73fd8a216cfb323b1f04da402d452853c3" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/91/2c/1fc5b9d33cd248c548ba19f2cef8e89cabaafab9858a602868a592cdc1b0/psycopg2-2.9.9-cp311-cp311-win32.whl" hash = "sha256:ade01303ccf7ae12c356a5e10911c9e1c51136003a9a1d92f7aa9d010fb98372" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/37/2c/5133dd3183a3bd82371569f0dd783e6927672de7e671b278ce248810b7f7/psycopg2-2.9.9-cp311-cp311-win_amd64.whl" hash = "sha256:121081ea2e76729acfb0673ff33755e8703d45e926e416cb59bae3a86c6a4981" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/13/13/f74ffe6b6f58822e807c70391dc5679a53feb92ce119ccb8a6546c3fb893/psycopg2-2.9.9-cp312-cp312-win32.whl" hash = "sha256:d735786acc7dd25815e89cc4ad529a43af779db2e25aa7c626de864127e5a024" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/58/4b/c4a26e191882b60150bfcb639e416524ae7f8249ab7ee854fb5247f16c40/psycopg2-2.9.9-cp312-cp312-win_amd64.whl" hash = "sha256:a7653d00b732afb6fc597e29c50ad28087dcb4fbfb28e86092277a559ae4e693" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/2b/77/ffeb9ac356b3d99d97ca681bf0d0aa74f6d1d8c2ce0d6c4f2f34e396dbc0/psycopg2-2.9.9-cp37-cp37m-win32.whl" hash = "sha256:5e0d98cade4f0e0304d7d6f25bbfbc5bd186e07b38eac65379309c4ca3193efa" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/be/a7/0a39176d369a8289191f3d327139cfb4923dcedcfd7105774e57996f63cd/psycopg2-2.9.9-cp37-cp37m-win_amd64.whl" hash = "sha256:7e2dacf8b009a1c1e843b5213a87f7c544b2b042476ed7755be813eaf4e8347a" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/1f/78/86b90d30c4e02e88379184ade34c2fd4883a4d3e420cc3c0f6da2b8f3a9a/psycopg2-2.9.9-cp38-cp38-win32.whl" hash = "sha256:ff432630e510709564c01dafdbe996cb552e0b9f3f065eb89bdce5bd31fabf4c" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/8e/e8/c439b378efc9f2d0fd1fd5f66b03cb9ed41423f179997a935f10374f3c0d/psycopg2-2.9.9-cp38-cp38-win_amd64.whl" hash = "sha256:bac58c024c9922c23550af2a581998624d6e02350f4ae9c5f0bc642c633a2d5e" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/6b/a8/5080c0e61a3b393a379ea2fa93402135c73baffcd5f08b9503e508aac116/psycopg2-2.9.9-cp39-cp39-win32.whl" hash = "sha256:c92811b2d4c9b6ea0285942b2e7cac98a59e166d59c588fe5cfe1eda58e72d59" [[distribution.wheel]] url = "https://files.pythonhosted.org/packages/f8/ec/ec73fe66d4317db006a38ebafbde02cb7e1d727ed65f5bbe54efb191d9e6/psycopg2-2.9.9-cp39-cp39-win_amd64.whl" hash = "sha256:de80739447af31525feddeb8effd640782cf5998e1a4e9192ebdf829717e3913" Ref #3351

BurntSushi force-pushed the ag/propagate-wheels branch 3 times, most recently from 7555f36 to f570b88 Compare May 15, 2024 15:27

BurntSushi requested a review from charliermarsh May 15, 2024 15:29

BurntSushi requested a review from konstin May 15, 2024 15:29

BurntSushi marked this pull request as ready for review May 15, 2024 15:29

BurntSushi mentioned this pull request May 15, 2024

distribution-types: allow RegistrySourceDist to have wheels attached to it #3610

Merged