Deprecate some non-PEP 625 source distributions#17467
Conversation
|
Noting: I put this warning at the sdist filename parsing layer to start, but that's going to be way too noisy since it'll ding on every legacy sdist on PyPI (and there are a lot of them in the tail of popular projects). I think the better place to put this is in the installation planner, and limit it to just to-be-fetched remote dists. Edit: more precisely, |
|
One option is to warn specifically when we unpack an xz/lzma source dist, where we could also do this for xz/lzma wheels, another is checking after resolution or after loading a lockfile for the files in the resolution type. |
|
I have it in
Oh, I didn't realize we actually allowed wheels to also be xz/lzma 😅 -- I was looking at |
|
zips can use a number of compression options, such as stored (no compression), DEFLATE (the default) but also LZMA. There's a list unter "Compression method" in https://users.cs.jmu.edu/buchhofp/forensics/formats/pkzip.html. There's nothing that says what kind of zips the wheel are, but generally, they are either DEFLATE (basically all published zips) or stored (e.g. for editables). |
| // for directory and Git installs. | ||
| if !matches!(dist.extension(), Some(SourceDistExtension::TarGz) | None) { | ||
| warn_user!( | ||
| "Legacy (non-PEP 625) source distributions are deprecated: {dist}. A future version of uv will reject source distributions that do not use '.tar.gz'", |
There was a problem hiding this comment.
This needs to include the dist name, I think?
I'd say something like "{dist} is not a standards-compliant source distribution: expected extension {} but found {}. A future version of uv will reject source distributions that do not match the specification defined in PEP 625."
Ah yeah, sorry -- I misunderstood you to be saying that So, there are two distinct things we want to warn on/deprecate:
This PR is focusing on (1) right now, but I could also do (2) (or in a follow-up). |
|
(1) is definitely a good unit to merge on it's own. I'm mainly bringing up (2) cause we also need it to remove the xz/lzma dependencies. |
| ----- stderr ----- | ||
| Resolved 2 packages in [TIME] | ||
| warning: Legacy (non-PEP 625) source distributions are deprecated: wsgiref==0.1.2. A future version of uv will reject source distributions that do not use '.tar.gz' | ||
| warning: wsgiref==0.1.2 is not a standards-compliant source distribution: expected '.tar.gz' but found 'zip'. A future version of uv will reject source distributions that do not match the specification defined in PEP 625 |
There was a problem hiding this comment.
wsgiref==0.1.2 doesn't seem quite right here. I think we need the filename too?
There was a problem hiding this comment.
I was thinking about that, and I think the filename might not be super useful information to a user who sees this: the average sdist consumer is probably getting one via an index resolution, so showing them wsgiref-0.1.2.zip isn't going to mean much to them (whereas wsgiref==0.1.2 at least tells them that it's a post-resolution candidate, so they could change it by updating their constraints).
But I'm speculating, if you think the filename is helpful here I'll change!
There was a problem hiding this comment.
I think we need the filename since the rest of the error is focused on it, I would maybe say {filename} ({package}=={version}) is not ...?
Unless you want to change the error in the case where the user has not requested a file directly, which may be better. In that case, I think we'd change the whole error message to be something like
{package}=={version} uses a legacy source distribution format (
.zip) that is not compliant with the specification in PEP 625. A future version of uv will .... Consider upgrading to a newer version of {package}
I think different messages for these scenarios might make a lot of sense?
There was a problem hiding this comment.
Also, I didn't realize there were packages on the index that were using the legacy format. We might want to drop support for that later / separately. We might also want to ignore them during resolution but allow opt-in during a transition period? Lots to consider there.
There was a problem hiding this comment.
I think we need the filename since the rest of the error is focused on it, I would maybe say
{filename} ({package}=={version}) is not ...?
Ah yeah, that's a good point. I'll add the filename.
I'm inclined to keep it as a single warning for both direct and indirect resolved candidates, since I think in both cases it's not super directly actionable. Although now that I say that I guess it is actionable when the user does e.g. a GitHub zipball, so maybe that deserves a discrete warning. I'll look at that tomorrow.
Also, I didn't realize there were packages on the index that were using the legacy format. We might want to drop support for that later / separately. We might also want to ignore them during resolution but allow opt-in during a transition period? Lots to consider there.
Yeah, there's lots of potential nuance. The "good" news is that the legacy sdists on the index should generally be pretty old (as in 2016 or earlier), meaning that they shouldn't be selected as candidates super often. wsgiref==0.1.2 is from 2006, for example -- I'm actually shocked that installs at all 😅
There was a problem hiding this comment.
Upgrading to a newer version seems more actionable than regenerating a local tarball too 🤷♀️
There was a problem hiding this comment.
OK, cleaned this up and split the warnings as suggested -- we now present a different (and clearer/more actionable) warning for the sdist-from-registry case.
## Summary This adds warnings to both our steam and sync ZIP handling on ZIP entries that aren't "well-known." For now, "well-known" means stored (i.e. no compression), DEFLATE, or zstd. In practice we have duplicated codepaths for this check: one for the "sync" (pre-downloaded) path, and one for the streaming path. See #16911 and #17467 for context. ## Test Plan Will update snapshots if/when they change. I'll also add a ZIP test for this. (Upd: added some "futzed" wheels for the ZIP tests.) --------- Signed-off-by: William Woodruff <william@astral.sh>
Signed-off-by: William Woodruff <william@astral.sh>
And update some snapshots. Signed-off-by: William Woodruff <william@astral.sh>
Signed-off-by: William Woodruff <william@astral.sh>
Signed-off-by: William Woodruff <william@astral.sh>
Signed-off-by: William Woodruff <william@astral.sh>
5d90ce8 to
2b4ce2a
Compare
Signed-off-by: William Woodruff <william@astral.sh>
Signed-off-by: William Woodruff <william@astral.sh>
Signed-off-by: William Woodruff <william@astral.sh>
Signed-off-by: William Woodruff <william@astral.sh>
| // materialized into a wheel. Observe that we also allow no extension, since we expect that | ||
| // for directory and Git installs. | ||
| if let Some(extension) = dist.extension() | ||
| && !matches!(extension, SourceDistExtension::TarGz) |
There was a problem hiding this comment.
We may have to allow zips for now, they used to be allowed until kinda recently (post-PEP 625 some docs advertised them, and PEP 625 was 2020), at least unless they use rare compression. But we're already warning on wheel extract, so non-deflate compression warnings may already be covered? That way, we're still on track to remove xz/lzma libraries.
There was a problem hiding this comment.
Makes sense, I've widened the exception to include .zip 🙂
Signed-off-by: William Woodruff <william@astral.sh>
Signed-off-by: William Woodruff <william@astral.sh>
Summary
This adds a deprecation warning for some (not all) source distributions that don't follow the PEP 625 filename scheme, which requires a
.tar.gzsuffix. I've also updated the docs with a warning about behavior + deprecation notice.Specifically, we warn unless the sdist ends with
.tar.gzor.zip. The latter is not in PEP 625 but it's common enough in the real world that we shouldn't blast users with warnings about it (yet).Towards #16911.
Test Plan
The only functional change here is a user-level warning. I'll bump the snapshots that change.