-
Notifications
You must be signed in to change notification settings - Fork 2.9k
cross platform lock file generation #3350
Description
Work items
- universal-lock: add a "universal" toggle to our resolver #3353
- universal-lock: add marker evaluation for just extras #3352
- universal-lock: refactor resolver state into explicit type #3354
- universal-lock: implement routine for computing intersection of two
MarkerTrees #3355 - universal-lock: propagate full set of artifacts for each distribution #3351
- universal-lock: implement resolver forking #3358
- universal-lock: add marker expressions to PubGrubPackage #3359
- universal-lock: implement merging of forked resolver states #3360
- universal-lock: check that markers are disjoint before forking #3926
This is a tracking ticket for exposing a way to generate a cross platform lock file using the uv CLI. This ticket is primarily about making this available as a library feature, with exposure on the CLI being a step for making it easy to interact with. For example, this should be done via uv pip compile --unstable-uv-lock-file.
A bit ago, I did a prototype of this by roughly doing the following:
- Ignore markers on dependencies.
- Make the resolver "fork" internally whenever multiple packages enter the dependency tree as a result of ignoring markers.
- Unify all forks of the resolver at the end into one set of locked versions for packages. The result is a "universal" or "cross platform." Its principally differs from a
requirements.txtin that it can have multiple versions of the same package in it, and it includes all of the artifacts (wheels and source dist) for each distribution. Installing from a lock file thus requires some filtering.
While this prototype demonstrated the idea, it took a number of shortcuts that made it work only in a limited number of cases.
NOTE: This is distinct from #3347 in that this goal of this ticket is make lock file generation work. In #3347, the goal is to integrate lock file generate with the rest of the uv workspace commands.
A good litmus test for this issue being completed is that we can run the following:
$ cat requirements.in
# Compile with Python 3.10 as minimal version.
anyio<4 ; python_version < "3.11"
anyio>=4.3.0 ; python_version >= "3.11"
idna<3 ; python_version < "3.10"
idna>=3.6.0 ; python_version >= "3.12"
$ uv pip compile -p3.10 --unstable-uv-lock-file requirements.in
And get a uv.lock file as output. This example is interesting because it requires the resolver to "fork." Namely, with 3.10 as the minimal version of Python, both anyio<4 and anyio>=4.3.0 are constraints that need to be resolved simultaneously. Since they must necessarily lead to different versions of anyio, the resulting lock file will contain multiple versions of anyio.