Skip to content

Starting with 0.2.6+ uv lowest-direct does not respect all limits that are specified conditionally #4136

@potiuk

Description

@potiuk

As of recently, we are using uv to generate lowest-direct dependencies in airflow. We do it at scale (we generate separate lowest-direct dependencies for every provider we have - we have 90 of them. And it works very well, or rather worked very well till 0.2.5. Something changed in 0.2.6 that causes uv to stop respecting at least some limits specified to be installed and attempt to install much lower versions of dependencies than the limits are.

An example of this can be seen here https://github.com/apache/airflow/actions/runs/9417755856/job/25943952035?pr=40110#step:7:43011

Due to amount of logs to be processed by github actions UI, it needs a bit of patience to see it, but generally speaking it boils down to this error (when uv 0.2.6+ is used (same behaviour is with latest 0.2.9 and versions between):

uv pip install --python /usr/local/bin/python --resolution lowest-direct --upgrade --editable '.[google]'

error: Failed to download and build `pandas==0.1`
  Caused by: Failed to build: `pandas==0.1`
  Caused by: Build backend failed to determine extra requires with `build_wheel()` with exit status: 1
--- stdout:

--- stderr:
Traceback (most recent call last):
  File "<string>", line 14, in <module>
  File "/tmp/.tmpSYnKSk/.tmpJOVZVf/.venv/lib/python3.8/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
    return self._get_build_requires(config_settings, requirements=['wheel'])
  File "/tmp/.tmpSYnKSk/.tmpJOVZVf/.venv/lib/python3.8/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
    self.run_setup()
  File "/tmp/.tmpSYnKSk/.tmpJOVZVf/.venv/lib/python3.8/site-packages/setuptools/build_meta.py", line 487, in run_setup
    super().run_setup(setup_script=setup_script)
  File "/tmp/.tmpSYnKSk/.tmpJOVZVf/.venv/lib/python3.8/site-packages/setuptools/build_meta.py", line 311, in run_setup
    exec(code, locals())
  File "<string>", line 5, in <module>
ModuleNotFoundError: No module named 'numpy'

This works perfectly fine

The thing is that it seems that uv 0.2.5 - works correctly. For Python 3.8 it will do this and this is pretty much expected.

- pandas==2.0.3
 + pandas==1.5.3

The problem might be connecte with the way how our (dynamic !) dependencies specify this for google extra
https://github.com/apache/airflow/blob/main/generated/provider_dependencies.json#L638 - it's dynamically generated from that .json file by our hatch_build.py hook but it boils down to

pandas>=1.5.3,<2.2;python_version<"3.12"
pandas>=2.1.1,<2.2;python_version>="3.12"

So - as expected - uv should not even attempt to try to build pandas 0.1 because for Python 3.8 it is >=1.5.3.

It's relatively easy to reproduce using Airflow CI dev tooling/image (due to the nature of our project - 790 dependencies using Breeze dev tooling and CI image built automatically is the easiest way to reproduce the same environment our CI has):

  1. checkout airflow
  2. install breeze pipx install -e ./dev/breeze
  3. build CI image with eager upgrade to latest dependencies (`breeze ci-image build --upgrade-to-newer-dependencies)
  4. enter the image breeze shell
  5. install the right version of uv: pip install uv==0.2.6 or pip install uv==0.2.5
  6. run "lowest-direct" dependency resolution: uv pip install --python /usr/local/bin/python --resolution lowest-direct --upgrade --editable '.[google]'
  • In case of uv 0.2.5 or below it will nicely downgrade dependencies to lowest direct as expected.
  • In case of uv 0.2.6 or above it will fail when attempting to compile pandas==0.1

Something must have changed 0.2.5 -> 0.2.6 to make uv stop respecting some kinds of lower bindings (I suspect the conditional python version might have something to do with it). I also made a small experiment and added (in generated/provider_dependencies.json) additional unconditional limit pandas>=1.5.3 effectively leading to:

pandas>=1.5.3
pandas>=1.5.3,<2.2;python_version<"3.12"
pandas>=2.1.1,<2.2;python_version>="3.12"

This one also fails in 0.2.6+ but with:

 error: Failed to download and build `alembic==0.1.0`
    Caused by: Failed to build: `alembic==0.1.0`
    Caused by: Build backend failed to determine extra requires with `build_wheel()` with exit status: 1

That suggests tha indeed the conditional python_version is the one causing uv to do way more backtracking that it should.

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions