My issue is similar to the one described in #6509, but I wanted to dig deeper, and I really fail to understand how the resolver works.

Description

I am working on a CLI tool, which depends on magika, which is a file detection software with deep learning. I want my tool to be available for every Python starting with 3.8. Magika does not support Python 3.13, and I want to handle it gracefully, so I only install it when the version of Python is under 3.13.

# pyproject.toml
[project]
name = "mytool"
version = "0.1.0"
dependencies = [
    "magika; python_version < '3.13'",
]
requires-python = ">=3.8"

[build-system]
requires = ["flit_core >=3.2,<4"]
build-backend = "flit_core.buildapi"

I then want to lock and sync the dependencies, but uv lock fails because it fails to find distutils.

$ uv lock
Using Python 3.12.5 interpreter at: /Library/Frameworks/Python.framework/Versions/3.12/bin/python3
Resolved 18 packages in 14ms

$ uv sync
Using Python 3.12.5 interpreter at: /Library/Frameworks/Python.framework/Versions/3.12/bin/python3
Creating virtualenv at: .venv
Resolved 18 packages in 1ms
error: Failed to prepare distributions
  Caused by: Failed to fetch wheel: numpy==1.24.4
  Caused by: Build backend failed to determine extra requires with `build_wheel()` with exit status: 1
--- stdout:

--- stderr:
Traceback (most recent call last):
  File "<string>", line 8, in <module>
  File "/Users/nikita/Library/Caches/uv/builds-v0/.tmp0idacA/lib/python3.12/site-packages/setuptools/__init__.py", line 10, in <module>
    import distutils.core
ModuleNotFoundError: No module named 'distutils'
---

Problem

The comments in #6509 correctly suggest that the numpy version is too old for the environment, as distutils is gone from 3.12. Alas, I do not understand why it resolves to this version in the first place.

The pyproject.toml from above defined the range for my CLI to be >=3.8, and it depends on magika if the version is <3.13. So, just from this I expect at least two resolution markers: "below 3.13" and "3.13 and above".

magika itself does have some variable dependencies. magika==0.5.0 has the following in its wheel's metadata:

...
Requires-Python: >=3.8,<3.12
...
Requires-Dist: numpy (>=1.24.4,<2.0.0)
...

So, For every Python between 3.8 and 3.11, use any numpy older than 1.24.24.

magika==0.5.1 added support for Python 3.12, with the following metadata:

...
Requires-Python: >=3.8,<3.13
...
Requires-Dist: numpy (>=1.24,<2.0) ; python_version >= "3.8" and python_version < "3.9"
Requires-Dist: numpy (>=1.26,<2.0) ; python_version >= "3.9" and python_version < "3.13"
...

So, now it says: use any numpy older than 1.24.24, unless you are above 3.9, in which case it's stricter.

Expected behaviour

Based on this, I expect three different lockfile resolutions for my package:

Python >=3.8,<3.9 aka 3.8.*

mytool
  - magika==0.5.1
    - numpy==1.24.*

Python >=3.9,<3.13

mytool
  - magika==0.5.1
    - numpy==1.26.*

Python >=3.13

mytool
  - (no magika because version to high)

This looks easy (at least for me). Just take the latest magika==0.5.1 (it supports 3.8), and take numpy 1.24 for Python 3.8 and 1.26 for anything else.

Actual behaviour

This is not what happens, though. uv.lock does define resolution markers that I expect:

# uv.lock
version = 1
requires-python = ">=3.8"
resolution-markers = [
    "python_full_version < '3.9'",
    "python_full_version >= '3.9' and python_full_version < '3.13'",
    "python_full_version >= '3.13'",
]
...

But then, it weirdly decides to use an older magika version for the newer Pythons:

[[package]]
name = "mytool"
version = "0.1.0"
source = { editable = "." }
dependencies = [
    { name = "magika", version = "0.5.0", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version >= '3.9' and python_full_version < '3.13'" },
    { name = "magika", version = "0.5.1", source = { registry = "https://pypi.org/simple" }, marker = "python_full_version < '3.9'" },
]

Which then proceeds to only pull numpy==1.24.4, as it theoretically satisfies both ranges:

[[package]]
name = "numpy"
version = "1.24.4"
...

I kinda understand why it might do this; after all, we want as little packages as possible to simplify the lock file, but isn't it a better idea to use the newer package whenever possible? I assume one can say that numpy is at fault for not releasing a patch for 1.24 that would specify that it does not support Python 3.12, but isn't there a different way to solve this?

Things I've tried

Set Python <3.13 for mytool

If I tell that my whole too does not support Python 3.13, uv just resolves to the latest magika==0.5.1 and installs without problem. It also correctly locks two numpy versions: 1.24 for Python 3.8 and 1.26 for everything else

Set magika>=0.5.1

If I explicitly request the latest version of magika, uv can't lock, giving me this error:

  × No solution found when resolving dependencies for split (python_full_version >= '3.9' and python_full_version < '3.13'):
  ╰─▶ Because only the following versions of numpy{python_full_version >= '3.9' and python_full_version < '3.13'} are available:
          numpy{python_full_version >= '3.9' and python_full_version < '3.13'}<=1.26.0
          numpy{python_full_version >= '3.9' and python_full_version < '3.13'}==1.26.1
          numpy{python_full_version >= '3.9' and python_full_version < '3.13'}==1.26.2
          numpy{python_full_version >= '3.9' and python_full_version < '3.13'}==1.26.3
          numpy{python_full_version >= '3.9' and python_full_version < '3.13'}==1.26.4
          numpy{python_full_version >= '3.9' and python_full_version < '3.13'}>2.0
      and the requested Python version (>=3.8) does not satisfy Python>=3.9, we can conclude that all of:
          numpy{python_full_version >= '3.9' and python_full_version < '3.13'}>=1.26.0,<1.26.2
          numpy{python_full_version >= '3.9' and python_full_version < '3.13'}>1.26.2,<1.26.3
          numpy{python_full_version >= '3.9' and python_full_version < '3.13'}>1.26.3,<1.26.4
          numpy{python_full_version >= '3.9' and python_full_version < '3.13'}>1.26.4,<2.0
       are incompatible.
      And because the requested Python version (>=3.8) does not satisfy Python>=3.9 and magika{python_full_version < '3.13'}==0.5.1 depends on numpy{python_full_version >= '3.9' and python_full_version <
      '3.13'}>=1.26,<2.0, we can conclude that magika{python_full_version < '3.13'}==0.5.1 cannot be used.
      And because only magika{python_full_version < '3.13'}<=0.5.1 is available and your project depends on magika{python_full_version < '3.13'}>=0.5.1, we can conclude that your project's requirements are
      unsatisfiable.

I'm surprised to read that "magika{python_full_version < '3.13'}==0.5.1 depends on numpy{python_full_version >= '3.9' and python_full_version < '3.13'}>=1.26,<2.0". Sure, it does, but it also does depend on numpy{python_full_version >= '3.8' and python_full_version < '3.9'}>=1.24,<2.0. Can't it use it for "the requested Python version (>=3.8)"?

Environment

uv platform: macOS 13.6.9 Ventura, Python 3.12.5 from python.org
uv version: uv 0.4.1 (Homebrew 2024-08-30)

I had troubles searching for similar issues, as I didn't know what keywords to use; sorry, if this turns out to be a duplicate!