Skip to content

Add [testenv] constraints option#3556

Merged
gaborbernat merged 4 commits intotox-dev:mainfrom
stephenfin:issue-3550
Jul 10, 2025
Merged

Add [testenv] constraints option#3556
gaborbernat merged 4 commits intotox-dev:mainfrom
stephenfin:issue-3550

Conversation

@stephenfin
Copy link
Contributor

@stephenfin stephenfin commented Jul 4, 2025 •
edited
Loading

Pushing as draft to ensure this is a sane direction. I still need to add tests. I also think I should subclass PythonConstraints from RequirementsFile rather than PythonDeps since we don't need a lot of the logic in the latter.

I've done both of these now.

Add a new option allowing us to specify constraints file(s) that will be applied when installing all dependencies except the package under test. For example, given the following tox.ini:

[testenv]
deps =
  requests
extras =
  test
constraints =
  upper-constraints.txt
command =
  pytest {posargs}

The all requirements in deps, all requirements in the test extra, and all dependencies of the package itself will be installed with -c upper-constraints.txt. The package itself will not.

Closes #3550

  • ran the linter to address style issues (tox -e fix)
  • wrote descriptive pull request text
  • ensured there are test(s) validating the fix
  • added news fragment in docs/changelog folder
  • updated/extended the documentation

@stephenfin
trivial: Dedent function
02d34ea 
Signed-off-by: Stephen Finucane <stephen@that.guru>
@stephenfin stephenfin force-pushed the issue-3550 branch 2 times, most recently from 3ab1c98 to 1e2cdf0 Compare July 4, 2025 17:43
stephenfin added 3 commits July 6, 2025 12:57
@stephenfin
trivial: Address TODO
6080adb 
Signed-off-by: Stephen Finucane <stephen@that.guru>
@stephenfin
Remove unnecessary dependency
c2d72c4 
The minimum version is Python 3.9 now.

Signed-off-by: Stephen Finucane <stephen@that.guru>
@stephenfin
Add '[testenv] constraints' setting
fa58e0a 
This allows users to override constraints files for all installations.
Without this, it is not possible to use constraints files that include
the package under test without introducing a separate pre-tox processing
step to scrub said package from the constraints file.

Signed-off-by: Stephen Finucane <stephen@that.guru>
@stephenfin stephenfin marked this pull request as ready for review July 9, 2025 16:38
@stephenfin stephenfin requested a review from gaborbernat as a code owner July 9, 2025 16:38
@gaborbernat
Copy link
Member

gaborbernat commented Jul 9, 2025

Can you explain how this differs from just setting the PIP_Constraint environment variable via set env?

@stephenfin
Copy link
Contributor Author

stephenfin commented Jul 9, 2025

Sure. Consider https://github.com/openstack/openstacksdk. Like all OpenStack projects, it uses constraints to ensure co-installability. We currently pass the -c option via [testenv] deps. I can make the following change to tox.ini:

diff --git tox.ini tox.ini
index ca14f55af..3c6e6d6a1 100644
--- tox.ini
+++ tox.ini
@@ -15,8 +15,8 @@ setenv =
     OS_LOG_CAPTURE={env:OS_LOG_CAPTURE:true}
     OS_STDOUT_CAPTURE={env:OS_STDOUT_CAPTURE:true}
     OS_STDERR_CAPTURE={env:OS_STDERR_CAPTURE:true}
+    PIP_CONSTRAINT={env:TOX_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/master}
 deps =
-    -c{env:TOX_CONSTRAINTS_FILE:https://releases.openstack.org/constraints/upper/master}
     -r{toxinidir}/test-requirements.txt
     -r{toxinidir}/requirements.txt
 commands =

I then run tox -e py310 --notest. It fails, and it will always fail:

❯ tox -e py310 --notest
.pkg: _optional_hooks> python /usr/lib/python3.13/site-packages/pyproject_api/_backend.py True pbr.build
.pkg: get_requires_for_build_sdist> python /usr/lib/python3.13/site-packages/pyproject_api/_backend.py True pbr.build
.pkg: get_requires_for_build_wheel> python /usr/lib/python3.13/site-packages/pyproject_api/_backend.py True pbr.build
.pkg: prepare_metadata_for_build_wheel> python /usr/lib/python3.13/site-packages/pyproject_api/_backend.py True pbr.build
.pkg: build_sdist> python /usr/lib/python3.13/site-packages/pyproject_api/_backend.py True pbr.build
py310: install_package> python -I -m pip install --force-reinstall --no-deps /home/stephenfin/code/openstack/openstacksdk/.tox/.tmp/package/68/openstacksdk-4.6.1.dev20.tar.gz
Processing ./.tox/.tmp/package/68/openstacksdk-4.6.1.dev20.tar.gz
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Preparing metadata (pyproject.toml) ... done

The conflict is caused by:
    The user requested openstacksdk 4.6.1.dev20 (from /home/stephenfin/code/openstack/openstacksdk/.tox/.tmp/package/68/openstacksdk-4.6.1.dev20.tar.gz)
    The user requested (constraint) openstacksdk===4.6.0

To fix this you could try to:
1. loosen the range of package versions you've specified
2. remove package versions to allow pip to attempt to solve the dependency conflict

ERROR: Cannot install openstacksdk 4.6.1.dev20 (from /home/stephenfin/code/openstack/openstacksdk/.tox/.tmp/package/68/openstacksdk-4.6.1.dev20.tar.gz) because these package versions have conflicting dependencies.
ERROR: ResolutionImpossible: for help visit https://pip.pypa.io/en/latest/topics/dependency-resolution/#dealing-with-dependency-conflicts

py310: exit 1 (8.10 seconds) /home/stephenfin/code/openstack/openstacksdk> python -I -m pip install --force-reinstall --no-deps /home/stephenfin/code/openstack/openstacksdk/.tox/.tmp/package/68/openstacksdk-4.6.1.dev20.tar.gz pid=782739
.pkg: _exit> python /usr/lib/python3.13/site-packages/pyproject_api/_backend.py True pbr.build
  py310: FAIL code 1 (11.17 seconds)
  evaluation failed :( (11.21 seconds)

This will always happen if you are developing a project that is itself included in constraints file, which is neither uncommon nor unreasonable (IMO) since those projects need to evolve too.

I considered simply ignoring the PIP_CONSTRAINT env var when installing the package under test, but then I also noted that when I make the above change, the testenv is rebuilt the first time I do this because constraints are no longer tracked.

❯ tox -e py310 --notest                                                                                                                                                                                                                                                                                                                                                                     
py310: recreate env because changed constraint(s) removed  APScheduler===3.11.0, {...snipped...}, zuul-sphinx===0.7.0

Suggesting I lose the ability to auto-regenerate when constraints change, which tox (4) is otherwise very good at. Thus, here is my proposed solution.

An alternative would be to ignore PIP_CONSTRAINT when installing the package under test and start pulling the file/URL from PIP_CONSTRAINT and tracking that .tox-info.json. However, that seem less intuitive to me and has end-users using a pip-derived config knob rather than tox-derived one.

gaborbernat
gaborbernat approved these changes Jul 10, 2025
View reviewed changes
Copy link
Member

@gaborbernat gaborbernat left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@gaborbernat gaborbernat merged commit e2ff114 into tox-dev:main Jul 10, 2025
28 checks passed
@stephenfin stephenfin deleted the issue-3550 branch July 11, 2025 10:06
webknjaz
webknjaz reviewed Jul 20, 2025
View reviewed changes
@stephenfin stephenfin mentioned this pull request Feb 10, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@gaborbernat gaborbernat gaborbernat approved these changes

+2 more reviewers

@webknjaz webknjaz webknjaz left review comments

@hroncok hroncok hroncok left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

bot:chronographer:provided

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

First-class constraints.txt file support

4 participants

@stephenfin @gaborbernat @webknjaz @hroncok