Fail on mismatched python spec attributes#2824
Conversation
The following is intended to be a correct tox configuration:
[tox]
min_version = 4.1
env_list = py{38,39,310,311},docs
[testenv]
base_python = python3.8
[testenv:docs]
commands =
sphinx-build ...
The goal of this is to use 'python3.8' (i.e. the value of '[testenv]
base_python') for all environments except those with specific 'pyXX'
factors in them. This helps eliminate issues where environments that do
not specify a 'pyXX' factor run with different Python versions in
different environments.
An earlier bug, tox-dev#477 [1], prevented us from doing this. Due to tox-dev#477, the
interpreter version indicated by '[testenv] base_python' (or rather
'[testenv] basepython' - the underscore-separated variant was only
introduced in tox 4) would override the value indicated by the 'pyXX'
factor. This was resolved with a PR, tox-dev#841 [2], which started warning
users if they were unintentionally running against the wrong interpreter
and introduced the 'ignore_basepython_conflict' value to opt into the
correct behavior.
Unfortunately, this has been partially broken in the move to tox 4.
While the above configuration still works, the following no longer does:
[tox]
min_version = 4.1
env_list = py{38,39,310,311},docs
[testenv]
base_python = python3
[testenv:docs]
commands =
sphinx-build ...
This configuration was common back during the move from Python 2 to
Python 3. It ensured that 'python3' was used for all testenvs that did
not request an explicit Python version via a factor or their own
'base_python' version. While it's no longer necessary, it is still quite
common. Currently, running with this configuration will result in
'python3' being used for every environment, rather than e.g. 'python3.8'
for a testenv with the 'py38' factor. This is clearly incorrect. This
issue stems from an errant bit of logic. When comparing interpreter
versions as part of the '_validate_base_python', we ignore various
attributes if they are unset on either '[testenv] base_python' or the
'pyXX' factor. This allows e.g. 'python3' to match 'python3.8', since
the minor version is unset for the '[testenv] base_python' value,
'python3'. The fix is simple: while we can ignore unset attributes for
factor-derived versions (to allow a 'py3' factor to work with e.g.
'base_python = python3.8'), we should insist on them for
'base_python'-derived versions.
[1] tox-dev#477
[2] tox-dev#841
Signed-off-by: Stephen Finucane <stephen@that.guru>
Closes: tox-dev#2754
9a929f2 to
79c86c7
Compare
|
Docs failures don't appear to be due to this... Edit: It looks like Sphinx 6.1.0 is the culprit. Works fine on 6.0.1. Edit: And it's this commit in Sphinx that's causing an incompatibility with the |
docs/changelog/2754.bugfix.rst
Outdated
| @@ -0,0 +1,2 @@ | |||
| Setting ``[testenv] basepython = python3`` will no longer override the Python | |||
There was a problem hiding this comment.
Wrap at 120 and add - by :user:`x`.
There was a problem hiding this comment.
The following is intended to be a correct tox configuration:
[tox] min_version = 4.1 env_list = py{38,39,310,311},docs [testenv] base_python = python3.8 [testenv:docs] commands = sphinx-build ...The goal of this is to use
python3.8(i.e. the value of[testenv] base_python) for all environments except those with specificpyXXfactors in them. This helps eliminate issues where testenvs that do not specify apyXXfactor run with different Python versions in different environments.
Strong disagree. This is not a good way to express this. This config should hard fail for all envs expect py38.
There was a problem hiding this comment.
This helps eliminate issues where testenvs that do not specify a
pyXXfactor run with different Python versions in different environments.
If those other envs care about a python version, they should specify so in their section; e.g. here, docs should set it to 3.8. Not setting it means I'm happy with any Python.
There was a problem hiding this comment.
The following is intended to be a correct tox configuration:
[tox] min_version = 4.1 env_list = py{38,39,310,311},docs [testenv] base_python = python3.8 [testenv:docs] commands = sphinx-build ...The goal of this is to use
python3.8(i.e. the value of[testenv] base_python) for all environments except those with specificpyXXfactors in them. This helps eliminate issues where testenvs that do not specify apyXXfactor run with different Python versions in different environments.Strong disagree. This is not a good way to express this. This config should hard fail for all envs expect py38.
Regarding the hard fail, currently tox does not hard fail with this configuration and it does the wrong thing (using python3 for all environments regardless of the version inferred from the factor). With this change, it does hard fail unless the user has explicitly set ignore_base_python_conflict = true. So I think what we're discussing here isn't actually this change itself, since this fixes an obvious bug.
Regarding whether this is a good way to express this, requiring users duplicate base_python in every section makes base_python a special-case, no? For everything else, the idea is that [testenv] allows you to set "default" values that you can later override (or extend) in [testenv:foo] sections. tox itself does this for various things like command:
Lines 27 to 35 in af4b558
Lines 47 to 49 in af4b558
My question is why can we do this for things like commands, deps, or passenv etc. but we should not be able to do it for base_python?
There was a problem hiding this comment.
But that makes
base_pythona special-case? For everything else, the idea is that[testenv]allows you to set "default" values that you can later override (or extend) in[testenv:foo]sections. tox itself does this for various things likecommand:
It's not a special case. Follows the same rule.
[testenv]
base_python = python3.8
[testenv:py39]
base_python = python3.9 Would work. Where you're mistaken is that py38 only sets base_python to py38 if at no point is base_python specified. If you set it at testenv or [testenv:py38] level that no longer activates.
| ("py3", ["py2"], ["py2"]), | ||
| ("py38", ["py39"], ["py39"]), | ||
| ("py38", ["py38", "py39"], ["py39"]), | ||
| ("py38", ["python3"], ["python3"]), |
There was a problem hiding this comment.
This shouldn't conflict, but py27 and python3 should 🤔
There was a problem hiding this comment.
Both should conflict. python3 on my system == python3.11.1. That is decidedly not python3.8, which is what I intended. I should be exploding on that combination - which I do (unless the user explicitly said not to, via ignore_base_python_conflict)
There was a problem hiding this comment.
You're miss understanding. base_python is not an executable match. It is a specification. And the specification states Python 3, without resolving it any further.
There was a problem hiding this comment.
That may be the case, but providing a specification of python3 (e.g. base_python = python3) causes the python3 executable to be used. For example, using a tox.ini like this:
[tox]
minversion = 3.1
envlist = py{37,38,39,310,311}
[testenv]
basepython = python3If I run tox -e py38, the py38 venv is using python3.11 :
❯ source .tox/py38/bin/activate
❯ python --version
Python 3.11.1
No warnings. Nothing. That's got to be incorrect behaviour, surely?
There was a problem hiding this comment.
Indeed, so we have two options:
- for py38
python3spec should conflict and hard fail, - hard fail once we resolve
python3and see its something else.
I'm inclined to go for option 1.
There was a problem hiding this comment.
Agree. Option 1 is the way to go. Option 2 will result in different behaviour depending on the environment making tox less deterministic. That's A Bad Thing ™️
Without this change, the py38 venv uses whatever python3 is on my host (python3.11). With this change, things hard fail as expected.
❯ tox -e py38
py38: failed with env name py38 conflicting with base python python3
py38: FAIL code 1 (0.00 seconds)
evaluation failed :( (0.08 seconds)
So it seems I have implemented option 1, yes?
There was a problem hiding this comment.
What happens for env py3, py, magic? Does that still work without this failure?
There was a problem hiding this comment.
It passes, as expected.
❯ tox -e py3
py3: install_deps> python -I -m pip install -r /tmp/tox-issue-2717/test-requirements.txt
.pkg: _optional_hooks> python /tmp/tox-venv/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: get_requires_for_build_sdist> python /tmp/tox-venv/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: prepare_metadata_for_build_wheel> python /tmp/tox-venv/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
.pkg: build_sdist> python /tmp/tox-venv/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
py3: install_package_deps> python -I -m pip install requests
py3: install_package> python -I -m pip install --force-reinstall --no-deps /tmp/tox-issue-2717/.tox/.tmp/package/11/foo-0.0.0.tar.gz
.pkg: _exit> python /tmp/tox-venv/lib/python3.11/site-packages/pyproject_api/_backend.py True setuptools.build_meta __legacy__
py3: OK (4.39 seconds)
congratulations :) (4.42 seconds)
Signed-off-by: Stephen Finucane <stephen@that.guru>
|
🥳 Thanks. Happy to have gotten this resolved 😄 |
|
Sorry for taking a while for me to understand 😆 |
tox now hard fails if there is mismatch in spec attributes[1]. This works around the issue temporarily. We probably have to drop py38 jobs and specify basepython for py39 target in the future. [1] tox-dev/tox#2824 Closes-Bug: #2002035 Change-Id: I4256a51afd011048eaaaf5d3c225ab36f75708ed
* Update tripleo-heat-templates from branch 'master'
to 0c0def6f81b6a71f2a07ae4bec9834d8048edb25
- Fix issues with tox 4.2.4
tox now hard fails if there is mismatch in spec
attributes[1]. This works around the issue
temporarily. We probably have to drop py38 jobs
and specify basepython for py39 target in the
future.
[1] tox-dev/tox#2824
Closes-Bug: #2002035
Change-Id: I4256a51afd011048eaaaf5d3c225ab36f75708ed
* Update tripleo-ansible from branch 'master'
to b88008dd64049de8a3ae7e38ee76d563134af2ef
- Fix issues with tox 4.2.4
tox now hard fails if there is mismatch in spec
attributes[1]. This works around the issue
temporarily. We probably have to drop py38 jobs
and specify basepython for py39 target in the
future.
Removes ansible-galaxy --timeout for py* jobs to
workaround. We may hit the timeout with py* jobs.
[1] tox-dev/tox#2824
Closes-Bug: #2001602
Closes-Bug: #2002035
Change-Id: Iad2455193885205236e03c1219352102bb931de8
tox now hard fails if there is mismatch in spec attributes[1]. This works around the issue temporarily. We probably have to drop py38 jobs and specify basepython for py39 target in the future. Removes ansible-galaxy --timeout for py* jobs to workaround. We may hit the timeout with py* jobs. [1] tox-dev/tox#2824 Closes-Bug: #2001602 Closes-Bug: #2002035 Change-Id: Iad2455193885205236e03c1219352102bb931de8
The patch bumps min version of tox to 3.18.0 in order to replace tox's whitelist_externals by allowlist_externals option: https://github.com/tox-dev/tox/blob/master/docs/changelog.rst#v3180-2020-07-23 Also removes skipdist=True as in Ib0fca00c92ef56fce57eb6355cb6be36e478a744. Also inlcludes: Fix issues with tox 4.2.4 tox now hard fails if there is mismatch in spec attributes[1]. This works around the issue temporarily. We probably have to drop py38 jobs and specify basepython for py39 target in the future. [1] tox-dev/tox#2824 Conflicts: tox.ini Backport note: This is now required to adapt to the new tox 4.0 release, which no longer supports the old deprecated options. Closes-Bug: #2002035 Change-Id: I5792c86745b875d074f670d92fe78a84a907bd74 (cherry picked from commit 3cb006d and 7fd4519) (cherry picked from commit 0c0def6)
tox now hard fails if there is mismatch in spec attributes[1]. This works around the issue temporarily. We probably have to drop py38 jobs and specify basepython for py39 target in the future. [1] tox-dev/tox#2824 Closes-Bug: #2002035 Change-Id: I4256a51afd011048eaaaf5d3c225ab36f75708ed (cherry picked from commit 0c0def6)
* use min version 4.2.5, for fixes [1][2][3] * passenv fixed as space-separated list is not allowed anymore * dock target uses requirements from requirements.txt as well as doc/requirements.txt * skipsdist is not supported * whitelist_externals has been removed in favour of allowlist_externals * reno was added to doc/requirements.txt to fix the releasenotes target * update setup.cfg to install aodh from tarball in the requirements The tarball wasn't being installed when specified in tox.ini, and the [extras] section in setup.cfg needed updating to support installing from a URL [1] tox-dev/tox#2754 [2] tox-dev/tox#2824 [3] tox-dev/tox#2828 Change-Id: I4122d0d05f297f864318e80392e6c77fb2e9fdcf
* Update python-aodhclient from branch 'master'
to 02176deb25b3a8f1bc02eb3d70f0a50ce22f02f7
- Make tox.ini tox 4.0 compatible
* use min version 4.2.5, for fixes [1][2][3]
* passenv fixed as space-separated list is not allowed anymore
* dock target uses requirements from requirements.txt as well as
doc/requirements.txt
* skipsdist is not supported
* whitelist_externals has been removed in favour of allowlist_externals
* reno was added to doc/requirements.txt to fix the releasenotes target
* update setup.cfg to install aodh from tarball in the requirements
The tarball wasn't being installed when specified in tox.ini, and the
[extras] section in setup.cfg needed updating to support installing
from a URL
[1] tox-dev/tox#2754
[2] tox-dev/tox#2824
[3] tox-dev/tox#2828
Change-Id: I4122d0d05f297f864318e80392e6c77fb2e9fdcf
1. Ignore base python conflict error New version of tox (4.2.4?) fails if there is mismatch in spec attributes [1]. We can change the base python to match with the test envs, but it may make it inconvenient to setup local development env, so just ignore the error to workaround the issue. 2. Allow running bash in bandit test [1] tox-dev/tox#2824
* Update aodh from branch 'master'
to 84d59a198257ea6f3b839ca5d96cbaa74f2f3b76
- Make tox.ini tox 4.0 compatible
* use min version 4.2.5, for fixes [1][2][3]
* passenv fixed as space-separated list is not allowed anymore
* doc target uses requirements.txt as well as docs/requirements.txt
* skipsdist is not supported
* Add usedevelop = False so that aodh-api gets installed
Update setup.cfg: [files] -> [options]
[1] tox-dev/tox#2754
[2] tox-dev/tox#2824
[3] tox-dev/tox#2828
Change-Id: I2422dc17e6c73ef346de80e57cdf61ef5d271d69
* use min version 4.2.5, for fixes [1][2][3] * passenv fixed as space-separated list is not allowed anymore * doc target uses requirements.txt as well as docs/requirements.txt * skipsdist is not supported * Add usedevelop = False so that aodh-api gets installed Update setup.cfg: [files] -> [options] [1] tox-dev/tox#2754 [2] tox-dev/tox#2824 [3] tox-dev/tox#2828 Change-Id: I2422dc17e6c73ef346de80e57cdf61ef5d271d69
1. Ignore base python conflict error New version of tox (4.2.4?) fails if there is mismatch in spec attributes [1]. We can change the base python to match with the test envs, but it may make it inconvenient to setup local development env, so just ignore the error to workaround the issue. 2. Allow running bash in bandit test [1] tox-dev/tox#2824
This fails now with 'failed with env name pyinstaller-32 conflicting with base python C:\hostedtoolcache\windows\Python\3.10.9\x86\python.exe' See tox-dev/tox#2824
This PR contains the following updates: | Package | Type | Update | Change | Pending | |---|---|---|---|---| | [tox](https://togithub.com/tox-dev/tox) ([changelog](https://tox.wiki/en/latest/changelog.html)) | dev | patch | `4.2.3` -> `4.2.4` | `4.3.5` (+9) | --- ### Release Notes <details> <summary>tox-dev/tox</summary> ### [`v4.2.4`](https://togithub.com/tox-dev/tox/releases/tag/4.2.4) [Compare Source](https://togithub.com/tox-dev/tox/compare/4.2.3...4.2.4) #### What's Changed - Also accept tab after colon before factor filter expansion by [@​pdecat](https://togithub.com/pdecat) in [https://github.com/tox-dev/tox/pull/2823](https://togithub.com/tox-dev/tox/pull/2823) - Fail on mismatched python spec attributes by [@​stephenfin](https://togithub.com/stephenfin) in [https://github.com/tox-dev/tox/pull/2824](https://togithub.com/tox-dev/tox/pull/2824) **Full Changelog**: tox-dev/tox@4.2.3...4.2.4 </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNC45OS4yIiwidXBkYXRlZEluVmVyIjoiMzQuOTkuMiJ9--> Co-authored-by: descope[bot] <descope[bot]@users.noreply.github.com>
* Update tripleo-image-elements from branch 'master'
to 7adf4508cf88eb9a2d69fb3c8e9468ea318c50ac
- Fix issues with tox 4.2.4
tox now hard fails if there is mismatch in spec
attributes[1]. This works around the issue
temporarily. We probably have to drop py38 jobs
and specify basepython for py39 target in the
future.
Removes ansible-galaxy --timeout for py* jobs to
workaround. We may hit the timeout with py* jobs.
[1] tox-dev/tox#2824
Change-Id: Ie9bacf18cf167139601eff80bba91f2b3454b5dd
tox now hard fails if there is mismatch in spec attributes[1]. This works around the issue temporarily. We probably have to drop py38 jobs and specify basepython for py39 target in the future. Removes ansible-galaxy --timeout for py* jobs to workaround. We may hit the timeout with py* jobs. [1] tox-dev/tox#2824 Change-Id: Ie9bacf18cf167139601eff80bba91f2b3454b5dd
1. Ignore base python conflict error New version of tox (4.2.4?) fails if there is mismatch in spec attributes [1]. We can change the base python to match with the test envs, but it may make it inconvenient to setup local development env, so just ignore the error to workaround the issue. 2. Allow running bash in bandit test [1] tox-dev/tox#2824
1. Ignore base python conflict error New version of tox (4.2.4?) fails if there is mismatch in spec attributes [1]. We can change the base python to match with the test envs, but it may make it inconvenient to setup local development env, so just ignore the error to workaround the issue. 2. Allow running bash in bandit test [1] tox-dev/tox#2824
1. Ignore base python conflict error New version of tox (4.2.4?) fails if there is mismatch in spec attributes [1]. We can change the base python to match with the test envs, but it may make it inconvenient to setup local development env, so just ignore the error to workaround the issue. 2. Allow running bash in bandit test [1] tox-dev/tox#2824
1. Ignore base python conflict error New version of tox (4.2.4?) fails if there is mismatch in spec attributes [1]. We can change the base python to match with the test envs, but it may make it inconvenient to setup local development env, so just ignore the error to workaround the issue. 2. Allow running bash in bandit test [1] tox-dev/tox#2824
1. Ignore base python conflict error New version of tox (4.2.4?) fails if there is mismatch in spec attributes [1]. We can change the base python to match with the test envs, but it may make it inconvenient to setup local development env, so just ignore the error to workaround the issue. 2. Allow running bash in bandit test [1] tox-dev/tox#2824
1. Ignore base python conflict error New version of tox (4.2.4?) fails if there is mismatch in spec attributes [1]. We can change the base python to match with the test envs, but it may make it inconvenient to setup local development env, so just ignore the error to workaround the issue. 2. Allow running bash in bandit test [1] tox-dev/tox#2824
The following is intended to be a correct tox configuration:
The goal of this is to use
python3.8(i.e. the value of[testenv] base_python) for all environments except those with specificpyXXfactors in them. This helps eliminate issues where testenvs that do not specify apyXXfactor run with different Python versions in different environments.An earlier bug, #477, prevented us from doing this. Due to #477, the interpreter version indicated by
[testenv] base_python(or rather[testenv] basepython- the underscore-separated variant was only introduced in tox 4) would override the value indicated by thepyXXfactor. This was resolved with a PR, #841, which started warning users if they were unintentionally running against the wrong interpreter and introduced theignore_basepython_conflictvalue to opt into the correct behavior.Unfortunately, this has been partially broken in the move to tox 4. While the above configuration still works, the following no longer does:
This configuration was common back during the move from Python 2 to Python 3. It ensured that
python3was used for all testenvs that did not request an explicit Python version via a factor or their ownbase_pythonversion. While it's no longer necessary, it is still quite common. Currently, running with this configuration will result inpython3being used for every environment, rather than e.g.python3.8for a testenv with thepy38factor. This is clearly incorrect. This issue stems from an errant bit of logic. When comparing interpreter versions as part of the_validate_base_python, we ignore various attributes if they are unset on either[testenv] base_pythonor thepyXXfactor. This allows e.g.python3to matchpython3.8, since the minor version is unset for the[testenv] base_pythonvalue,python3. The fix is simple: while we can ignore unset attributes for factor-derived versions (to allow apy3factor to work with e.g.base_python = python3.8), we should insist on them forbase_python-derived versions.Closes: #2754