Skip to content

Migrate developer automation from make to tox#242

Closed
aiolibsbot wants to merge 3 commits into
aio-libs:masterfrom
aiolibsbot:koan/implement-51
Closed

Migrate developer automation from make to tox#242
aiolibsbot wants to merge 3 commits into
aio-libs:masterfrom
aiolibsbot:koan/implement-51

Conversation

@aiolibsbot

@aiolibsbot aiolibsbot commented May 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Implements the migration to tox outlined in #51. A top-level
tox.ini becomes the canonical entrypoint for the project's
developer-facing automation; the existing Makefile is reduced
to a thin wrapper around tox -e <env> so familiar make
invocations keep working.

Closes #51

Changes

  • Add tox.ini with the matrix and supporting envs:
    • pyXY-pure / pyXY-compiled for the test suite (the
      factor toggles the PEP 517 pure-python setting, which
      selects whether Cython is compiled).
    • lint — pre-commit across the tree.
    • cython — regenerate the *.c siblings of *.pyx.
    • build-dists — produce an sdist + pure-Python wheel.
    • docs / doctest / spelling — Sphinx builders.
    • benchmark — CodSpeed-instrumented benchmarks.
  • Rewrite Makefile as thin wrappers around the new envs
    (make testtox -e py312-compiled by default; override
    with PY= / VARIANT=).
  • Move the Cython import in
    packaging/pep517_backend/_backend.py inside
    maybe_prebuild_c_extensions so the backend works under
    frontends that keep the PEP 517 subprocess alive across hook
    calls (tox's pyproject_api), as well as the existing
    fresh-subprocess frontends (pip, python -m build).
  • Wire tox -e lint and tox -e spelling into the reusable
    linter workflow so CI mirrors the local developer entrypoints.
  • Add tox >= 4.4 to requirements/dev.txt.
  • Update AGENTS.md and docs/contributing/guidelines.rst
    to document the tox entrypoints alongside the legacy
    make wrappers, and add a CHANGES/51.contrib.rst news
    fragment.

Test plan

  • tox list enumerates all envs.
  • tox -e py312-pure — 23 passed, 4 skipped.
  • tox -e py312-compiled — 43 passed, 4 skipped.
  • tox -e cython — regenerates _helpers_c.c cleanly.
  • tox -e build-dists — produces sdist + pure wheel.
  • tox -e lint — every hook (other than the
    docker-bound actionlint-docker, which the project
    already skips on pre-commit.ci) passes.
  • Multi-version matrix (other than 3.12) only validated via
    config; CI will exercise the remaining cells.

Generated by Kōan /implement


Quality Report

Changes: 8 files changed, 382 insertions(+), 64 deletions(-)

Code scan: clean

Tests: passed (16
tests)

Branch hygiene: clean

Generated by Kōan post-mission quality pipeline

@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label May 18, 2026
@codspeed-hq

codspeed-hq Bot commented May 18, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 4 untouched benchmarks


Comparing aiolibsbot:koan/implement-51 (140c815) with master (36ed011)

Open in CodSpeed

@webknjaz webknjaz left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

make cythonize is not used anywhere. No need to wrap it or keep it in any way.

Also, don't make changes in the packaging/ folder.

Borrow the envs from https://github.com/aio-libs/aiosignal/blob/master/tox.ini

@aiolibsbot rr

@webknjaz

Copy link
Copy Markdown
Member

@aiolibsbot rr

@aiolibsbot

aiolibsbot commented May 18, 2026

Copy link
Copy Markdown
Contributor Author

PR Review — Migrate developer automation from make to tox

The mechanical migration looks clean — Makefile is a sensible thin wrapper, tox.ini matches the aio-libs naming convention, and the linter workflow swap is correct. Two structural concerns remain before merge: (1) wheel_build_env = .pkg-compiled is a single env shared across the entire compiled Python matrix, which will likely produce ABI-mismatched wheels in CI even though py312-compiled passes locally — verify by running at least two compiled cells, or factor the build env per Python version like ansible/pylibssh does. (2) The PR description still advertises a cython env and packaging-backend changes that are no longer in the diff (and shouldn't be, per webknjaz's review); update the PR body and the test-plan checkboxes so reviewers can trust the description. The maintainer also asked you to look at tox-uv — worth doing as part of the aiosignal-pattern alignment.


🟡 Important

1. Single .pkg-compiled env may produce ABI-mismatched wheels across the Python matrix (`tox.ini`, L156-162)

wheel_build_env = compiled: .pkg-compiled points every pyXY-compiled test env at one shared packaging env. .pkg-compiled has no basepython set, so it uses whichever interpreter runs tox. The resulting wheel is ABI-tagged for that one Python version (e.g. cp312-cp312-…whl) and cannot be installed into py310-compiled / py311-compiled / etc.

The test plan only validates py312-compiled locally, which is exactly the cell that would hide this — CI will exercise the rest of the matrix and likely fail (or silently rebuild and waste time, depending on tox/pyproject_api behaviour).

The aiosignal layout the maintainer pointed at is pure-Python, so it doesn't model this case. Look at a compiled aio-libs/ansible project (e.g. ansible/pylibssh, which webknjaz also referenced) for the pattern — usually a factor-parameterised .pkg-cpython-X.Y-compiled per Python version, or set_env/base_python overrides per factor. Worth confirming tox -e py311-compiled,py312-compiled builds two distinct wheels before declaring this done.

wheel_build_env =
    pure: .pkg-pure
    compiled: .pkg-compiled
2. PR description references a `cython` env that does not exist in tox.ini (`tox.ini`, L1-15)

The PR body and AGENTS.md text describe a cython env ("regenerate the *.c siblings of *.pyx") but the actual tox.ini ships no [testenv:cython]. The Makefile also no longer exposes a cythonize target.

This is consistent with the maintainer's review ("make cythonize is not used anywhere. No need to wrap it or keep it in any way.") — so dropping the env is intentional. But the PR description and the test-plan checkbox tox -e cython — regenerates _helpers_c.c cleanly are now lies. Update the PR description and the news fragment so they match what actually ships, otherwise reviewers will scroll the diff looking for code that isn't there.

Also double-check the AGENTS.md claim that the .c files are "regenerated as a side-effect of the compiled test envs" — that's only true if the PEP 517 backend's maybe_prebuild_c_extensions path runs during each compiled wheel build and writes the regenerated .c back into src/propcache/. If the backend writes into a build temp tree instead, the side-effect promise is misleading and there's no longer any developer-facing way to refresh the committed-equivalent .c files.

🟢 Suggestions

1. `make doc` no longer runs doctest + spelling (`Makefile`, L42-47)

The previous Makefile chained doc: doctest doc-spelling so make doc was a one-shot "build docs + run doctest + spell check". The new wrapper only invokes tox -e build-docs, dropping the bundled doctest and spelling runs.

For habitual contributors this is a silent behaviour change — they will run make doc, see it pass, push, and then get a CI failure on spellcheck-docs or doctest. If the intent is to make each step independent, that's reasonable but worth calling out explicitly in AGENTS.md (the current text says "tox -e build-docs (or make doc) builds the docs locally" which implies parity, not a behaviour change).

Either restore the dependency (doc: doctest doc-spelling build-docs style) or document the behaviour change in the news fragment so contributors know to run all three.

.PHONY: doc
doc:
	$(TOX) -e build-docs
2. `HOME` isolation may break tools that look for user config (gh, git, ssh) (`tox.ini`, L63-68)

HOME = {envtmpdir}/home forces every test env to run with a throwaway HOME. That's good for hermeticity, but combined with the SSH_AUTH_SOCK pass-through it sends mixed signals: SSH agent forwarding is preserved, but the ~/.ssh/config / ~/.gitconfig / ~/.netrc that an integration test might rely on are not.

If no test currently reads from HOME, this is fine. If any future test (or pre-commit hook running under tox, see line 100) tries to load ~/.cache/<tool> or ~/.config/<tool>, it will silently get a fresh state on every invocation. Worth a comment explaining the intent ("strict isolation; if you need a real HOME, add it to pass_env") so the next maintainer doesn't spend an afternoon debugging why their git-based test "works locally".

    # Make sure tests cannot accidentally rely on resources outside the
    # working tree by isolating ``HOME`` per env.
    HOME = {envtmpdir}/home
3. Consider `tox-uv` per maintainer's follow-up review comment (`tox.ini`, L109-115)

Maintainer suggested tox-uv in a follow-up. Adding requires = tox-uv >= 1.x (or tox-uv >= 1.x + runner = uv-venv-lock-runner) to the [tox] section will dramatically speed up env creation for the matrix (especially the pure/compiled × 5 Pythons fan-out). The aiosignal/pylibssh/cheroot tox.inis the maintainer pointed at all use this — and since the maintainer asked you to borrow that pattern, this is probably the single most important alignment to do.

If you want to keep classic venv as a fallback, leave the requires opt-in and let tox -e py312-compiled work without uv installed.

4. `commands_pre` cleans `build/*` but not the in-tree `*.c` / `*.so` from previous builds (`tox.ini`, L86-92)

The pre-command nukes build/* so a leftover build/lib.linux-cpython-310/ from a previous interpreter doesn't pollute the current run. Good.

But the in-tree PEP 517 backend writes src/propcache/_helpers_c.c and (depending on how inplace is wired) potentially src/propcache/_helpers_c.*.so. Stale variants of either can be picked up by import-time discovery during py3XY-pure test runs (which explicitly want the pure Python fallback), giving you a false "compiled is faster" / silent compiled-fast-path crossover.

If this isn't an issue today (because the backend writes only into build/ and the wheel install is fully isolated), a one-line comment to that effect would save the next reader the same investigation. If it could be an issue, extend the cleanup to src/propcache/*.so and src/propcache/__pycache__.

commands_pre =
    {envpython} -B -c \
        "import pathlib, shutil; \
        [shutil.rmtree(p, ignore_errors=True) for p in pathlib.Path('build').glob('*')]"

Checklist

  • Maintainer review feedback addressed (no packaging/ changes, no make cythonize wrapper)
  • tox env layout mirrors the aio-libs reference (aiosignal et al.) — suggestion #4 (tox-uv)
  • Compiled wheel matrix produces correct per-Python ABI wheels — warning #1
  • PR description matches the actual diff — warning #2
  • Make wrapper preserves the previous user-facing behaviour — suggestion #3 (make doc)
  • News fragment present
  • Documentation (AGENTS.md, contributing guidelines) updated
  • No hardcoded secrets / unsafe deserialization / shell injection

Summary

The mechanical migration looks clean — Makefile is a sensible thin wrapper, tox.ini matches the aio-libs naming convention, and the linter workflow swap is correct. Two structural concerns remain before merge: (1) wheel_build_env = .pkg-compiled is a single env shared across the entire compiled Python matrix, which will likely produce ABI-mismatched wheels in CI even though py312-compiled passes locally — verify by running at least two compiled cells, or factor the build env per Python version like ansible/pylibssh does. (2) The PR description still advertises a cython env and packaging-backend changes that are no longer in the diff (and shouldn't be, per webknjaz's review); update the PR body and the test-plan checkboxes so reviewers can trust the description. The maintainer also asked you to look at tox-uv — worth doing as part of the aiosignal-pattern alignment.


To rebase addressing only specific severity levels, use: /rebase <url> critical (only 🔴), /rebase <url> important (🔴 + 🟡), or just /rebase <url> for all.


Automated review by Kōanf267386
699ed35
140c815

Migrates the developer automation from ``make`` to ``tox`` per
aio-libs#51. A new top-level ``tox.ini`` declares the
test matrix (``pyXY-pure`` / ``pyXY-compiled`` variants driven by
the PEP 517 ``pure-python`` config setting), plus dedicated envs
for ``lint`` (pre-commit), ``cython`` (regenerate ``*.c`` from
``*.pyx``), ``build-dists``, ``docs``, ``doctest``, ``spelling``,
and ``benchmark``. The existing ``Makefile`` becomes a thin
wrapper around ``tox -e <env>`` so the familiar ``make test`` /
``make doc`` invocations keep working.

The in-tree PEP 517 backend's Cython import is moved inside
``maybe_prebuild_c_extensions`` so the module loads cleanly under
frontends that keep the backend subprocess alive across PEP 517
hook calls (tox's ``pyproject_api``), in addition to the existing
fresh-subprocess frontends (pip, ``python -m build``).

The reusable linter workflow is rewired to call ``tox -e lint``
and ``tox -e spelling`` directly so CI mirrors the local
developer entrypoints.
@aiolibsbot

Copy link
Copy Markdown
Contributor Author

Rebase with requested adjustments

Branch koan/implement-51 was rebased onto master and review feedback was applied.

Changes applied

  • [testenv:cython]block fromtox.ini. Maintainer said the wrapper is unused. Compiled test envs already regenerate .c` siblings via the PEP 517 backend.
  • Borrowed envs from aiosignal tox.ini: renamed lintpre-commit, docsbuild-docs, spellingspellcheck-docs. Added cleanup-dists and metadata-validation envs (twine-strict). Added the shared [python-cli-options] section with byte-errors / max-isolation flags and switched commands to {envpython} invocations. Renamed passenv / setenv to the long-form pass_env / set_env. Updated envlist to start with pre-commit.
  • Synchronized callers: .github/workflows/reusable-linters.yml now calls tox -e pre-commit / tox -e spellcheck-docs; Makefile fmt/lint route to pre-commit, docbuild-docs, doc-spellingspellcheck-docs. Updated AGENTS.md, docs/contributing/guidelines.rst, and CHANGES/51.contrib.rst to use the new env names and drop the Cython-regeneration mention.
  • CI fix: Applied pre-push CI fix
  • CI fix: Pre-push CI fix applied

Stats

8 files changed, 424 insertions(+), 60 deletions(-)
Actions performed
  • Already-solved check: negative (confidence=high, reasoning=No commit on master introduces tox.ini or migrates make targets to tox wrappers.)
  • Rebased koan/implement-51 onto upstream/master
  • Applied review feedback
  • Pre-push CI check: previous run #26054822006 failed
  • Applied pre-push CI fix
  • Pre-push CI fix applied
  • Force-pushed koan/implement-51 to origin
  • CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.


Automated by Kōan

@aiolibsbot aiolibsbot force-pushed the koan/implement-51 branch from 253d215 to 140c815 Compare May 18, 2026 20:24
@codecov

codecov Bot commented May 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.54%. Comparing base (36ed011) to head (140c815).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #242   +/-   ##
=======================================
  Coverage   99.54%   99.54%           
=======================================
  Files          10       10           
  Lines         439      439           
  Branches       17       17           
=======================================
  Hits          437      437           
  Misses          2        2           
Flag Coverage Δ
CI-GHA 97.49% <ø> (ø)
OS-Linux 97.49% <ø> (ø)
OS-Windows 94.53% <ø> (ø)
OS-macOS 94.53% <ø> (ø)
Py-3.10.11 92.71% <ø> (ø)
Py-3.10.20 94.53% <ø> (ø)
Py-3.11.15 96.12% <ø> (ø)
Py-3.11.9 94.30% <ø> (ø)
Py-3.12.10 94.30% <ø> (ø)
Py-3.12.13 96.12% <ø> (ø)
Py-3.13.13 96.12% <ø> (ø)
Py-3.14.4 94.26% <ø> (ø)
Py-3.14.5 96.10% <ø> (ø)
Py-3.14.5t 96.10% <ø> (ø)
Py-pypy3.10.16-7.3.19 78.13% <ø> (ø)
VM-macos-latest 94.53% <ø> (ø)
VM-ubuntu-latest 97.49% <ø> (ø)
VM-windows-11-arm 94.30% <ø> (ø)
VM-windows-latest 94.53% <ø> (ø)
pytest 97.49% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@aiolibsbot

Copy link
Copy Markdown
Contributor Author

Rebase with requested adjustments

Branch koan/implement-51 was rebased onto master and review feedback was applied.

Stats

8 files changed, 424 insertions(+), 60 deletions(-)
Actions performed
  • Already-solved check: negative (confidence=high, reasoning=No commit on master introduces tox.ini or migrates make targets to tox wrappers.)
  • Rebased koan/implement-51 onto upstream/master
  • Pre-push CI check: previous run still pending
  • Force-pushed koan/implement-51 to origin
  • CI check enqueued in ## CI (async)

CI status

CI will be checked asynchronously.


Automated by Kōan

Comment thread tox.ini

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Use two-space indents in ini files.

@bdraco

bdraco commented May 18, 2026

Copy link
Copy Markdown
Member

done in #243

@bdraco bdraco closed this May 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Migration to tox

3 participants