Skip to content

Integrate tox-dev/workflow's reusable-tox.yml#243

Merged
bdraco merged 22 commits into
masterfrom
ci-tox-dev-workflow
May 18, 2026
Merged

Integrate tox-dev/workflow's reusable-tox.yml#243
bdraco merged 22 commits into
masterfrom
ci-tox-dev-workflow

Conversation

@bdraco

@bdraco bdraco commented May 18, 2026

Copy link
Copy Markdown
Member

What do these changes do?

Adds a top-level tox.ini and integrates
tox-dev/workflow's
reusable-tox.yml into propcache's CI/CD. The
build-pure-python-dists job is replaced by a build job that
runs the new build-dists tox env through the reusable
workflow, and a new lint job runs pre-commit the same way.
The existing reusable-linters.yml stays in place as
retro-lint so MyPy/Coveralls, twine check and
make doc-spelling keep gating CI during the transition.

Mirrors aio-libs/aiosignal#692 (build job) and
aio-libs/aiosignal#693 (lint/retro-lint split + pre-commit
cache hook).

Are there changes in behavior for the user?

No. CI-only change; end-user-visible behavior of propcache is
unaffected.

Related issue number

N/A; tracks aio-libs/aiosignal#692 and aio-libs/aiosignal#693.

Checklist

  • I think the code is well written
  • Unit tests for the changes exist — N/A (CI configuration)
  • Documentation reflects the changes — N/A (no user-visible change)
  • Add a new news fragment into the CHANGES/ folder
Agent run details (optional, for reviewers)

Drafted with Claude Code (Opus 4.7); reviewed by @bdraco.

Local verification:

  • tox r -e build-dists produces propcache-0.5.2.tar.gz and
    propcache-0.5.2-py3-none-any.whl in dist/.
  • tox r -e metadata-validation passes on both dists.
  • tox r -e build-docs builds the Sphinx HTML output.
  • tox r -e doctest-docs runs the doc doctests.
  • tox r -e spellcheck-docs reproduces the two pre-existing
    spelling failures on master (subdirectory in
    CHANGES/207.contrib.rst, postfix in CHANGES.rst:150) and
    nothing new; the new CHANGES/243.contrib.rst fragment is
    clean after adding env and tox to
    docs/spelling_wordlist.txt.
  • Pre-commit pass on every modified file (yamllint, actionlint,
    codespell, GHA workflow validators).

Scope intentionally narrow:

  • The new lint matrix only enables pre-commit; the other
    envs (metadata-validation, build-docs, doctest-docs,
    spellcheck-docs) are defined in tox.ini but commented out
    in the matrix. Flipping them on is a follow-up once
    retro-lint is retired.
  • Test matrix and benchmark job are untouched; migrating them
    requires per-cell binary wheel glob construction and is a
    separate follow-up.

Note: PR #242 (Migrate developer automation from make to tox)
opened in parallel introduces an alternate tox.ini with
different env naming (py312-compiled / lint / spelling).
This PR uses the aiosignal-aligned naming (pre-commit,
build-dists, metadata-validation, etc.) so it matches the
tox-dev/workflow ecosystem. Whichever lands first will need a
follow-up merge.

Mirrors aio-libs/aiosignal#692 and aio-libs/aiosignal#693 for
propcache.

Adds a top-level `tox.ini` with envs for building dists,
metadata validation, pre-commit, and the doc builders so the
same automation runs locally and in CI.

Expands `pre-setup` to compute the cache key, project name, dist
version (parsed from `src/propcache/__init__.py`) and the exact
sdist/wheel artifact filenames so reusable workflows and hooks
can address them by name.

Replaces `build-pure-python-dists` with a `build` job that calls
`tox-dev/workflow/.github/workflows/reusable-tox.yml@89de3c6` with
`toxenv: build-dists`. A new `.github/reusables/.../post-tox-run`
hook verifies the expected artifact names exist and uploads them
under `python-package-distributions` so the cibuildwheel, test,
benchmark and deploy jobs keep finding them.

Adds a new `lint` job that runs `pre-commit` through the same
reusable workflow, with a `post-src-checkout` hook for caching
`~/.cache/pre-commit`. The existing `reusable-linters.yml` is
kept as `retro-lint` so MyPy coverage upload to Coveralls,
`twine check` and `make doc-spelling` continue to gate CI during
the transition. Other lint envs (`metadata-validation`,
`build-docs`, `doctest-docs`, `spellcheck-docs`) are defined in
`tox.ini` but left commented out in the matrix; flipping them on
is a follow-up once `retro-lint` is retired.

Adds `env` and `tox` to the spell-check wordlist for the
`CHANGES/243.contrib.rst` fragment.
@psf-chronographer psf-chronographer Bot added the bot:chronographer:provided There is a change note present in this PR label May 18, 2026
@codspeed-hq

codspeed-hq Bot commented May 18, 2026

Copy link
Copy Markdown

Merging this PR will improve performance by 29.57%

⚠️ Different runtime environments detected

Some benchmarks with significant performance changes were compared across different runtime environments,
which may affect the accuracy of the results.

Open the report in CodSpeed to investigate

⚡ 4 improved benchmarks

Performance Changes

Benchmark BASE HEAD Efficiency
test_cached_property_cache_miss 134.4 µs 102.4 µs +31.29%
test_cached_property_cache_hit 33.5 µs 25.6 µs +30.99%
test_under_cached_property_cache_hit 70.4 µs 54.6 µs +29.02%
test_under_cached_property_cache_miss 128.9 µs 101.5 µs +27.05%

Tip

Curious why this is faster? Comment @codspeedbot explain why this is faster on this PR, or directly use the CodSpeed MCP with your agent.


Comparing ci-tox-dev-workflow (1611618) with master (36ed011)

Open in CodSpeed

Comment thread .github/workflows/ci-cd.yml
Comment thread .github/workflows/ci-cd.yml Outdated
Comment thread .github/workflows/ci-cd.yml
@bdraco

bdraco commented May 18, 2026

Copy link
Copy Markdown
Member Author

Still some of the slowness is from using actions/setup-python instead of astral-sh/setup-uv which has the prebuilt uv

Comment thread CHANGES/243.contrib.rst
@bdraco

bdraco commented May 18, 2026

Copy link
Copy Markdown
Member Author

Comment thread .github/workflows/reusable-linters.yml Outdated
bdraco added 4 commits May 18, 2026 13:44
The `Setup Python` step in the test matrix was taking ~62s on
`windows-latest` for `3.14t` because hosted runners don't keep
free-threaded CPython in the tool-cache and `actions/setup-python`
falls back to downloading it from the `python-versions` repo on
each run.

Switches the test matrix, the benchmark job, and the linter job
in `reusable-linters.yml` to `astral-sh/setup-uv@v8.1.0` with
`activate-environment: true` and `enable-cache: true`. uv pulls
python-build-standalone from a CDN in a few seconds and persists
its wheel cache across runs in place of the
`re-actors/cache-python-deps` wrapper.

Drops the `pip --dry-run --report=- | jq` wheel-discovery trick
and the follow-up `pip install <url>` from both the test and
benchmark jobs in favour of a single `uv pip install
--find-links=./dist --no-index --no-deps --force-reinstall
--only-binary=:all: propcache` call; uv resolves the matching
wheel for the active interpreter directly.

Also drops the `tox-tool-deps: tox` override on the `build` and
`lint` jobs so the reusable workflow's default `tox tox-uv` kicks
in and tox env installs use uv instead of pip.

`allow-prereleases` has no equivalent on setup-uv; uv treats
prereleases as first-class. The Codecov flag derived from
`steps.python-install.outputs.python-version` now reports the
matrix spec (`3.14t`) rather than the resolved patch (`3.14.5t`).
The new lint matrix already wires `built-wheel-names` for the
`metadata-validation` toxenv and the `tox.ini` env depends on
`build-dists`, so flipping it on downloads the pre-built dists
from the `build` job's artifact and runs `twine check --strict
dist/*` exactly like the old `Run twine checker` step.

That makes the `Prepare twine checker` (`uv pip install -U build
twine` + a redundant in-place `python -m build`) and the
`Run twine checker` steps in `reusable-linters.yml` redundant.
Drop them. `retro-lint` now only covers MyPy/Coveralls and
`make doc-spelling`; everything else has moved to the tox-dev
matrix.
`metadata-validation` was running in the tox-dev/workflow lint
matrix with `built-wheel-names` set to the single pure-Python
wheel, so `twine check` only saw `propcache-*-py3-none-any.whl`
and the sdist. The cibuildwheel matrix produces a binary wheel
for every Python + OS + arch cell and those went unchecked.

Pulls `metadata-validation` out of the lint matrix into a
standalone job that depends on both `build` (sdist + pure
wheel) and `build-wheels-for-tested-arches` (the full
cibuildwheel matrix), downloads every artifact whose name
starts with `python-package-distributions`, and runs
`tox r -e metadata-validation` — which is `twine check --strict
dist/*` and now covers every built dist.

Setup is uv-only: `astral-sh/setup-uv@v8.1.0`, then `uv pip
install tox tox-uv` bootstraps tox without touching pip; the
env itself runs under `UvVenvRunner` and installs `twine` via
uv.
Comment thread .github/workflows/reusable-linters.yml Outdated
webknjaz flagged
#243 (comment)
that the `wheel-artifact-name` output reads like *the* wheel
filename, but propcache produces a matrix of binary wheels via
cibuildwheel as well; only the pure-Python wheel goes through
the `build` job and matches this format.

Renames the output to `pure-python-wheel-artifact-name` and
updates the `post-tox-run` hook accordingly. Also drops the
now-dead `built-wheel-names` expression from the lint matrix
since `metadata-validation` moved to its own job in the
previous commit and was the only consumer.
Comment thread .github/workflows/reusable-linters.yml Outdated
The `spellcheck-docs` toxenv already exists and the lint matrix
expression already special-cases it (Git checkout instead of
sdist, longer 6-minute timeout for the I/O). Flipping it on
gives the same `make -C docs spelling` coverage with one fewer
hand-rolled step in `reusable-linters.yml`.

retro-lint now only carries the MyPy/Coveralls upload.
Comment thread .github/workflows/reusable-linters.yml Outdated
Comment thread .github/workflows/ci-cd.yml Outdated
…-lint

The `pre-commit` toxenv in the new lint matrix already runs
pre-commit (and through it, the MyPy hook that writes
`.tox/.tmp/.mypy/python-3.11/cobertura.xml`). Move the Coveralls
upload of that report into a `post-tox-job` hook gated on
`toxenv == 'pre-commit'`, mirroring the awx-plugins pattern flagged
in
#243 (comment).

With that, `retro-lint` has no more unique coverage:
`spellcheck-docs`, `metadata-validation`, and `pre-commit` all run
through tox-dev/workflow. Drop the `retro-lint` job and delete
`reusable-linters.yml`. `test-summary.needs` is updated to drop
`retro-lint`.

Also tightens the `calling-job-context` input description across
the three hooks (`post-src-checkout`, `post-tox-run`,
`post-tox-job`).
Comment thread CHANGES/243.contrib.rst Outdated
bdraco added 4 commits May 18, 2026 14:07
… propcache

`sphinxcontrib-towncrier` invokes `python -m towncrier build --draft`
to render the unreleased changelog into `docs/changes.rst`.
Towncrier reads `tool.towncrier.package = "propcache"` and tries
to import `propcache` to look up the project name, which fails in
the `spellcheck-docs` tox env because it has `package = skip` and
no installed propcache:

    ERROR: tried to import propcache, but ran into this error:
    No module named 'propcache'

`towncrier.get_project_name` returns the configured `name` value
without importing when it's set. Set `name = "propcache"` so the
docs build no longer needs the package to be importable.

(Versioning is unaffected: `sphinxcontrib-towncrier` always passes
`--version`, so the version lookup path that still imports the
package is not exercised here.)
The fragment described the transitional state where `retro-lint`
still ran the legacy linter reusable workflow alongside the new
tox-dev matrix. That workflow has been removed; coverage now
breaks down as:

- `pre-commit` and `spellcheck-docs` run through the tox-dev/workflow
  `lint` matrix.
- MyPy coverage upload to Coveralls happens via the `post-tox-job`
  hook.
- `twine check` runs in a dedicated `metadata-validation` job
  against every built dist (pure-python wheel plus every
  cibuildwheel matrix output).

Updates the news fragment to match.
bdraco added 2 commits May 18, 2026 14:23
The tox envs were already defined and run cleanly locally. They
were left commented out by inheritance from the aiosignal narrow
rollout pattern; enabling them gates docs build failures and
exercises any doctest blocks that get added later.
The `lint` matrix cells that aren't `pre-commit` or
`spellcheck-docs` (i.e. `metadata-validation`, `build-docs`,
`doctest-docs`) source their working directory from the sdist
artifact via `re-actors/checkout-python-sdist`. `MANIFEST.in`
didn't list `tox.ini`, so the extracted sdist had no tox
config and `tox -e doctest-docs --notest` failed with
`provided environments not found in configuration file`.

Add `include tox.ini` so the sdist carries the canonical tox
config used by the reusable workflow.
@bdraco bdraco changed the title ci: integrate tox-dev/workflow's reusable-tox.yml integrate tox-dev/workflow's reusable-tox.yml May 18, 2026
@codecov

codecov Bot commented May 18, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 99.54%. Comparing base (36ed011) to head (1611618).

Additional details and impacted files
@@           Coverage Diff           @@
##           master     #243   +/-   ##
=======================================
  Coverage   99.54%   99.54%           
=======================================
  Files          10       10           
  Lines         439      439           
  Branches       17       17           
=======================================
  Hits          437      437           
  Misses          2        2           
Flag Coverage Δ
CI-GHA 97.49% <ø> (ø)
OS-Linux 97.49% <ø> (ø)
OS-Windows 94.53% <ø> (ø)
OS-macOS 94.53% <ø> (ø)
Py-3.10 94.53% <ø> (?)
Py-3.10.11 ?
Py-3.10.20 ?
Py-3.11 96.12% <ø> (?)
Py-3.11.15 ?
Py-3.11.9 ?
Py-3.12 96.12% <ø> (?)
Py-3.12.10 ?
Py-3.12.13 ?
Py-3.13 96.12% <ø> (?)
Py-3.13.13 ?
Py-3.14 96.10% <ø> (?)
Py-3.14.4 ?
Py-3.14.5 ?
Py-3.14.5t ?
Py-3.14t 96.10% <ø> (?)
Py-pypy-3.10 78.13% <ø> (?)
Py-pypy3.10.16-7.3.19 ?
VM-macos-latest 94.53% <ø> (ø)
VM-ubuntu-latest 97.49% <ø> (ø)
VM-windows-11-arm 94.30% <ø> (ø)
VM-windows-latest 94.53% <ø> (ø)
pytest 97.49% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

bdraco added 4 commits May 18, 2026 14:46
…source

`metadata-validation`, `build-docs`, and `doctest-docs` use the
sdist as their working directory in CI (via
`re-actors/checkout-python-sdist`). The tox-dev/workflow reusable
gates each hook on `hashFiles(...) != ''`, which evaluates against
the extracted sdist tree — so any hook not packaged into the sdist
is silently skipped.

That's exactly how `metadata-validation` blew up: the
`post-src-checkout` step that downloads every
`python-package-distributions*` artifact into `dist/` was skipped,
so `twine check dist/*` had nothing to validate and failed with
`InvalidDistribution: Cannot find file (or expand pattern)`.

Add `graft .github/reusables` so the three hook action.yml files
travel with the sdist.
Force uv to always provision its own python-build-standalone
interpreter instead of falling back to the hosted runner's
preinstalled CPython. Keeps the test + benchmark matrices
deterministic across runner OS images, and means slow
free-threaded / pre-release variants come from the uv CDN every
time rather than depending on whether the hosted tool-cache
happens to carry them on that runner.
Mirrors the pre-release detection tox-dev/workflow uses for
`continue-on-error`: tilde-pinned latest, `-dev` suffix, or an
alpha/beta/rc segment. For those interpreters the wheel ABI is
volatile across snapshots, so each run resolves freshly. Stable
interpreters (including `3.14t`) keep the warm uv cache.
@bdraco bdraco changed the title integrate tox-dev/workflow's reusable-tox.yml Integrate tox-dev/workflow's reusable-tox.yml May 18, 2026
@bdraco bdraco marked this pull request as ready for review May 18, 2026 22:25
@bdraco bdraco merged commit e6a5b70 into master May 18, 2026
59 of 60 checks passed
@bdraco bdraco deleted the ci-tox-dev-workflow branch May 18, 2026 22:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bot:chronographer:provided There is a change note present in this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants