DEP: migrate dev-only extra dependencies to PEP 735 dependency groups #17623
Description
This was initially proposed by @nstarman in #16963 (comment)
PEP 735, recently accepted, defines standard syntax to specify dev-only dependencies in pyproject.toml in a way that is not visible by users, contrary to extra dependencies defined in the [project.optional-dependencies] table.
This is a natural fit for some of our dependencies, currently defined as extras for convenience, but not intended for users to consume, e.g.
test_alltypingdocsdevdev_all
There are however 2 issues to be addressed before we can hope to migrate: tooling, and backward compatibility.
Tooling
Dependency groups are already supported by tox, cibuildwheel, pixi, uv and presumably other modern installers, but not by pip yet. pixi/uv can replace pip (and do much more too), though I would assume most of us don't want to (or can't) move away from pip at the moment and even if it was only a minority, we still don't want to leave them behind, so that's a blocker in my book. Here's the issue to keep track of
Backward compatibility
Dependencies like docs are not intended for user consumption, however they are exposed at the moment (it is possible to pip install astropy[docs] from PyPI), so hiding them might be considered a breaking change, and should probably only be allowed in a major release.
If we worry about potential disruption from breaking these suddenly, there is a way to deprecate them first:
- create and publish a marker (empty) package
astropy-docs-extra - add it to the
astropy[docs]extra - at runtime, make
import astropy...raise a deprecation warning ifastropy-docs-extrais installed
Credit to urllib3 for this clever deprecation strategy and to Anthony Sottile for advertising it on youtube.
In my opinion it's largelly overkill in our case; this hack was designed for a library with massively more users than astropy, and was used to deprecate an extra that was intended for user consumption, but I wanted to show that it existed for completeness.