Skip to content

Add pyproject.toml, and stop calling setup.py directly#629

Merged
graebm merged 14 commits intomainfrom
pyproject-toml
Mar 14, 2025
Merged

Add pyproject.toml, and stop calling setup.py directly#629
graebm merged 14 commits intomainfrom
pyproject-toml

Conversation

@graebm
Copy link
Copy Markdown
Contributor

@graebm graebm commented Mar 13, 2025

Issue:

For a few years now, we've been seen the occasional deprecation warning about setup.py. Including:

********************************************************************************
Please avoid running ``setup.py`` directly.
Instead, use pypa/build, pypa/installer or other
standards-based tools.

See https://blog.ganssle.io/articles/2021/10/setup-py-deprecated.html for details.
********************************************************************************

and

********************************************************************************
Ensure that any custom bdist_wheel implementation is a subclass of
setuptools.command.bdist_wheel.bdist_wheel.

By 2025-Oct-15, you need to update your project and remove deprecated calls
or your builds will no longer be supported.

See https://github.com/pypa/wheel/pull/631 for details.
********************************************************************************

2025-Oct is on its way, so let's finally do something about it...

Description of changes:

Follow advice from "Is setup.py deprecated?" page at packaging.python.org

tldr; having a setup.py file is not deprecated, but directly invoking python setup.py is deprecated.

  • Add pyproject.toml file (the modern way to configure your package)
    • Move data from setup.py -> pyproject.toml wherever possible (can't move dynamic stuff, like build steps for C extension)
    • Move dev dependencies from requirements-dev.txt -> pyproject.toml [project.optional-dependencies]
      • Now you call pip install ".[dev]", instead of pip install -r requirements-dev.txt. The dev dependencies will be installed by the same command that builds and installs our project.
      • This change isn't 100% necessary, but seemed like the modern way to do things. It's one less step to set things up on your machine. However, you can't install dependencies without ALSO building/installing the package (we can roll this back if we end up hating it)
  • Use python3 -m build instead of python3 setup.py sdist bdist_wheel (the modern way to build a wheel for distribution)
    • Sadly, build is a non-core package, so might need to be pip installed before using. Fortunately it's been preinstalled on manylinux images since 2021, so our Linux release process will be OK.
    • I updated our MacOS and Windows machines to have build installed for all python versions, so our release process should be OK.
  • STOP calling pip install --upgrade setuptools before pip install .
    • Modern pip installs the "build dependencies" declared in pyproject.toml (setuptools & wheel) in a special isolated environment before it builds our project. We declared setuptools>=75.3.1, so we're guaranteed it will get installed, and be a recent version.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

graebm added 9 commits March 13, 2025 10:17
what's up with: WARNING: unknown 1.0.0.dev0 does not provide the extra 'dev'
why does it think the project's name is "unknown"?
why can't it find the "dev" in [project.optional-dependencies]?
try upgrading pip and see if that fixes it
…oblems where an old version doesn't work.

Pick the latest version that still supports Python 3.8
…wscrt.

We hit some weird cases in "check-docs" CI where an old pip could find the project name in pyproject.toml.
Let's avoid future problems by ensuring pip is up to date
run: |
python3 -m pip install --upgrade --requirement requirements-dev.txt
python3 -m pip install . --verbose
python3 -m pip install --upgrade pip
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

semi related: do we still think upgrading pip is a good idea. i remember we had some issues in the past year with python blocking global pip upgrade in some cases and instead pointing to upgrading pip within venv

Copy link
Copy Markdown
Contributor Author

@graebm graebm Mar 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll try taking these out and see what happens ... and pip failed to build our project, like before.

also tried moving the job to ubuntu-24.04, which would have newer pip ... and that failed too due to formatting errors raised by sphinx.

The docs are sadly very fragile. Seem to only work with certain sphinx-versions combined with certain python-versions. Don't have time to diagnose. Just making it work for now :(

graebm added 5 commits March 13, 2025 17:41
maybe this was superstition, maybe declaring a min version of setuptools was the real fix to the problems I was seeing
not sure why install fails without pip upgrade
not sure why sphinx fails with newer ubuntu
@graebm graebm merged commit bc205fa into main Mar 14, 2025
63 checks passed
@graebm graebm deleted the pyproject-toml branch March 14, 2025 16:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants