Skip to content

Generate requirements files from pyproject.toml#7085

Merged
jarrodmillman merged 5 commits intoscikit-image:mainfrom
mwtoews:gen-requirements-from-pyproject
Aug 7, 2023
Merged

Generate requirements files from pyproject.toml#7085
jarrodmillman merged 5 commits intoscikit-image:mainfrom
mwtoews:gen-requirements-from-pyproject

Conversation

@mwtoews
Copy link
Contributor

@mwtoews mwtoews commented Aug 4, 2023

This PR is based on numpy/numpydoc#483 which makes pyproject.toml a primary file without any template, and adds a pre-commit script to write requirements files. Some of the comments from the requirements files are moved to pyproject.toml.

@mwtoews
Copy link
Contributor Author

mwtoews commented Aug 4, 2023

Question: why is there a "default" extra? It repeats the same info in the dependencies. The script can easily be modified to generate this file too, if that extra were removed.

@lagru lagru added the 🤖 type: Infrastructure CI, packaging, tools and automation label Aug 4, 2023
@lagru
Copy link
Member

lagru commented Aug 4, 2023

Hey @mwtoews thanks for the suggestion! It seems useful to me to have the source of truth all originate from one place. Especially because it may make it easier to use comments in the pyproject.toml. Before I review, I am curious, did the previous discussion in numpydoc bring up any other pros or cons?

The default.txt is used in several places in the CI to install the runtime dependencies in a controlled manner independent of the package itself e.g. see this search. I am not sure if there is a clear advantage to doing it that way but for now would hope to keep the noise to our infrastructure minimal.

* Generate default.txt from [project.dependencies]
* Remove [project.optional-dependencies.default]
* Update requirements/README.md with new entries with default at top
@mwtoews
Copy link
Contributor Author

mwtoews commented Aug 4, 2023

Before I review, I am curious, did the previous discussion in numpydoc bring up any other pros or cons?

Most of the discussion was from a superseded PR numpy/numpydoc#481 where I was aiming to use a dynamic tool-dependant feature of setuptools to read optional dependencies (rather than using a script and template approach). Ultimately, there are more strengths in having a single statically-analyzable pyproject.toml (complete with all dependencies), but some tools also need to pip install -r with individual requirements files too. The applied PR for numpydoc and in this PR is to also remove the template approach, since requirements files don't really need a template (except for a comment header). A clear benefit is that edits to pyproject.toml now directly go to that file, and where present, editors may enable syntax highlighting (template files can't do that).

As for the requirements/default.txt file, I have not problems with it. But in pyproject.toml it is duplicated. From pip's perspective pip install scikit-image and pip install scikit-image[default] do the same thing. My guess is removing the default extra and generating a requirements files from [project.dependencies] will not change anything, and resolve the duplication.

Copy link
Member

@lagru lagru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing my suggestions! I have only a minor one left.

mwtoews and others added 2 commits August 6, 2023 08:51
Update the hooks name and make the trigger pattern a
more specific, in case we include other tool scripts
with a "generate" in their file name.
Copy link
Member

@lagru lagru left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for addressing all my concerns so quickly! I tweaked the hook configuration a bit in 8d8bf24 to make it more specific otherwise ready to merge IMO.

@jarrodmillman jarrodmillman merged commit 89fa48a into scikit-image:main Aug 7, 2023
@stefanv stefanv added this to the 0.22 milestone Aug 7, 2023
@mwtoews mwtoews deleted the gen-requirements-from-pyproject branch August 7, 2023 18:00
jarrodmillman added a commit to jarrodmillman/networkx that referenced this pull request Oct 5, 2023
This PR is based on numpy/numpydoc#483 and scikit-image/scikit-image#7085,
which make pyproject.toml a primary file without any template, and adds a pre-commit script to write requirements files.
It doesn't really change anything, but I would like to keep everything
consistent.
jarrodmillman added a commit to networkx/networkx that referenced this pull request Oct 6, 2023
* Generate requirements files from pyproject.toml

This PR is based on numpy/numpydoc#483 and scikit-image/scikit-image#7085,
which make pyproject.toml a primary file without any template, and adds a pre-commit script to write requirements files.
It doesn't really change anything, but I would like to keep everything
consistent.

* Cleanup formatting
Alex-Markham pushed a commit to Alex-Markham/networkx that referenced this pull request Oct 13, 2023
* Generate requirements files from pyproject.toml

This PR is based on numpy/numpydoc#483 and scikit-image/scikit-image#7085,
which make pyproject.toml a primary file without any template, and adds a pre-commit script to write requirements files.
It doesn't really change anything, but I would like to keep everything
consistent.

* Cleanup formatting
dschult pushed a commit to BrunoBaldissera/networkx that referenced this pull request Oct 23, 2023
* Generate requirements files from pyproject.toml

This PR is based on numpy/numpydoc#483 and scikit-image/scikit-image#7085,
which make pyproject.toml a primary file without any template, and adds a pre-commit script to write requirements files.
It doesn't really change anything, but I would like to keep everything
consistent.

* Cleanup formatting
cvanelteren pushed a commit to cvanelteren/networkx that referenced this pull request Apr 22, 2024
* Generate requirements files from pyproject.toml

This PR is based on numpy/numpydoc#483 and scikit-image/scikit-image#7085,
which make pyproject.toml a primary file without any template, and adds a pre-commit script to write requirements files.
It doesn't really change anything, but I would like to keep everything
consistent.

* Cleanup formatting
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

🤖 type: Infrastructure CI, packaging, tools and automation

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants