Add a project section to pyproject.toml, making uv sync work#153020
Closed
ezyang wants to merge 2 commits intogh/ezyang/3069/basefrom
Closed
Add a project section to pyproject.toml, making uv sync work#153020ezyang wants to merge 2 commits intogh/ezyang/3069/basefrom
ezyang wants to merge 2 commits intogh/ezyang/3069/basefrom
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/153020
Note: Links to docs will display an error until the docs builds have been completed. ❌ 11 New FailuresAs of commit 90a6b77 with merge base 08f5371 ( NEW FAILURES - The following jobs have failed:
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
Collaborator
|
After moving dependencies to pyproject.toml, we should remove requirements.txt. Possibly fix all commands to use pyproject.toml? |
Contributor
Author
|
@cyyever It's going to be more work to dump requirements.txt because unfortunately I'm guessing people are depending on it somehow... |
Collaborator
|
Let me propose a PR later for moving. |
Skylion007
reviewed
May 7, 2025
| version = "2.8.0a0" | ||
| description = "Tensors and Dynamic neural networks in Python with strong GPU acceleration" | ||
| readme = "README.md" | ||
| requires-python = ">=3.9" |
Collaborator
There was a problem hiding this comment.
So, if we update ruff we can now remove the ruff specific Python min version, which is nice.
| @@ -1,3 +1,34 @@ | |||
| [project] | |||
| name = "torch" | |||
| version = "2.8.0a0" | |||
Collaborator
There was a problem hiding this comment.
Ugh, I wish there was a way to automatically infer the version
Collaborator
There was a problem hiding this comment.
Suggested change
| version = "2.8.0a0" | |
| dynamic = ["version"] |
Make the build backend responsible for it.
| readme = "README.md" | ||
| requires-python = ">=3.9" | ||
| license = {text = "BSD"} | ||
| dependencies = [ |
Collaborator
There was a problem hiding this comment.
This adds another place to maintain the dependencies and make sure they all stay in sync...
| "pyyaml", | ||
| "requests", | ||
| # issue on Windows after >=75.8.2 – https://github.com/pytorch/pytorch/issues/148877 | ||
| "setuptools>=62.3.0,<75.9", |
Collaborator
There was a problem hiding this comment.
Do we sill not have a fix for the setuptools stuff? This is going to hurt Python 3.14 since it won't support setuptools that old, won't it?
Contributor
Author
|
CI fails seems to be real. We need to somehow have a +git version from pyproject.toml Not sure how matplotlib was showing up before, but it's no longer showing up with uv. |
Collaborator
|
@ezyang Use https://setuptools.pypa.io/en/latest/userguide/pyproject_config.html#dynamic-metadata to dynamically fetch the requirements.txt dependencies and the version metadata. 👍 |
Skylion007
reviewed
May 7, 2025
Skylion007
reviewed
May 7, 2025
| readme = "README.md" | ||
| requires-python = ">=3.9" | ||
| license = {text = "BSD"} | ||
| dynamic = ["version", "dependencies"] |
Collaborator
There was a problem hiding this comment.
Suggested change
| dynamic = ["version", "dependencies"] | |
| dynamic = [ | |
| "version", | |
| "dependencies", | |
| "optional-dependencies", | |
| "readme", | |
| "description", | |
| ] |
albanD
reviewed
May 7, 2025
| description = "Tensors and Dynamic neural networks in Python with strong GPU acceleration" | ||
| readme = "README.md" | ||
| requires-python = ">=3.9" | ||
| license = {text = "BSD"} |
Collaborator
There was a problem hiding this comment.
Do we use pure BSD? Or BSD-like? Does it matter? haha
Collaborator
|
#153055 Unblocks uv sync support at least (tested it and works great), and shows us what we need to do migrate to more declarative pyproject.toml metadata |
Skylion007
requested changes
May 7, 2025
Collaborator
Skylion007
left a comment
There was a problem hiding this comment.
Better populated the fields using #153055
pytorchmergebot
pushed a commit
that referenced
this pull request
May 8, 2025
Follow up to @ezyang's PR #153020 , but better uses PEP621 to reduce redundant fields and pass through metadata better to uv, setuptools, poetry and other tooling. * Enables modern tooling like uv sync and better support for tools like poetry. * Also allows us to set project wide settings that are respected by linters and IDE (in this example we are able centralize the minimum supported python version). * Currently most of the values are dynamically fetched from setuptools, eventually we can migrate all the statically defined values to pyproject.toml and they will be autopopulated in the setuptool arguments. * This controls what additional metadata shows up on PyPi . Special URL Names are listed here for rendering on pypi: https://packaging.python.org/en/latest/specifications/well-known-project-urls/#well-known-labels These also clearly shows us what fields will need to be migrated to pyproject.toml over time from setup.py per #152276. Static fields be fairly easy to migrate, the dynamically built ones like requirements are a bit more challenging. Without this, `uv sync` complains: ``` error: No `project` table found in: `pytorch/pyproject.toml` ``` Pull Request resolved: #153055 Approved by: https://github.com/ezyang
This was referenced May 10, 2025
pytorchmergebot
pushed a commit
that referenced
this pull request
May 10, 2025
Unblocks #153020 which accidentally improves the CircularImportLinter to check all Python files. It doesn't set a logname so it errors, there is another FSDP script which already defaults LOGNAME to '' if not specified, this does the same. Pull Request resolved: #153324 Approved by: https://github.com/awgu
pytorchmergebot
pushed a commit
that referenced
this pull request
May 11, 2025
Fixes unnecessary import for torch script. Unblocks #153020 as it appears to fix circular importer linter into importing every Python file under torch Pull Request resolved: #153323 Approved by: https://github.com/ngimel, https://github.com/cyyever
pytorchmergebot
pushed a commit
that referenced
this pull request
May 12, 2025
Follow up to @ezyang's PR #153020 , but better uses PEP621 to reduce redundant fields and pass through metadata better to uv, setuptools, poetry and other tooling. * Enables modern tooling like uv sync and better support for tools like poetry. * Also allows us to set project wide settings that are respected by linters and IDE (in this example we are able centralize the minimum supported python version). * Currently most of the values are dynamically fetched from setuptools, eventually we can migrate all the statically defined values to pyproject.toml and they will be autopopulated in the setuptool arguments. * This controls what additional metadata shows up on PyPi . Special URL Names are listed here for rendering on pypi: https://packaging.python.org/en/latest/specifications/well-known-project-urls/#well-known-labels These also clearly shows us what fields will need to be migrated to pyproject.toml over time from setup.py per #152276. Static fields be fairly easy to migrate, the dynamically built ones like requirements are a bit more challenging. Without this, `uv sync` complains: ``` error: No `project` table found in: `pytorch/pyproject.toml` ``` Pull Request resolved: #153055 Approved by: https://github.com/ezyang
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
4 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack from ghstack (oldest at bottom):
With this change, I can now run
uv sync -vand get all dependencies I need and then trigger build of PyTorch. (The-vis good because the build takes a long time and uv hides progress by default.)Signed-off-by: Edward Z. Yang ezyang@mit.edu