WIP: a builder API for pip extension 1/n - platform defs#2909
Closed
aignas wants to merge 23 commits intobazel-contrib:mainfrom
Closed
WIP: a builder API for pip extension 1/n - platform defs#2909aignas wants to merge 23 commits intobazel-contrib:mainfrom
aignas wants to merge 23 commits intobazel-contrib:mainfrom
Conversation
rickeylev
reviewed
May 19, 2025
python/private/pypi/extension.bzl
Outdated
|
|
||
| EXPERIMENTAL: this may be removed without notice. | ||
|
|
||
| :::{versionadded} 1.4.0 |
Collaborator
There was a problem hiding this comment.
Suggested change
| :::{versionadded} 1.4.0 | |
| :::{versionadded} VERSION_NEXT_FEATURE |
rickeylev
reviewed
May 19, 2025
Collaborator
rickeylev
left a comment
There was a problem hiding this comment.
I think I'm following the idea here. Basically, right now you can do:
pip.parse(requirements_by_platform={
"linux*": "requirements_linux.txt",
}, ...)
And it uses the PLATFORMS values for what is defined as "linux".
The idea here is to move those definitions into the bzlmod API calls:
pip.default(
platform="my-platform",
...
)
pip.parse(requirements_by_platform={
"my-platform": "requirements_linux.txt",
}, ...)
And now instead of using the PLATFORMS global, it'll use that pip.default() information.
Ok, mostly LGTM so far. Under the hood, I can imagine:
pip.default(platform="linux_glibc", ...)
pip.default(platform="linux_musl", ...)
pip.parse(requirements_by_platform={
"linux_glibc": "requirements_a.txt",
"linux_musl": "requirements_b.txt",
}, ...)
# -> generates a BUILD file like
alias(
name = "foo"
actual = select({
":is_linux_glibc": "@requirements_a//foo:foo",
":is_linux_musl": "@requirements_b//foo:foo",
})
)
Am I imagining the same as you?
Questions:
- What can this express that env markers can't? i.e. why would one do this instead of creating a requirements.txt with
foo; os_name="my-platform-os"(I might have answered this with my example above -- I don't think "musl" can be expressed by env markers?) - This seems to create a duplicate mapping of platform names and their metadata (presumably, the python.toolchain() APIs won't be entirely able to not use the platform keys). Is this the part you were mentioning in the recent issue?
- Does this need to feed into the simple-index-url walking? The part I'm thinking of is how it maps wheel names to the config the wheel supports.
- In the example above, all we actually need is a config_setting label. The user could pass that in directly.
Collaborator
Author
|
OK to correctly model the whl selection to allow users to limit the number of selected wheels to minimum requires me to rewrite the I think the current logic is very complex and this could be a chance to simplify it. I think I may back out the code for that from this PR and leave it for the future. |
Collaborator
Author
|
I have something that builds the docs (i.e. can download sphinx) but I'll need more time to tidy this up fully. |
aignas
added a commit
to aignas/rules_python
that referenced
this pull request
Jun 14, 2025
Summary: - Allow switching to the Starlark implementation of the marker evaluation function. - Add a way for users to modify the `env` for the marker evaluation when parsing the requirements. This can only be done by `rules_python` or the root module. - Limit the platform selection when parsing the requirements files. Work towards bazel-contrib#2747 Work towards bazel-contrib#2949 Split out from bazel-contrib#2909
aignas
added a commit
to aignas/rules_python
that referenced
this pull request
Jun 14, 2025
Summary: - Allow switching to the Starlark implementation of the marker evaluation function. - Add a way for users to modify the `env` for the marker evaluation when parsing the requirements. This can only be done by `rules_python` or the root module. - Limit the platform selection when parsing the requirements files. Work towards bazel-contrib#2747 Work towards bazel-contrib#2949 Split out from bazel-contrib#2909
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jun 19, 2025
Parse env markers in pip.parse using starlark Summary: - Allow switching to the Starlark implementation of the marker evaluation function. - Add a way for users to modify the `env` for the marker evaluation when parsing the requirements. This can only be done by `rules_python` or the root module. - Limit the platform selection when parsing the requirements files. Work towards #2747 Work towards #2949 Split out from #2909 --------- Co-authored-by: Richard Levasseur <richardlev@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is just some stuff that I wanted to write down before I forget. Feel free
to continue/hack on this.
The goal is to have:
platform_versionconfiguration for the users.requirements.txtevaluation. In the future the same API should be good enough for
uv.lockand
py.lockfiles.rules_pythonwill onlycare about the top 2 tiers of the supported platforms, but users should be
able to customize things.
Work towards #2821
Work towards #2747
Work towards #2548
Work towards #1708