feat: render aliases for extra_targets of whl_mods.#1903
Closed
lalten wants to merge 1 commit intobazel-contrib:mainfrom
Closed
feat: render aliases for extra_targets of whl_mods.#1903lalten wants to merge 1 commit intobazel-contrib:mainfrom
extra_targets of whl_mods.#1903lalten wants to merge 1 commit intobazel-contrib:mainfrom
Conversation
Merged
Collaborator
|
Thanks for this PR! At some point I had experimented with different ideas on this topic and this solution was one of them. Other solutions that could achieve the goal in the repository context:
However, a better way could be to solve this in the build phase where we use one of the predefined targets and we can extract the headers, the easiest way that I could figure out was: genrule(
name = "numpy_headers",
srcs = ["@pypi//numpy:whl"],
outs = [
# "numpy/core/include/numpy/__multiarray_api.c",
# "numpy/core/include/numpy/__multiarray_api.h",
# "numpy/core/include/numpy/__ufunc_api.c",
# "numpy/core/include/numpy/__ufunc_api.h",
"numpy/core/include/numpy/_dtype_api.h",
"numpy/core/include/numpy/_neighborhood_iterator_imp.h",
"numpy/core/include/numpy/_numpyconfig.h",
"numpy/core/include/numpy/arrayobject.h",
"numpy/core/include/numpy/arrayscalars.h",
"numpy/core/include/numpy/experimental_dtype_api.h",
"numpy/core/include/numpy/halffloat.h",
"numpy/core/include/numpy/ndarrayobject.h",
"numpy/core/include/numpy/ndarraytypes.h",
"numpy/core/include/numpy/noprefix.h",
"numpy/core/include/numpy/npy_1_7_deprecated_api.h",
"numpy/core/include/numpy/npy_3kcompat.h",
"numpy/core/include/numpy/npy_common.h",
"numpy/core/include/numpy/npy_cpu.h",
"numpy/core/include/numpy/npy_endian.h",
"numpy/core/include/numpy/npy_interrupt.h",
"numpy/core/include/numpy/npy_math.h",
"numpy/core/include/numpy/npy_no_deprecated_api.h",
"numpy/core/include/numpy/npy_os.h",
"numpy/core/include/numpy/numpyconfig.h",
"numpy/core/include/numpy/old_defines.h",
"numpy/core/include/numpy/random/bitgen.h",
"numpy/core/include/numpy/random/distributions.h",
"numpy/core/include/numpy/random/libdivide.h",
"numpy/core/include/numpy/ufuncobject.h",
"numpy/core/include/numpy/utils.h",
],
cmd = "unzip $< -d $(RULEDIR) -x $(OUTS)",
)I wonder if there are better ways to extracting filegroups from the whls. A custom build rule that extract particular files from a whl could be the best solution because that is something that can live in The API that I think we could have could be load("@rules_python//python:pip.bzl", "whl_filegroup")
whl_filegroup(
name = "numpy_includes",
whl = "@pypi//numpy:whl",
prefixes = [
"numpy/core/include/numpy",
],
)Because each |
github-merge-queue bot
pushed a commit
that referenced
this pull request
May 20, 2024
This adds a `whl_filegroup` rule which can take a wheel as input and extract it, making the output available to consumers. A pattern is allowed to better control what files are extracted. This is handy to, e.g. get the numpy headers from the numpy wheel so they can be used in a cc_library. e.g., Closes #1903 Fixes #1311
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
I'm trying to get rules_ros2 to Bzlmod. There is a dependency on numpy headers that are exposed via an additive_build_content cc_library. One of the last migration challenges is how to properly consume these headers in both Bzlmod and Workspace and without hardcoding a Python version. See mvukov/rules_ros2#238 (comment) for the WIP PR.
I dug a little into how the aliasing works and come up with a proof of concept of how a list of extra_targets could be propagated from pip.whl_mods like
This makes it possible to depend on
"@rules_ros2_pip_deps//numpy:headers"instead of e.g."@rules_ros2_pip_deps_310_numpy//:headersWorkspace doesn't have this problem, but the label looks slightly different.
That can be worked around with an alias for now.
You can see this in action in mvukov/rules_ros2@e2b926c
Please let me know what you think of this. Maybe there is a more elegant way to achieve this?