-
Notifications
You must be signed in to change notification settings - Fork 4.4k
Specifying repo mapping in bzlmod module extension repos. #17493
Copy link
Copy link
Closed
Labels
P2We'll consider working on this in future. (Assignee optional)We'll consider working on this in future. (Assignee optional)area-BzlmodBzlmod-specific PRs, issues, and feature requestsBzlmod-specific PRs, issues, and feature requeststeam-ExternalDepsExternal dependency handling, remote repositiories, WORKSPACE file.External dependency handling, remote repositiories, WORKSPACE file.type: feature request
Description
Description of the feature request:
One of the principles of module extensions is that each module has its own "repository namespace". This tends to result in the hub-and-spoke model, where we generate a hub for each module. However, this falls apart when you have multiple modules that define the same repo using the same module extension (which is relatively common). Consider:
workspace/Cargo.toml
same_dep = { version = "1.2.3" }
different_version = { version = "1.2.3"}
different_config = { version = "1.2.3", features = ["blah"]}
workspace/MODULE.bazel
bazel_dep(name = "my-dep")
...
rust_crates.crates_repository(repo_name = "crates", manifests=[":Cargo.toml"], lockfile=":Cargo.lock"))```
use_repo(rust_crates, "crates")
my_dep/Cargo.toml
same_dep = { version = "1.2.3" }
different_version = { version = "2.3.4"}
different_config = { version = "1.2.3", features = []}
implicit_dep = { version = "1.2.3", features = ["blah"]}
my_dep/MODULE.bazel
rust_crates.crates_repository(repo_name = "crates", manifests=[":Cargo.toml"], lockfile=":Cargo.lock"))
use_repo(rust_crates, "crates")
For an example such as above, the ideal semantics would be to generate repositories like the following:
same_dep_123_sources,different_version_123_sources,different_version_234_config,different_config_123_sources,implicit_dep_123_sources(private repos, cannot be use_repo'd)same_dep_123_config1,different_version_123_config1,different_version_234_config1,different_config_123_config1,different_config_123_config2,implicit_dep_123_config1(private repos, cannot be use_repo'd)workspace_crates(visible ascratesto workspace), invisible to my_depmy_dep_crates(visible ascratesto my_dep), invisible to workspace
In terms of implementation details, my idea of the top of my head was something along the lines of:
for mod in module_ctx.modules:
repository_rule(name = "hub_repo_%s" % mod, repo_mapping = {mod: "crates"}, ...)
for spoke in spokes:
other_repository_rule(name = spoke.name, repo_mapping = {}) # make repo invisible.
What underlying problem are you trying to solve with this feature?
In the example above, there are a few different things where we could go wrong:
- Implicit dependencies
- If I remove same-dep from cargo.toml in the workspace, my
deps = ["@crates//:same_dep"]will still compile - Conversely, I can add
deps = ["@crates//:implicit_dep"], and if they remove implicit_dep in a future update to their module, my code will fail to compile.
- If I remove same-dep from cargo.toml in the workspace, my
- Unintended access to implementation details
- I can write
use_repo(rust_crates, "crates_same_dep_sources")to access something that might be changed in future versions. This makes public vs private API unclear.
- I can write
- Impossible semantics
- In the different config & different version examples, there is no way that I can provide
@crates//:different_config, as they need to refer to different things, but can't.
- In the different config & different version examples, there is no way that I can provide
- If my_dep adds the use_repo line, when I update my module may break.
Which operating system are you running Bazel on?
Linux
What is the output of bazel info release?
release 6.0.0-pre.20221012.2
If bazel info release returns development version or (@non-git), tell us how you built Bazel.
No response
What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?
N/A
Have you found anything relevant by searching the web?
No
Any other information, logs, or outputs that you want to share?
No
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
P2We'll consider working on this in future. (Assignee optional)We'll consider working on this in future. (Assignee optional)area-BzlmodBzlmod-specific PRs, issues, and feature requestsBzlmod-specific PRs, issues, and feature requeststeam-ExternalDepsExternal dependency handling, remote repositiories, WORKSPACE file.External dependency handling, remote repositiories, WORKSPACE file.type: feature request