Skip to content

Specifying repo mapping in bzlmod module extension repos. #17493

@matts1

Description

@matts1

Description of the feature request:

One of the principles of module extensions is that each module has its own "repository namespace". This tends to result in the hub-and-spoke model, where we generate a hub for each module. However, this falls apart when you have multiple modules that define the same repo using the same module extension (which is relatively common). Consider:

workspace/Cargo.toml
same_dep = { version = "1.2.3" }
different_version = { version = "1.2.3"}
different_config = { version = "1.2.3", features = ["blah"]}
workspace/MODULE.bazel
bazel_dep(name = "my-dep")
...
rust_crates.crates_repository(repo_name = "crates", manifests=[":Cargo.toml"], lockfile=":Cargo.lock"))```
use_repo(rust_crates, "crates")
my_dep/Cargo.toml
same_dep = { version = "1.2.3" }
different_version = { version = "2.3.4"}
different_config = { version = "1.2.3", features = []}
implicit_dep = { version = "1.2.3", features = ["blah"]}
my_dep/MODULE.bazel
rust_crates.crates_repository(repo_name = "crates", manifests=[":Cargo.toml"], lockfile=":Cargo.lock"))
use_repo(rust_crates, "crates")

For an example such as above, the ideal semantics would be to generate repositories like the following:

  • same_dep_123_sources, different_version_123_sources, different_version_234_config, different_config_123_sources, implicit_dep_123_sources (private repos, cannot be use_repo'd)
  • same_dep_123_config1, different_version_123_config1, different_version_234_config1, different_config_123_config1, different_config_123_config2, implicit_dep_123_config1 (private repos, cannot be use_repo'd)
  • workspace_crates (visible as crates to workspace), invisible to my_dep
  • my_dep_crates (visible as crates to my_dep), invisible to workspace

In terms of implementation details, my idea of the top of my head was something along the lines of:

for mod in module_ctx.modules:
  repository_rule(name = "hub_repo_%s" % mod, repo_mapping = {mod: "crates"}, ...)
for spoke in spokes:
  other_repository_rule(name = spoke.name, repo_mapping = {}) # make repo invisible.

What underlying problem are you trying to solve with this feature?

In the example above, there are a few different things where we could go wrong:

  • Implicit dependencies
    • If I remove same-dep from cargo.toml in the workspace, my deps = ["@crates//:same_dep"] will still compile
    • Conversely, I can add deps = ["@crates//:implicit_dep"], and if they remove implicit_dep in a future update to their module, my code will fail to compile.
  • Unintended access to implementation details
    • I can write use_repo(rust_crates, "crates_same_dep_sources") to access something that might be changed in future versions. This makes public vs private API unclear.
  • Impossible semantics
    • In the different config & different version examples, there is no way that I can provide @crates//:different_config, as they need to refer to different things, but can't.
  • If my_dep adds the use_repo line, when I update my module may break.

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

release 6.0.0-pre.20221012.2

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse master; git rev-parse HEAD ?

N/A

Have you found anything relevant by searching the web?

No

Any other information, logs, or outputs that you want to share?

No

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2We'll consider working on this in future. (Assignee optional)area-BzlmodBzlmod-specific PRs, issues, and feature requeststeam-ExternalDepsExternal dependency handling, remote repositiories, WORKSPACE file.type: feature request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions