Skip to content

FR: Share extracted external libraries across workspaces #12227

@dan-cohn

Description

@dan-cohn

ATTENTION! Please read and follow:

  • if this is a question about how to build / test / query / deploy using Bazel, or a discussion starter, send it to bazel-discuss@googlegroups.com
  • if this is a bug or feature request, fill the form below as best as you can.

Description of the problem / feature request:

I would like Bazel to be able to share extracted repos across workspaces. In other words, the "repository cache" should go deeper than simply making downloaded archives available in a centralized location. It would help save time and space if the unpacked contents were also shared.

Feature requests: what underlying problem are you trying to solve with this feature?

Bazel's repository cache is shared across workspaces, which avoids downloading the same external libraries multiple times. This saves a lot of time and network utilization, especially in CI environments where builds occur across many different repos and versions. The problem is that only the original downloaded artifacts are shared and not the extracted files. This means that Bazel has to repeatedly unzip/unpack archives. This takes a lot of time, consumes CPU, and fills the filesystem with multiple copies of the exact same extracted libraries.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

N/A

What operating system are you running Bazel on?

RHEL 7/8 and CentOS 7/8

What's the output of bazel info release?

release 3.4.1

If bazel info release returns "development version" or "(@Non-Git)", tell us how you built Bazel.

N/A

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

N/A

Have you found anything relevant by searching the web?

No.

Any other information, logs, or outputs that you want to share?

N/A

Metadata

Metadata

Assignees

Labels

P1I'll work on this now. (Assignee required)area-BzlmodBzlmod-specific PRs, issues, and feature requeststeam-ExternalDepsExternal dependency handling, remote repositiories, WORKSPACE file.type: feature request

Type

No type

Projects

Status

Done

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions