Skip to content

Allow remote-caching for Symlink and SolibSymlink actions #11119

@sohaibiftikhar

Description

@sohaibiftikhar

Description of the problem / feature request:

Bazel does not cache Symlink and SolibSymlink actions. Since the remote-cache API allows caching of symlinks this should be possible to implement.

Feature requests: what underlying problem are you trying to solve with this feature?

When used on a clean output base but with the entire build/test cached remotely this leads to unnecessary downloads or action outputs even when the actual build/test action is already cached. Coupled with the --remote_download_minimal this leads to repeated downloads for a fully cached build without even shutting down the bezel server. For larger binaries/libraries this leads to significant time wastage.

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

  1. Create a file main.cpp
#include <stdio.h>
#include <thread>
#include <chrono>

template<int I> void f() { printf("%d\n", I); }

template<int N, int I>
struct C {
  void operator()() {
    f<I>();
    C<N/2, I+N>()();
    C<N/2, I-N>()();
  }
};

template<int I>
struct C<0, I> {
  void operator()() {}
};

int main() {
  // This is just to create a sufficiently sized binary
  C<65535, 0>()();
  printf("Sleeping\n");
  std::this_thread::sleep_for(std::chrono::seconds(5));
  printf("done\n");
}

And create a build file

load("@rules_cc//cc:defs.bzl", "cc_binary")

cc_binary(
    name = "big",
    srcs = ["main.cpp"]
)

sh_test(
    name = "big_test",
    size = "small",
    srcs = ["big"],
)

Now run the command:

bazel test --remote_cache=grpc://mycache:8980" --remote_download_minimal  //:big_test

Everything should be sent to the remote cache now. Now clean your output base to simulate another system running the same build. Then re run the build.

bazel clean && bazel shutdown
bazel test --remote_cache="--remote_cache=grpc//mycache:8980" --remote_download_minimal  //:big_test

Bazel will download the remote binaries again on repeated invocations (remote_download_minimal ensures that we don't even need a clean anymore).

What operating system are you running Bazel on?

Linux

What's the output of bazel info release?

3.0.0

If bazel info release returns "development version" or "(@Non-Git)", tell us how you built Bazel.

NA

What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?

NA

Have you found anything relevant by searching the web?

There is #6862 but it doesn't cover this particular usage pattern.

Any other information, logs, or outputs that you want to share?

The problem can be circumvented by using --remote_download_outputs=all or even toplevel in some scenarios depending on the dependency graph but this is suboptimal.

Metadata

Metadata

Assignees

Labels

P2We'll consider working on this in future. (Assignee optional)team-Remote-ExecIssues and PRs for the Execution (Remote) teamtype: bug

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions