BwtB deletes needed runfiles from output base #12635
Description
Description of the problem:
With --remote_download_toplevel, runfiles are not downloaded even if they are missing from the output base under certain circumstances.
If a target that is used as data in a bazel test is cached, it will be downloaded for the test, but then deleted right afterward.
Then, if you bazel run something that also uses that cached target as data, the bazel server will think the file already exists in the output base, and won't download it. This leads to broken symlinks in the binary's runfiles tree.
Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
Steps to reproduce:
https://gist.github.com/JaredNeil/aaba7e65e22c0a064251f2c2f4e6c6b8
Download zip and run bash -x repro.sh.
What operating system are you running Bazel on?
Ubuntu 18.04
What's the output of bazel info release?
release 3.7.1
Any other information, logs, or outputs that you want to share?
Full output of running repro.sh from the gist linked above:
$ bash -x repro.sh
+ chmod +x test.sh publish.sh
+ bazelisk build //:data
Starting local Bazel server and connecting to it...
INFO: Invocation ID: dccc83b8-10b1-4651-b4f6-3ad19c2d1902
INFO: Analyzed target //:data (5 packages loaded, 7 targets configured).
INFO: Found 1 target...
Target //:data up-to-date:
bazel-bin/data.txt
INFO: Elapsed time: 1.617s, Critical Path: 0.08s
INFO: 2 processes: 1 internal, 1 linux-sandbox.
INFO: Build completed successfully, 2 total actions
+ bazelisk clean
INFO: Invocation ID: 185a7ec5-29ed-45fb-a5c3-3bd9d7fe7cad
INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.
+ bazelisk test //:test
INFO: Invocation ID: 3c3b3346-d8d3-46e2-988c-ab64e12c1e5e
INFO: Analyzed target //:test (22 packages loaded, 229 targets configured).
INFO: Found 1 test target...
Target //:test up-to-date:
bazel-bin/test
INFO: Elapsed time: 4.144s, Critical Path: 0.11s
INFO: 6 processes: 1 remote cache hit, 3 internal, 2 linux-sandbox.
INFO: Build completed successfully, 6 total actions
//:test PASSED in 0.0s
Executed 1 out of 1 test: 1 test passes.
INFO: Build completed successfully, 6 total actions
+ bazelisk run //:publish
INFO: Invocation ID: d87693e1-3cca-4087-beca-63a9290c93db
INFO: Analyzed target //:publish (0 packages loaded, 2 targets configured).
INFO: Found 1 target...
Target //:publish up-to-date:
bazel-bin/publish
INFO: Elapsed time: 0.160s, Critical Path: 0.01s
INFO: 4 processes: 4 internal.
INFO: Build completed successfully, 4 total actions
INFO: Build completed successfully, 4 total actions
lrwxrwxrwx 1 jaredneil jaredneil 132 Dec 4 21:43 data.txt -> /home/jaredneil/.cache/bazel/_bazel_jaredneil/e78557b1a5d5d19319bf331b359d3c1d/execroot/__main__/bazel-out/k8-fastbuild/bin/data.txt
Symlink is broken because the target file doesn't exist in the output base
+ echo 'Pubish exit code: 1'
Pubish exit code: 1