remote_download_minimal: Error running tests with a directory dependency #9386
Description
Description of the problem / feature request:
Hello there! It looks like latest bazel fails when running tests:
- with remote execution
- with the flags
--nobuild_runfile_linksor--remote_download_minimal - when the test specifies a directory in
srcsordata. See the minimal example below.
I reckon this is most likely related to the changes in #9244 plus #9315? +cc @buchgr
That said, the changes in #9244 are very very compelling from a performance perspective!
The stacktrace looks like this.
ERROR: /home/efreeman/bazel_regression/BUILD:1:1: failed due to unexpected I/O exception: /home/efreeman/.cache/bazel/_bazel_efreeman/74164ebd52c68743881f0bc0b22c6057/execroot/__main__/bazel-out/k8-fastbuild/bin/nop.runfiles/__main__/test/data (No such file or directory)
java.io.FileNotFoundException: /home/efreeman/.cache/bazel/_bazel_efreeman/74164ebd52c68743881f0bc0b22c6057/execroot/__main__/bazel-out/k8-fastbuild/bin/nop.runfiles/__main__/test/data (No such file or directory)
at com.google.devtools.build.lib.unix.NativePosixFiles.readdir(Native Method)
at com.google.devtools.build.lib.unix.NativePosixFiles.readdir(NativePosixFiles.java:284)
at com.google.devtools.build.lib.unix.UnixFileSystem.readdir(UnixFileSystem.java:156)
at com.google.devtools.build.lib.vfs.Path.readdir(Path.java:395)
at com.google.devtools.build.lib.remote.merkletree.InputTree.explodeDirectory(InputTree.java:339)
at com.google.devtools.build.lib.remote.merkletree.InputTree.explodeDirectory(InputTree.java:332)
at com.google.devtools.build.lib.remote.merkletree.InputTree.build(InputTree.java:304)
at com.google.devtools.build.lib.remote.merkletree.InputTree.build(InputTree.java:255)
at com.google.devtools.build.lib.remote.merkletree.MerkleTree.build(MerkleTree.java:94)
at com.google.devtools.build.lib.remote.RemoteSpawnRunner.exec(RemoteSpawnRunner.java:166)
at com.google.devtools.build.lib.exec.SpawnRunner.execAsync(SpawnRunner.java:225)
at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:122)
at com.google.devtools.build.lib.exec.AbstractSpawnStrategy.exec(AbstractSpawnStrategy.java:87)
at com.google.devtools.build.lib.actions.SpawnActionContext.beginExecution(SpawnActionContext.java:41)
at com.google.devtools.build.lib.exec.ProxySpawnActionContext.beginExecution(ProxySpawnActionContext.java:60)
at com.google.devtools.build.lib.exec.StandaloneTestStrategy.beginTestAttempt(StandaloneTestStrategy.java:304)
at com.google.devtools.build.lib.exec.StandaloneTestStrategy.access$100(StandaloneTestStrategy.java:73)
at com.google.devtools.build.lib.exec.StandaloneTestStrategy$StandaloneTestRunnerSpawn.beginExecution(StandaloneTestStrategy.java:447)
at com.google.devtools.build.lib.analysis.test.TestRunnerAction$1.execute(TestRunnerAction.java:765)
at com.google.devtools.build.lib.analysis.test.TestRunnerAction$RunAttemptsContinuation.execute(TestRunnerAction.java:977)
at com.google.devtools.build.lib.analysis.test.TestRunnerAction.beginExecution(TestRunnerAction.java:771)
at com.google.devtools.build.lib.analysis.test.TestRunnerAction.execute(TestRunnerAction.java:790)
at com.google.devtools.build.lib.analysis.test.TestRunnerAction.execute(TestRunnerAction.java:781)
at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$4.execute(SkyframeActionExecutor.java:854)
at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.continueAction(SkyframeActionExecutor.java:988)
at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor$ActionRunner.run(SkyframeActionExecutor.java:960)
at com.google.devtools.build.lib.skyframe.ActionExecutionState.runStateMachine(ActionExecutionState.java:116)
at com.google.devtools.build.lib.skyframe.ActionExecutionState.getResultOrDependOnFuture(ActionExecutionState.java:77)
at com.google.devtools.build.lib.skyframe.SkyframeActionExecutor.executeAction(SkyframeActionExecutor.java:579)
at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.checkCacheAndExecuteIfNeeded(ActionExecutionFunction.java:768)
at com.google.devtools.build.lib.skyframe.ActionExecutionFunction.compute(ActionExecutionFunction.java:275)
at com.google.devtools.build.skyframe.AbstractParallelEvaluator$Evaluate.run(AbstractParallelEvaluator.java:454)
at com.google.devtools.build.lib.concurrent.AbstractQueueVisitor$WrappedRunnable.run(AbstractQueueVisitor.java:399)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at java.base/java.lang.Thread.run(Unknown Source)
Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.
I can reproduce the issue using the following minimal setup:
$ cat BUILD
sh_test(
name = "nop",
srcs = ["nop.sh"], # A trivial shell script that just exits 0
data = ["test/data"], # A directory
)
(Note that if I replace the data dependency with data = glob(["test/data/**/*"]) no error is thrown.)
$ cat WORKSPACE # This just has the boilerplate for remote execution
load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "bazel_toolchains",
urls = [
"https://mirror.bazel.build/github.com/bazelbuild/bazel-toolchains/archive/0.29.3.tar.gz",
"https://github.com/bazelbuild/bazel-toolchains/archive/0.29.3.tar.gz",
],
strip_prefix = "bazel-toolchains-0.29.3",
sha256 = "1411f2648185b0e7d8c2bb88b25cc8f2c477cc4223133461652ddce2b3154ac4",
)
load("@bazel_toolchains//rules:rbe_repo.bzl", "rbe_autoconfig")
rbe_autoconfig(name = "rbe_default")
$ cat .bazelrc
build --remote_cache=grpcs://remotebuildexecution.googleapis.com
build --remote_executor=grpcs://remotebuildexecution.googleapis.com
build --remote_instance_name=projects/[fill in a project here]
build --google_default_credentials=true
build --strategy=TestRunner=remote
build --platforms=@rbe_default//config:platform
build --host_platform=@rbe_default//config:platform
build --nocache_test_results
build --remote_download_minimal
What operating system are you running Bazel on?
Ubuntu 18.04
What's the output of bazel info release?
development version
If bazel info release returns "development version" or "(@Non-Git)", tell us how you built Bazel.
Latest master, built from source.
What's the output of git remote get-url origin ; git rev-parse master ; git rev-parse HEAD ?
https://github.com/bazelbuild/bazel
54ec37db189c839f8d2b5b9626f7dd78b3add0d8
54ec37db189c839f8d2b5b9626f7dd78b3add0d8
Have you found anything relevant by searching the web?
I have not.