Skip to content

Crash while DirectoryTreeDigestFunction.getSubDirTreeDigests #27942

@sluongng

Description

@sluongng

Description of the bug:

I was running this Bazel test a few times and got multiple crashes in a row

## 1st attempt

!sluongng/fix-list_source_repo ~/work/bazelbuild/bazel> bazel test --config=bb-remote src/test/shell/bazel:srcs_test
INFO: Invocation ID: dfc2fbd5-31cd-4ae9-ac87-d6c6c623923c
INFO: Analyzed target //src/test/shell/bazel:srcs_test (0 packages loaded, 0 targets configured).
INFO: Found 1 test target...
Target //src/test/shell/bazel:srcs_test up-to-date:
  bazel-bin/src/test/shell/bazel/srcs_test
INFO: Elapsed time: 2.910s, Critical Path: 0.00s
INFO: 1 process: 2 action cache hit, 1 internal.
INFO: Build completed successfully, 1 total action
//src/test/shell/bazel:srcs_test                                (cached) PASSED in 0.0s

Executed 0 out of 1 test: 1 test passes.


## 2nd attempt

!sluongng/fix-list_source_repo ~/work/bazelbuild/bazel> bazel test --config=bb-remote src/test/shell/bazel:srcs_test
INFO: Invocation ID: 040b0905-78ef-4f56-94d2-7ea31387b9df
Analyzing: target //src/test/shell/bazel:srcs_test (0 packages loaded, 0 targets configured)

FATAL: bazel crashed due to an internal error. Printing stack trace:
java.lang.NullPointerException
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(Unknown Source)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.copyInto(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(Unknown Source)
	at java.base/java.util.stream.ReduceOps$ReduceOp.evaluateSequential(Unknown Source)
	at java.base/java.util.stream.AbstractPipeline.evaluate(Unknown Source)
	at java.base/java.util.stream.ReferencePipeline.collect(Unknown Source)
	at com.google.devtools.build.lib.skyframe.DirectoryTreeDigestFunction.getSubDirTreeDigests(DirectoryTreeDigestFunction.java:140)
	at com.google.devtools.build.lib.skyframe.DirectoryTreeDigestFunction.compute(DirectoryTreeDigestFunction.java:65)
	at com.google.devtools.build.skyframe.ParallelEvaluator.bubbleErrorUp(ParallelEvaluator.java:410)
	at com.google.devtools.build.skyframe.ParallelEvaluator.waitForCompletionAndConstructResult(ParallelEvaluator.java:203)
	at com.google.devtools.build.skyframe.ParallelEvaluator.doMutatingEvaluation(ParallelEvaluator.java:169)
	at com.google.devtools.build.skyframe.ParallelEvaluator.eval(ParallelEvaluator.java:664)
	at com.google.devtools.build.skyframe.AbstractInMemoryMemoizingEvaluator.evaluate(AbstractInMemoryMemoizingEvaluator.java:179)
	at com.google.devtools.build.lib.skyframe.SkyframeExecutor.evaluateBuildDriverKeys(SkyframeExecutor.java:2268)
	at com.google.devtools.build.lib.skyframe.SkyframeBuildView.analyzeAndExecuteTargets(SkyframeBuildView.java:735)
	at com.google.devtools.build.lib.analysis.BuildView.update(BuildView.java:356)
	at com.google.devtools.build.lib.buildtool.AnalysisAndExecutionPhaseRunner.runAnalysisAndExecutionPhase(AnalysisAndExecutionPhaseRunner.java:206)
	at com.google.devtools.build.lib.buildtool.AnalysisAndExecutionPhaseRunner.execute(AnalysisAndExecutionPhaseRunner.java:124)
	at com.google.devtools.build.lib.buildtool.BuildTool.buildTargetsWithMergedAnalysisExecution(BuildTool.java:450)
	at com.google.devtools.build.lib.buildtool.BuildTool.buildTargets(BuildTool.java:247)
	at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:722)
	at com.google.devtools.build.lib.buildtool.BuildTool.processRequest(BuildTool.java:690)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.doTest(TestCommand.java:166)
	at com.google.devtools.build.lib.runtime.commands.TestCommand.exec(TestCommand.java:119)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.execExclusively(BlazeCommandDispatcher.java:687)
	at com.google.devtools.build.lib.runtime.BlazeCommandDispatcher.exec(BlazeCommandDispatcher.java:257)
	at com.google.devtools.build.lib.server.GrpcServerImpl.executeCommand(GrpcServerImpl.java:607)
	at com.google.devtools.build.lib.server.GrpcServerImpl.lambda$run$0(GrpcServerImpl.java:677)
	at io.grpc.Context$1.run(Context.java:566)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
exit 37

Which category does this issue belong to?

No response

What's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

I have this small patch

--- a/src/test/shell/bazel/list_source_repository.bzl
+++ b/src/test/shell/bazel/list_source_repository.bzl
@@ -38,10 +38,18 @@ genrule(
     "grep -Ev '%s'",
     "sort -u > $@",
   ]),
-)
+    )
 """ % srcs_excludes)

-    rctx.watch_tree(workspace)
+    # Watch everything under the workspace except the VCS metadata. Disable
+    # directory-entry watching during the read to avoid redundant triggers.
+    for entry in workspace.readdir(watch = "no"):
+        if entry.basename == ".git":
+            continue
+        if entry.is_dir:
+            rctx.watch_tree(entry)
+        else:
+            rctx.watch(entry)

applied on top of this commit 9274aaa

Then I tried running the test a few times.

Which operating system are you running Bazel on?

Linux

What is the output of bazel info release?

release 8.4.2

If bazel info release returns development version or (@non-git), tell us how you built Bazel.

No response

What's the output of git remote get-url origin; git rev-parse HEAD ?


If this is a regression, please try to identify the Bazel commit where the bug was introduced with bazelisk --bisect.

No response

Have you found anything relevant by searching the web?

No response

Any other information, logs, or outputs that you want to share?

No response

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions