Skip to content

Improper handling of subprocess stream causes Gradle to hang #6114

@blindpirate

Description

@blindpirate

Context

Recently we've investigated the issue of build execution hanging. The stack trace is:

 "gradle-shutdown-hook" #18 prio=5 os_prio=0 tid=0x0000000055320800 nid=0x1b24 waiting on condition [0x000000005521e000]
 java.lang.Thread.State: WAITING (parking)
 	at sun.misc.Unsafe.park(Native Method)
 	- parking to wait for  <0x00000000feef70a0> (a java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject)
 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
 	at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039)
 	at org.gradle.process.internal.DefaultExecHandle.waitForFinish(DefaultExecHandle.java:300)
 	at org.gradle.process.internal.DefaultExecHandle.abort(DefaultExecHandle.java:289)
 	at org.gradle.process.internal.ExecHandleShutdownHookAction.run(ExecHandleShutdownHookAction.java:39)
 	at org.gradle.process.internal.shutdown.ShutdownHookActionRegister$GradleShutdownHook.run(ShutdownHookActionRegister.java:41)
 	at java.lang.Thread.run(Thread.java:748)
 
 "pool-38-thread-3" #337 prio=5 os_prio=0 tid=0x0000000054f7f800 nid=0x1c14 runnable [0x000000006116e000]
 java.lang.Thread.State: RUNNABLE
 	at java.io.FileInputStream.readBytes(Native Method)
 	at java.io.FileInputStream.read(FileInputStream.java:233)
 	at org.gradle.process.internal.streams.ExecOutputHandleRunner.forwardContent(ExecOutputHandleRunner.java:61)
 	at org.gradle.process.internal.streams.ExecOutputHandleRunner.run(ExecOutputHandleRunner.java:51)
 	at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:38)
 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 	at java.lang.Thread.run(Thread.java:748)
 
 "pool-38-thread-2" #336 prio=5 os_prio=0 tid=0x0000000054f79800 nid=0x1ecc runnable [0x0000000060abf000]
 java.lang.Thread.State: RUNNABLE
 	at java.io.FileInputStream.readBytes(Native Method)
 	at java.io.FileInputStream.read(FileInputStream.java:255)
 	at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
 	at java.io.BufferedInputStream.read1(BufferedInputStream.java:286)
 	at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
 	- locked <0x00000000feef28c0> (a java.io.BufferedInputStream)
 	at java.io.FilterInputStream.read(FilterInputStream.java:107)
 	at org.gradle.process.internal.streams.ExecOutputHandleRunner.forwardContent(ExecOutputHandleRunner.java:61)
 	at org.gradle.process.internal.streams.ExecOutputHandleRunner.run(ExecOutputHandleRunner.java:51)
 	at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:38)
 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 	at java.lang.Thread.run(Thread.java:748)
 
 "pool-38-thread-1" #334 prio=5 os_prio=0 tid=0x0000000055316800 nid=0x2c24 waiting on condition [0x000000006093e000]
 java.lang.Thread.State: WAITING (parking)
 	at sun.misc.Unsafe.park(Native Method)
 	- parking to wait for  <0x00000000feef1db8> (a java.util.concurrent.CountDownLatch$Sync)
 	at java.util.concurrent.locks.LockSupport.park(LockSupport.java:175)
 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:836)
 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:997)
 	at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1304)
 	at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:231)
 	at org.gradle.process.internal.streams.OutputStreamsForwarder.stop(OutputStreamsForwarder.java:68)
 	at org.gradle.process.internal.DefaultExecHandle$CompositeStreamsHandler.stop(DefaultExecHandle.java:429)
 	at org.gradle.process.internal.ExecHandleRunner.run(ExecHandleRunner.java:81)
 	at org.gradle.internal.operations.CurrentBuildOperationPreservingRunnable.run(CurrentBuildOperationPreservingRunnable.java:38)
 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 	at java.lang.Thread.run(Thread.java:748)
 
 
 "main" #1 prio=5 os_prio=0 tid=0x0000000000a3f000 nid=0xe34 in Object.wait() [0x0000000000c1e000]
 java.lang.Thread.State: WAITING (on object monitor)
 	at java.lang.Object.wait(Native Method)
 	- waiting on <0x00000000e07b4ee8> (a java.lang.Thread)
 	at java.lang.Thread.join(Thread.java:1252)
 	- locked <0x00000000e07b4ee8> (a java.lang.Thread)
 	at java.lang.Thread.join(Thread.java:1326)
 	at java.lang.ApplicationShutdownHooks.runHooks(ApplicationShutdownHooks.java:106)
 	at java.lang.ApplicationShutdownHooks$1.run(ApplicationShutdownHooks.java:46)
 	at java.lang.Shutdown.runHooks(Shutdown.java:123)
 	at java.lang.Shutdown.sequence(Shutdown.java:167)
 	at java.lang.Shutdown.exit(Shutdown.java:212)
 	- locked <0x00000000e0745ff0> (a java.lang.Class for java.lang.Shutdown)
 	at java.lang.Runtime.exit(Runtime.java:109)
 	at java.lang.System.exit(System.java:971)
 	at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:68)

Obviously, the process hangs at this line.

int exitValue = process.waitFor();
streamsHandler.stop(); <--- stuck at this line

which means, although subprocess already returns (waitFor() returns), its stdout and stderr streams are still in block state. This is rare but not impossible. Searched StackOverflow we found two similar issues:

On this scenario, subprocess' stdout and stderr stream should not be read any more because the subprocess no longer exists.

Steps to Reproduce (for bugs)

To reproduce this bug, you have to have JDK6 installed and be on Windows.

import org.gradle.process.internal.ExecHandleFactory
import org.gradle.internal.classloader.ClasspathUtil

class ParentProcess {
   static String JAVA6 = "C:\\Program Files\\Java\\jdk1.6.0_45\\bin\\java.exe"
   public static void main(String[] args) throws Exception {
       System.out.println("Before call child process");
       ProcessBuilder builder = new ProcessBuilder(JAVA6, '-cp', System.getProperty('java.class.path'), ChildProcess.class.name);
       builder.start();
       System.out.println("After call child process");
   }
}

public class ChildProcess {
   public static void main(String[] args) throws Exception {
       Thread.sleep(3600*1000);
   }
}

task hang {
   doLast {
       String classpath = ClasspathUtil.getClasspath(ParentProcess.class.classLoader)
               .asFiles
               .collect { it.absolutePath}
               .join(';')

       def execHandler = project.services.get(ExecHandleFactory).newExec()
               .executable(ParentProcess.JAVA6)
               .args('-cp', classpath, ParentProcess.class.name)
               .build()

       execHandler.start()

       Thread.sleep(5000)
       execHandler.abort()
   }
}

Your Environment

Currently this issue is only observed on Windows.

Metadata

Metadata

Assignees

Labels

a:bugThis doesn't work as expected

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions