deadlock in repl using sys.process

[scabug]

Cay reported the following program that hangs in M4 (https://groups.google.com/forum/#!topic/scala-user/CMD_oZK_4Ns)

scala> import scala.sys.process._
import scala.sys.process._

scala> println(("ls -al /" #| "grep u").!!)

also hangs with -Ydelambdafy:inline (though with the library itself is compiled with delambdafy-method -- i did not try a scala library compiled with delambdafy-inline)

This also hangs in the class-based repl.

---

(The rest of the description duplicates #8119 parallel collections + object initialization = hang)

Also reported by cay horstmann on https://groups.google.com/forum/#!topic/scala-user/f5YJE5AB5aA

scala> for (i <- (1 to 100).par) yield i

hangs in 2.12.0-M4 and current master, thread dump below. it completes with -Ydelambdafy:inline, or by using -Yrepl-class-based.

thread dump:

scala> for (i <- (1 to 100).par) yield i

"scala-execution-context-global-18" #18 daemon prio=5 os_prio=31 tid=0x00007f90eb4fd800 nid=0x6a03 in Object.wait() [0x000070000571e000]
   java.lang.Thread.State: RUNNABLE
	at $line4.$read$$iw$$iw$$$Lambda$1143/1492822331.apply$mcII$sp(Unknown Source)
	at scala.runtime.java8.JFunction1$mcII$sp.apply(JFunction1$mcII$sp.java:12)
	at scala.collection.parallel.immutable.ParRange$ParRangeIterator.map2combiner(ParRange.scala:104)
	at scala.collection.parallel.ParIterableLike$Map.leaf(ParIterableLike.scala:1052)
	at scala.collection.parallel.Task.scala$collection$parallel$Task$$$anonfun$1(Tasks.scala:49)
	at scala.collection.parallel.Task$$Lambda$1146/190975371.apply$mcV$sp(Unknown Source)
	at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:12)
	at scala.util.control.Breaks$$anon$1.catchBreak(Breaks.scala:63)
	at scala.collection.parallel.Task.tryLeaf(Tasks.scala:52)
	at scala.collection.parallel.ParIterableLike$Map.tryLeaf(ParIterableLike.scala:1049)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.internal(Tasks.scala:156)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.internal(Tasks.scala:440)
	at scala.collection.parallel.AdaptiveWorkStealingTasks$WrappedTask.compute(Tasks.scala:146)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.compute(Tasks.scala:440)
	at java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:189)
	at java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:289)
	at java.util.concurrent.ForkJoinPool$WorkQueue.runTask(ForkJoinPool.java:1056)
	at java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1692)
	at java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:157)

[... more of those ...]

"process reaper" #10 daemon prio=10 os_prio=31 tid=0x00007f90eb4e7800 nid=0x5803 waiting on condition [0x0000700002a03000]
   java.lang.Thread.State: TIMED_WAITING (parking)
	at sun.misc.Unsafe.park(Native Method)
	- parking to wait for  <0x0000000760008f60> (a java.util.concurrent.SynchronousQueue$TransferStack)
	at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
	at java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:460)
	at java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362)
	at java.util.concurrent.SynchronousQueue.poll(SynchronousQueue.java:941)
	at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1066)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:745)

"main" #1 prio=5 os_prio=31 tid=0x00007f90ea00d000 nid=0x1703 in Object.wait() [0x0000700000617000]
   java.lang.Thread.State: WAITING (on object monitor)
	at java.lang.Object.wait(Native Method)
	- waiting on <0x00000007a9ae3970> (a scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask)
	at java.util.concurrent.ForkJoinTask.externalAwaitDone(ForkJoinTask.java:334)
	- locked <0x00000007a9ae3970> (a scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask)
	at java.util.concurrent.ForkJoinTask.doJoin(ForkJoinTask.java:391)
	at java.util.concurrent.ForkJoinTask.join(ForkJoinTask.java:719)
	at scala.collection.parallel.ForkJoinTasks$WrappedTask.sync(Tasks.scala:375)
	at scala.collection.parallel.AdaptiveWorkStealingForkJoinTasks$WrappedTask.sync(Tasks.scala:440)
	at scala.collection.parallel.ForkJoinTasks.executeAndWaitResult(Tasks.scala:423)
	at scala.collection.parallel.ForkJoinTaskSupport.executeAndWaitResult(TaskSupport.scala:56)
	at scala.collection.parallel.ExecutionContextTasks.executeAndWaitResult(Tasks.scala:555)
	at scala.collection.parallel.ExecutionContextTaskSupport.executeAndWaitResult(TaskSupport.scala:80)
	at scala.collection.parallel.immutable.ParRange.map(ParRange.scala:34)
	at $line4.$read$$iw$$iw$.<init>(<console>:12)
	at $line4.$read$$iw$$iw$.<clinit>(<console>)
	at $line4.$eval$.$print$lzycompute(<console>:7)
	- locked <0x00000007a9abdbb0> (a $line4.$eval$)
	at $line4.$eval$.$print(<console>:6)
	at $line4.$eval.$print(<console>)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at scala.tools.nsc.interpreter.IMain$ReadEvalPrint.call(IMain.scala:781)
	at scala.tools.nsc.interpreter.IMain$Request.loadAndRun(IMain.scala:1042)
	at scala.tools.nsc.interpreter.IMain$WrappedRequest.scala$tools$nsc$interpreter$IMain$WrappedRequest$$$anonfun$54(IMain.scala:633)
	at scala.tools.nsc.interpreter.IMain$WrappedRequest$$Lambda$981/1862816278.apply(Unknown Source)
	at scala.reflect.internal.util.AbstractFileClassLoader.asContext(AbstractFileClassLoader.scala:19)
	at scala.tools.nsc.interpreter.IMain$WrappedRequest.loadAndRunReq(IMain.scala:632)
	at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:564)
	at scala.tools.nsc.interpreter.IMain.interpret(IMain.scala:560)
	at scala.tools.nsc.interpreter.ILoop.interpretStartingWith(ILoop.scala:804)
	at scala.tools.nsc.interpreter.ILoop.command(ILoop.scala:678)
	at scala.tools.nsc.interpreter.ILoop.processLine(ILoop.scala:393)
	at scala.tools.nsc.interpreter.ILoop.loop(ILoop.scala:413)
	at scala.tools.nsc.interpreter.ILoop.scala$tools$nsc$interpreter$ILoop$$$anonfun$91(ILoop.scala:915)
	at scala.tools.nsc.interpreter.ILoop.process(ILoop.scala:901)
	at scala.tools.nsc.MainGenericRunner.runTarget$1(MainGenericRunner.scala:74)
	at scala.tools.nsc.MainGenericRunner.run$1(MainGenericRunner.scala:87)
	at scala.tools.nsc.MainGenericRunner.process(MainGenericRunner.scala:98)
	at scala.tools.nsc.MainGenericRunner$.main(MainGenericRunner.scala:103)
	at scala.tools.nsc.MainGenericRunner.main(MainGenericRunner.scala)

[scabug]

Imported From: https://issues.scala-lang.org/browse/SI-9824?orig=1
Reporter: @lrytz
Affected Versions: 2.12.0
See #9076, #8119
Duplicates #10007

[scabug]

@retronym said:
Duplicate of #8119 ?

[scabug]

@retronym said:
http://stackoverflow.com/questions/15176199/scala-parallel-collection-in-object-initializer-causes-a-program-to-hang/15176433#15176433

[scabug]

@retronym said:
See also: #9076

[scabug]

@lrytz said:
the parallel collections example passes with -Yrepl-class-based, so that's a duplicate of #8119.

[scabug]

@SethTisue said:
regretfully postponed to 2.12.1. we should list as a known bug in the release notes.

[scabug]

@som-snytt said:
The linked ticket includes a bug which is different from generic "threads + static init = deadlock"; also, the deadlock problem is somewhat more insidious with lambdas, apparently.

[scabug]

@SethTisue said:
@adriaanm you put this in Backlog, but it's in the 2.12.0 release notes as a notable known bug...?

[scabug]

@adriaanm said:
Looks like A.P.'s PR around sys.process actually fixed this in 2.12.1. There's a lot of things going on in these tickets, so I'm not sure anymore whether we can close this based on this evidence alone:

Welcome to Scala 2.12.1 (Java HotSpot(TM) 64-Bit Server VM, Java 1.8.0_112).
Type in expressions for evaluation. Or try :help.

scala> import scala.sys.process._
import scala.sys.process._

scala>  println(("ls -al /" #| "grep u").!!)
drwxr-xr-x@   11 root  wheel    374 Nov  2 12:03 usr
// more stuff from my / ;-)

[scabug]

@SethTisue said:
maybe it's #8119 we should still consider doing something about for 2.12.x, though. for (i <- (1 to 100).par) yield i still hangs. that's actually the hang I had considered release-note-worthy

[scabug]

@som-snytt said (edited by @adriaanm on Dec 19, 2016 7:14:39 PM UTC):
Pretty sure this dupes #10007 and #9076 is the canonical REPL ticket, superseding #8119 (which is marred by par).