steps

scripted test running on Windows (e.g. appveyor)

build.sbt

libraryDependencies += "com.novocode" % "junit-interface" % "0.11" % "test"
ivyLoggingLevel := UpdateLogging.Quiet
exportJars := true
fork := true

test

# this test is only relevant on Windows
# where stale references to the jars will
# block file deletion of the jars.
> test
$ exists target/scala-2.10/exportjars_2.10-0.1-SNAPSHOT.jar
$ exists target/scala-2.10/exportjars_2.10-0.1-SNAPSHOT-tests.jar
$ delete target/scala-2.10/exportjars_2.10-0.1-SNAPSHOT.jar
$ delete target/scala-2.10/exportjars_2.10-0.1-SNAPSHOT-tests.jar
$ absent target/scala-2.10/exportjars_2.10-0.1-SNAPSHOT.jar
$ absent target/scala-2.10/exportjars_2.10-0.1-SNAPSHOT-tests.jar

problem

There is a well-known bug in the JVM, that classloaders do not release their jars

• http://management-platform.blogspot.co.uk/2009/01/classloaders-keeping-jar-files-open.html
• https://github.com/ancoron/apache-cxf/blob/master/common/common/src/main/java/org/apache/cxf/common/logging/JDKBugHacks.java
• http://stackoverflow.com/questions/7371277

this manifests as a memory leak on all platforms, but on Windows it is particularly bad because file handles mean a lock on the jars, which means they cannot be deleted, moved or changed.

A catastrophic consequence is that compilations silently fail (!!!) on Windows. This is not a corner case, it happens every single time reliably.

I experience this when I run tests when the junit-interface is loaded but after spending the evening reading the junit-interface code, I am pretty convinced that the bug can only be worked around at the level of the classloader, and sbt is the owner of that lifecycle.

expectation

no leaks, no file locks.

notes

Originally raised at sbt/junit-interface#67

This escapes the workaround proposed in #1223 as classloader jars seem to get "lost" in the heap and are never reclaimed by explicit GC or runFinalizers.

The scala compiler is also doing a similar thing, as I observed in https://issues.scala-lang.org/browse/SI-9682