What is the reason for reverting this change? We will need a new docker image for the CI because this PR change a plugin version

The generated jar contained the non-bootstrapped version :(

@allanrenucci is it possible to have some mapping of type system between the host OS and the docker, so that we don't need to build a new docker image when we change plugins or libraries.

test performance please

performance test scheduled: 1 job(s) in queue, 1 running.

Performance test finished successfully:

Visit http://dotty-bench.epfl.ch/3124 to see the changes.

Benchmarks is based on merge(s) with master

test performance please

performance test scheduled: 1 job(s) in queue, 0 running.

Performance test finished successfully:

Visit http://dotty-bench.epfl.ch/3124 to see the changes.

Benchmarks is based on merge(s) with master

Now we have the setting measure "tests/implicitNums.scala" 100 tests/implicitNums.scala.

test performance please

performance test scheduled: 2 job(s) in queue, 0 running.

performance test failed:

[check /data/workspace/bench/logs/pull-3124-09-14-18.47.out for more information]

Performance test finished successfully:

Visit http://dotty-bench.epfl.ch/3124 to see the changes.

Benchmarks is based on merge(s) with master

Exhaustivity T and V drop 200ms of runtime; I wonder why those tests in particular show such a big difference.

T and V are quite noisy, maybe because we need more warmups. This test is intended to check if more warmups can bring down the number of mini-tests. We set 100 warmups for implicit cache II, but it seems to have no effect -- the result is the same as 10 warmups + 20 runs.

JMH has a handy profiler, -prof hs_comp, that logs JIT compiler activity after each iteration. Observing the output after each measurement iteration to ensure that the overall time spent compiling is no longer rapidly growing helps to validate the chosen warmup duration.

You can disable your warmup altogether to see the output of this profiler periodically after each iteration.

Here's an example of 200s of execution of of a scalac benchmark: https://gist.github.com/109d3b6a3b72c9d87967f62985b78df9

Notice that in steady state, the delta of compiler.totalTime is low (~0.5s compile time per 10s measurement iteration), and the delta of compiler.totalCompiles is low (<100 methods).

In addition to selecting a warmup duration, selecting the number of forks is important. We tend to use 3, which I think is even too low and probably our biggest cause of jitter.

Thanks @retronym I will have a look.

test performance please

performance test scheduled: 1 job(s) in queue, 0 running.

Performance test finished successfully:

Visit http://dotty-bench.epfl.ch/3124 to see the changes.

Benchmarks is based on merge(s) with master