Skip to content

partial parallelisation of genbcode, and code that it touches [ci: last-only]#5800

Closed
mkeskells wants to merge 15 commits intoscala:2.12.xfrom
rorygraves:2.12.x_genbCodeParallel_noprofile++
Closed

partial parallelisation of genbcode, and code that it touches [ci: last-only]#5800
mkeskells wants to merge 15 commits intoscala:2.12.xfrom
rorygraves:2.12.x_genbCodeParallel_noprofile++

Conversation

@mkeskells
Copy link
Contributor

@mkeskells mkeskells commented Mar 22, 2017

Genbcode has internal phases
worker1
optimisation
Worker2
Worker3

This parallelises optimisation ( under some circumstances) with worker1
when worker1 has finished multiple worker2 and 3 can commence

the unit of work for the paralelisation is changes to be a source file ( was a class)

There is some modification of the IO patterns

I/O is very expensive in windows, so reduceing the file operation reduces the stat calls

minor inlining changes to reduce memory usage

Partial move to nio for performance

canched to data structures for thread safety

small changes to IO library for type refinement

added a -Y option to enable/disable parallel running

running benchmarks on a warmed up VM using sbt to compile akka-actor I get the following times based on an quad core I7 laptop running windows 10 with SSD

variance is based on 60 compile cycles, removing the first 10 as warmup

the tool used to measure these results will be contributed as a seperate PR shortly

                 RunName                  AllWallMS                  JVMWallMS                 AllocatedAll                 AllocatedJVM
              00_baseline   10492.68 [+1.05% -0.93%]    3702.35 [+1.05% -0.91%]       2777.80 [+1.00% -1.00%]        610.50 [+1.00% -1.00%]
  01_genBcodeBaseDisabled    9317.29 [+1.05% -0.92%]    2281.07 [+1.10% -0.92%]       2755.94 [+1.00% -1.00%]        593.23 [+1.00% -1.00%]
       02_genBCodeEnabled    8123.83 [+1.06% -0.92%]    1194.58 [+1.11% -0.90%]      

for a 8 core I7 running unix with SSD the reduction is not as high but still measurable

                 RunName	                AllWallMS	                JVMWallMS	             AllocatedAll	             AllocatedJVM
              00_baseline	  8372.79 [+1.04% -0.97%]	  1095.86 [+1.05% -0.96%]	  2669.99 [+1.00% -1.00%]	   625.46 [+1.00% -1.00%]
  01_genBcodeBaseDisabled	  8104.36 [+1.05% -0.96%]	  1036.74 [+1.05% -0.94%]	  2644.63 [+1.00% -1.00%]	   609.01 [+1.00% -1.00%]
       02_genBCodeEnabled	  7987.16 [+1.04% -0.94%]	   846.31 [+1.09% -0.90%]	  

@scala-jenkins scala-jenkins added this to the 2.12.2 milestone Mar 22, 2017
@adriaanm adriaanm changed the title partial parallelisation of genbcode, and code that it touches partial parallelisation of genbcode, and code that it touches [ci: last-only] Mar 23, 2017
@mkeskells mkeskells closed this Mar 30, 2017
@mkeskells
Copy link
Contributor Author

replaced with #5815

@SethTisue SethTisue removed this from the 2.12.2 milestone Apr 10, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants