partial parallelisation of genbcode, and code that it touches [ci: last-only]#5800
Closed
mkeskells wants to merge 15 commits intoscala:2.12.xfrom
Closed
partial parallelisation of genbcode, and code that it touches [ci: last-only]#5800mkeskells wants to merge 15 commits intoscala:2.12.xfrom
mkeskells wants to merge 15 commits intoscala:2.12.xfrom
Conversation
group by outer class
prioritization of threads better file/dir handling
Contributor
Author
|
replaced with #5815 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Genbcode has internal phases
worker1
optimisation
Worker2
Worker3
This parallelises optimisation ( under some circumstances) with worker1
when worker1 has finished multiple worker2 and 3 can commence
the unit of work for the paralelisation is changes to be a source file ( was a class)
There is some modification of the IO patterns
I/O is very expensive in windows, so reduceing the file operation reduces the stat calls
minor inlining changes to reduce memory usage
Partial move to nio for performance
canched to data structures for thread safety
small changes to IO library for type refinement
added a -Y option to enable/disable parallel running
running benchmarks on a warmed up VM using sbt to compile akka-actor I get the following times based on an quad core I7 laptop running windows 10 with SSD
variance is based on 60 compile cycles, removing the first 10 as warmup
the tool used to measure these results will be contributed as a seperate PR shortly
for a 8 core I7 running unix with SSD the reduction is not as high but still measurable