Describe the bug
We were running fourmolu-0.3.0.0 to format all of our code at work, and it would take about a minute for 3,000+ source files and 250kloc. I upgraded to fourmolu-0.7.0.0 which is based on ormolu-0.5.0.0 and this resulted in a massive slowdown: the same task now takes 20-30 minutes to run. I've tested ormolu-0.5.0.0 (installed from Hackage, as well as HEAD on master here) and the performance problem remains.
As described in #896 , you can get nice multithreading which ameliorates the problem using fd (thanks @amesgen!). On my machine, this decreased the time to about 5 minutes. This is acceptable, but not ideal.
I got HEAD building with this diff to stack.yaml:
diff --git a/stack.yaml b/stack.yaml
index cf17657..64026cf 100644
--- a/stack.yaml
+++ b/stack.yaml
@@ -1,4 +1,4 @@
-resolver: lts-19.2
+resolver: lts-19.11
packages:
- '.'
@@ -7,7 +7,7 @@ packages:
extra-deps:
- Cabal-3.6.2.0
-- ghc-lib-parser-9.2.1.20211101
+ # - ghc-lib-parser-9.2.1.20211101
nix:
packages:
Building with this command:
stack install . --executable-profiling --resolver nightly --library-profiling --profile
And finally running on three large test files (about 9kloc between them):
ormolu --mode check src/Client/Apex/Types/Atlas.hs test/Gen.hs src/Client/Bank/Primitive.hs +RTS -p -RTS
The top few lines of the profile give everywhere and thenP as our pproblems, taking up 38% of runtime.
Tue Jun 14 10:15 2022 Time and Allocation Profiling Report (Final)
ormolu +RTS -p -RTS --mode check src/Client/Apex/Types/Atlas.hs test/Gen.hs src/Client/Bank/Primitive.hs
total time = 8.49 secs (8487 ticks @ 1000 us, 1 processor)
total alloc = 5,711,086,104 bytes (excludes profiling overheads)
COST CENTRE MODULE SRC %time %alloc
everything.go Data.Generics.Schemes src/Data/Generics/Schemes.hs:122:5-37 22.1 4.1
thenP.\ GHC.Parser.Lexer compiler/GHC/Parser/Lexer.x:(2396,9)-(2398,40) 16.4 1.8
== Distribution.Utils.ShortText src/Distribution/Utils/ShortText.hs:96:29-30 4.2 0.0
matchIgnoringSrcSpans.genericQuery Ormolu.Diff.ParseResult src/Ormolu/Diff/ParseResult.hs:(77,5)-(104,32) 3.9 0.0
<>.combineNub Distribution.Types.BuildInfo src/Distribution/Types/BuildInfo.hs:215:7-44 3.7 0.1
getExtensionAndDepsMap.buildMap Ormolu.Utils.Cabal src/Ormolu/Utils/Cabal.hs:(164,5)-(167,40) 2.1 0.4
ext1 Data.Generics.Aliases src/Data/Generics/Aliases.hs:326:1-43 1.5 1.4
linesInRegion Ormolu.Processing.Common src/Ormolu/Processing/Common.hs:(38,1)-(41,67) 1.3 4.6
gfoldlAccum Data.Generics.Twins src/Data/Generics/Twins.hs:(91,1)-(94,26) 1.2 0.1
I've generated a flamegraph from this profile:
This points to a potential regression in the Data.Generics.Schemes code. I'm curious if we have a regression that is related to Simplified Subsumption, since that module makes heavy use of the GenericQ type alias, which is type GenericQ r = forall a. Data a => a -> r.
everything is defined here, and it is used in ormolu through the listify function.
Describe the bug
We were running
fourmolu-0.3.0.0to format all of our code at work, and it would take about a minute for 3,000+ source files and 250kloc. I upgraded tofourmolu-0.7.0.0which is based onormolu-0.5.0.0and this resulted in a massive slowdown: the same task now takes 20-30 minutes to run. I've testedormolu-0.5.0.0(installed from Hackage, as well as HEAD onmasterhere) and the performance problem remains.As described in #896 , you can get nice multithreading which ameliorates the problem using
fd(thanks @amesgen!). On my machine, this decreased the time to about 5 minutes. This is acceptable, but not ideal.I got
HEADbuilding with this diff tostack.yaml:Building with this command:
And finally running on three large test files (about 9kloc between them):
The top few lines of the profile give
everywhereandthenPas our pproblems, taking up 38% of runtime.I've generated a flamegraph from this profile:
This points to a potential regression in the
Data.Generics.Schemescode. I'm curious if we have a regression that is related to Simplified Subsumption, since that module makes heavy use of theGenericQtype alias, which istype GenericQ r = forall a. Data a => a -> r.everythingis defined here, and it is used inormoluthrough thelistifyfunction.