macOS users, @Commodore64user, @kodermike, how are build times on your machines? Our long does time make re takes? Is it faster without -l (time make re PARALLEL_LOAD=) too?

Real Macs just have the number of CPU cores they have without any virtualization weirdness so my hypothesis would be it'll matter less in whichever direction it goes.

That's what I'm thinking, but I want to make sure.

mac pro, 2.6 GHz 6-Core Intel Core i7
16 GB 2400 MHz DDR4

ran make clean and ccache -c [EDIT: SHOULD BE ccache -C] before each run. lmk if A) I misunderstood what you were looking for and/or B) you need me to rerun with distclean

time make re:

________________________________________________________
Executed in  636.88 secs    fish           external
   usr time  627.43 secs    0.77 millis  627.43 secs
   sys time  546.38 secs    1.78 millis  546.38 secs

time make re PARALLEL_LOAD=:

________________________________________________________
Executed in  239.42 secs    fish           external
   usr time  673.87 secs    0.35 millis  673.87 secs
   sys time  572.52 secs    1.03 millis  572.52 secs

for context (if it matters), full build:

▸100% | Installing 'koreader'
________________________________________________________
Executed in   15.94 mins    fish           external
   usr time   38.71 mins    0.48 millis   38.71 mins
   sys time   13.62 mins    1.95 millis   13.62 mins

Thanks. make re is equivalent to make clean all, so no need to clean beforehand.

Around ~10 minutes… Are those timings really with ccache -c, not ccache -C, so from a hot cache?

For comparison, I have an Intel Core i7-3770 (4 cores, 8 processors detected with multithreading), and make KODEBUG= -j8 -l6 re takes me ~14 minutes from a cold cache, and ~1 minute from a hot cache.

If you install ninjatracing, you can get more precise build timings after a build by running make buildstats.

Made me curious. On my Ryzen 7 7800X3D:

$ ccache -c
Cleaning... 100.0% [===================================================================================================]
$ time make re
make[1]: Entering directory '/home/frans/src/kobo/koreader-base'

[…]

▸ 98% | Building 'koreader'
      | [koreader 100%] Linking CXX shared library libkoreader-cre.so
▸100% | Installing 'koreader'
make[1]: Leaving directory '/home/frans/src/kobo/koreader-base'

real	1m25.084s
user	10m58.757s
sys	1m39.967s

make re  303.80s user 223.10s system 100% cpu 8:43.30 total

make re PARALLEL_LOAD=  421.78s user 305.49s system 360% cpu 3:21.80 total

On my Ryzen 7 7800X3D

show off ;)

Before that I did a decade with an i7-4790. :-)

The nice (or annoying?) thing is that if you now buy a reasonably priced i3 you get something similar but better.

🤦 - that was with ccache -c; second set, with -C

time make re

________________________________________________________
Executed in   20.47 mins    fish           external
   usr time   41.44 mins    3.50 millis   41.44 mins
   sys time   15.26 mins    5.78 millis   15.26 mins

time make re PARALLEL_LOAD=

________________________________________________________
Executed in  530.02 secs    fish           external
   usr time   48.83 mins    0.48 millis   48.83 mins
   sys time   17.49 mins    1.22 millis   17.49 mins

make re  303.80s user 223.10s system 100% cpu 8:43.30 total

make re PARALLEL_LOAD=  421.78s user 305.49s system 360% cpu 3:21.80 total

On my Ryzen 7 7800X3D

show off ;)

Not really, if this is again with ccache -c (cleanup), and not ccache -C (clear), depending on how hot the cache was.

🤦 - that was with ccache -c; second set, with -C

time make re

________________________________________________________
Executed in   20.47 mins    fish           external
   usr time   41.44 mins    3.50 millis   41.44 mins
   sys time   15.26 mins    5.78 millis   15.26 mins

time make re PARALLEL_LOAD=

________________________________________________________
Executed in  530.02 secs    fish           external
   usr time   48.83 mins    0.48 millis   48.83 mins
   sys time   17.49 mins    1.22 millis   17.49 mins

That's pretty bad performance (for the first command)… Did you forget to use ccache -C before the second command too?

I simply copied the command given above. I didn't realize it was wrong.

In any event, the difference seems to be minimal, and just as likely caused by playing a video.

▸100% | Installing 'koreader'
make[1]: Leaving directory '/home/frans/src/kobo/koreader-base'

real	1m27.625s
user	11m9.577s
sys	1m35.485s

depending on how hot the cache was.

Close to the complete opposite I'd imagine. I never compile for x86_64 release.

Anyway, it looks like -l is detrimental when building on a regular macOS desktop too.

-C

make re  1687.16s user 554.42s system 135% cpu 27:28.23 total

make re PARALLEL_LOAD=  2521.93s user 786.38s system 478% cpu 11:31.88 total

For unnecesary reference, ccache -C and make re PARALLEL_LOAD=

real	1m27.764s
user	11m17.608s
sys	1m36.143s

And then with a hot ccache:

time make re

real	0m23.289s
user	0m55.171s
sys	0m25.569s

time make re PARALLEL_LOAD=

real	0m16.984s
user	0m55.992s
sys	0m25.321s

For unnecesary reference

I think I'll make PARALLEL_LOAD use opt-in. And now that ninja has official job server support, it may be a good idea to revisit if overriding -j in sub-invocations of make / ninja is a good idea.

I think I'll make PARALLEL_LOAD use opt-in. And now that ninja has job server support, it may be a good idea to revisit if overriding -j / -l in sub-invocations of make / ninja is a good idea.

opt-out, no? non-parallel is taking like 30~40 mins here, currently at 75% still

PARALLEL_LOAD= does mean no parallelism: it disable the use of -l for taking cpu load into account before spawning new jobs (I have PARALLEL_LOAD=6 set in my dev env so I still have more or less one core left for other stuff).

updated the -C run

can you make it so we all get Frezie's results please...

Well, you made me paranoid I got distracted by rl, so ran again for posterity

w/ PARALLEL_LOAD=

________________________________________________________
Executed in  553.53 secs    fish           external
   usr time   53.39 mins    3.44 millis   53.39 mins
   sys time   19.17 mins    2.08 millis   19.17 mins

w / parallel

________________________________________________________
Executed in   16.33 mins    fish           external
   usr time   37.27 mins    0.35 millis   37.27 mins
   sys time   13.12 mins    1.05 millis   13.12 mins

letting the aircraft carrier rest now.