Description

🚀 Using a 16c/32t Intel chip - it has dropped non-cached, cold start build times by a whole minute. Reduces memory footprint and speeds up developer iteration.

This PR introduces unity builds into our project for Perl / Lua Bindings and Commands. The reason for this is to strike a sweet spot between frontend and backend compilation times parsing giant headers recursively for many different cpp files. It causes giant memory spikes and is extremely slow to do for many individual files. Many of these files introduced into these unity build units all need similar headers so they get "batched" together, reducing translation units, compilation time, less header parsing. Below details more of the nitty gritty and testing done to achieve a balance in batch #'s below -

This is based off of efforts to cleanup the codebase, modernize and make it more efficient.

I have a branch called build-analyze that runs a trace of what takes up all of the build time (example here http://drone.akkadius.com/EQEmu/Server/18259/1/2)

This has brought to light inefficiencies of our code base and with that some measurable things that we can do to substantially cut down on problem areas.

There are other things we are addressing through other strategies in other PRs.

⏱️ Build Time Improvements

Using a 16c/32t Intel chip - it has dropped non-cached, cold start build times by a whole minute.

💾 Massive Memory Reduction

• Previously, building these subsystems consumed multiple gigabytes of memory, especially under Clang with LLD or Gold
• With Unity builds, compiler frontends and backends operate on fewer, more efficient translation units, reducing peak memory usage dramatically and avoiding out-of-memory issues

🧠 Developer Productivity Boost

• Simplifies IDE indexing (fewer TUs)
• Reduces the compile/link impact of making small changes

Type of change

• Code build / compiler optimizations

Testing

Tested compiling, running basic processes etc.

Ran many tests using different variations of unity build batch size params --

Linux (16c/32t Core Intel - 5Ghz)

Master

rm -rf ~/.ccache/* && ninja clean && time ninja -j$(expr $(nproc) - 2)

2m22.041s

Unity Build Batch Size (Perl) 4 / (Lua) 4 / (GM Commands) 8

1m47.413s

Unity Build Batch Size (Perl) 8 / (Lua) 8 / (GM Commands) 16

1m39.186s

Unity Build Batch Size (Perl) 16 / (Lua) 16 / (GM Commands) 32

1m39.686s
1m41.284s

Unity Build Batch Size (Perl) 16 / (Lua) 16 / (GM Commands) 64

1m40.933s
1m40.275s

Unity Build Batch Size (Perl) 8 / (Lua) 8 / (GM Commands) 32

1m35.557s
1m36.772s

Incremental Build (Editing eqemu_logsys.h) (Perl) 8 / (Lua) 8 / (GM Commands) 32

0m15.853s
0m16.900s
0m17.065s

Checklist

• I have tested my changes
• I have performed a self-review of my code. Ensuring variables, functions and methods are named in a human-readable way, comments are added only where naming of variables, functions and methods can't give enough context.
• I own the changes of my code and take responsibility for the potential issues that occur