Description:
Many tests/test libraries over-include especially mock libraries which causes increased memory usage, disk usage, and compilation time. This is especially a problem on Windows with cl.exe but we have also observed significant resource usage that can be reduced by as much as half on Linux with clang.

Attached to this issue is a file showing the peak working set memory usage of cl.exe compiling various tests/test libraries (units in KB). We can see many of the tests cause the compiler to use multiple GB of memory, often for simple tests that do not actually require many of the includes they are being built with. We have been able to reduce this usage (see linked PR) by cleaning up these includes. Our primary focus has been enabling faster/less resource intensive compilation of tests in Windows CI but this endeavor seems like it would be relevant to compilation performance on other platforms. We can potentially reduce compilation time, disk usage of artifacts/parameter files, and memory usage of the compiler (by orders of magnitude in some cases, multiple GB reduction in others).

While we have had some success empirically assessing the memory usage of compiling tests and finding the most expensive, it is not super feasible to go one-by-one and manually assess/fix the includes etc. to pare-down each test. We are hoping to potentially get some help reducing the overhead of compiling these tests from other contributors working toward similar changes as in the PR linked below as well as finding a programmatic solution that could potentially be automated (in CI or otherwise) that could identify over-inclusion of mocks and other large libraries that are not used. Other strategies to help solve this could be further breaking down larger libraries with many classes into their component parts so tests only include exactly what interfaces etc. they need.

Relevant Links:

• See Lower heap and disk space used by compiling kafka tests #10915 for a relevant PR that addresses this for the kafka network filter extension tests
• Gist with peak working set memory usage for tests that exceed 2GB: https://gist.github.com/sunjayBhatia/d2d4a0af1eab16777f94a4dc4959b342
• Powershell script used to collect data: https://github.com/greenhouse-org/envoy/blob/84fca545722432a48371fca1ced02253cddb86d0/get-cl-workingset-peak.ps1