The compiler server caches DLLs on the assumption that compilations across a solution are likely to have a large number of identical DLL references. Given that our assembly symbols are immutable caching is safe and can provide significant speed up for builds.

The size of the cache though is presently 100 DLLs. This number was chosen back in the .NET Framework days when the number of references for a project was small. In the more modern .NET Core world this number is significantly higher. An ASP.NET Core app for instance can easily have 300 references hence the cache isn't really serving its intended purpose anymore.

We should update this number to be inline with modern .NET applications.