I'm seeing UnpackLibraryResources take over a second for a minimal project, and I'm guessing the most time is taken by reading all the reference assemblies:

Perhaps if we know that none of the in the box assemblies come with resources, we should exclude them from the list being passed to the task? I think this should shave off a second from each project build.

This is dotnet 6.0.202