On current master (7f52377) on Windows 10.0.19546 prerelease 200110-1443, SurfaceBook 2 13'' (Surface_Book_1832) it took 38 minutes to run a clean build (build -subsetCategory coreclr-libraries -runtimeConfiguration Release). The task itself reports only Time Elapsed 00:31:22.36, my guess is that the additional time is spent on restoring the build tools.

I ran the same on my personal Linux machine (PC not a notebook; Manjaro 19.0.2 Kyria kernel x86_64 Linux 5.5.7-1; Intel Core i7-4771 @ 8x 3.9GHz) and it took 19 minutes, reported 15:44:45.

Note that my PC is ~6yo, mid-level budget, custom build machine, nothing fancy.

I can share binlogs from both runs, but they're too big to attach here.