In #338, we discussed the performance penalties incurred by runtime PMREM generation.
It is feasible to move most of this work off the main thread into a worker. This should be possible to do even without OffscreenCanvas support (although it would probably be notably slower to compute off of the GPU).
This work might tie in well with #37 , but we should take care to design it to support scenarios where GL rendering is still confined to the main thread.