While UmbralPrivateKey objects are in memory, they could cache a copy of their corresponding UmbralPublicKey, so they don't have to compute it every time it's called.