I think it is not worth optimizing. Because ConsensusPayload will not be stored.

But the messages are smaller

I think we can even go as far as

if (Witness.VerificationScript.Length != 0) {
    return false
}

Or really just leave the invocation script in the payload. And then remove GetNextBlockValidators()/CreateSignatureRedeemScript() from GetScriptHashesForVerifying() as it's already done in the Verify.

Smaller messages are always good to me, especially if they cost nothing like here. We need ConsensusPayloads to be signed by very specific keys, we know them all, so carrying these scripts with every payload just to check that they really are the same as the ones we produce in GetScriptHashesForVerifying() is just a waste of bandwidth (and time to serialize/deserialize them, and memory to hold them for a while). Not a huge waste, but still lowering it to zero can't be bad.

It should be done in neo-modules, I think that it's a good solution to speed up p2p protocol, if we decide to use it, I can copy the code to neo-modules.