Is this a BUG REPORT or FEATURE REQUEST? (choose one):
FEATURE REQUEST

The current implementation has the following definition of validator (according to spec)

type Validator struct {
    Address     []byte
    PubKey      PubKey
    VotingPower int64
}

This implies there is a corresponding private key which is very valuable. In production, either the key is copied to the validating server (at risk of theft), or according to current best practice the key pair is generated on a HSM. In which case the risk of theft is lowered, but still non-zero. Additionally, failover of the validating server becomes impossible unless one has a clone of the HSM pre-attached to a different machine. I remark that in case of theft, there is no on-chain mechanism to deal with this, except a slow painful death by slashing (in Cosmos) for those who do not unbond fast enough...

In order to be concrete, as a basis for discussion, I propose changing the validator struct to the following:

type Validator struct {
    Address       []byte
    OwningPubKey  PubKey  // used to authorize changes of SigningPubKey
    SigningPubKey PubKey  // used to sign proposals+votes
    VotingPower   int64
}

Where

• the OwningPubKey can send a transaction to change the SigningPubKey to an arbitary value
• the consensus protocol would generally accept such a transaction.

From my understanding of the consensus reactor, the ProposalMessage and VoteMessage would be signed by the current SigningPubKey, whereas a new "SigningKeyChangeMessage" would be added, to be signed by the OwningPubKey.

Effectively, the OwningPubKey becomes a certification authority for its SigningPubKey, taking cues from PKI theory:

• Setting the SigningPubKey to zero means revoking the key.
• Setting a new non-zero value means rotating the key.

I believe this change would not weaken the safety of the protocol, but I do not have a deep enough understanding to make such a claim. Does it?

This proposal allows keeping the OwningPubKey completely offline and separated from the operational setup. It also allows anonymous owners of bonded stake (in the Cosmos case) to listen to changes in the SigningPubKey.

Possible additional extensions (not proposed here, mentioned to illustrate the advantage of separating the roles of the current "one PubKey to rule them all"):

• field Operational bool, indicating if a validator is in operation. Alternatively, a zero SigningPubKey value could imply that a validator is not operational
• additional subkeys could be added, e.g. MessagingPubKey, with associated transaction types, communicating for instance various fees the validator extracts