-
Notifications
You must be signed in to change notification settings - Fork 4.1k
storage: protect against raft proposal replay/reordering #6287
Description
See #6166.
Implement protection against Raft replays/reordering, ideally such that the leader knows at proposal time the log index at which commands commit (unless they have to be reproposed).
From #6166:
Replay protection is naively an issue because applying a logicless batch of writes is at odds with our current method, which largely relies on write/timestamp conflicts. However, as @bdarnell pointed out, as of recently (#5973) we (mostly) colocate the Raft leader and the leader lease holder on the same node and have more options for dealing with this (i.e. making the application of a command conditional on the Raft log position it was originally proposed at).
More concretely (via @bdarnell):
The RaftCmd would also include a term and log index, to be populated with the leader's latest log information when it is proposed. The writes would only be applied if it committed at that index; they would be dropped (and reproposed) if they committed at any other index. This would only be a possibility when the lease holder is not the raft leader (and even then it's unlikely in stable lease situations). We may be able to be even stricter about colocating the two roles (in which case we could perhaps remove raft-level replays completely), but I'm wary of forcing too many raft elections.