@@            Coverage Diff            @@
##           master   #10861     +/-   ##
=========================================
- Coverage   62.82%   62.73%   -0.1%     
=========================================
  Files         398      398             
  Lines       37446    37446             
=========================================
- Hits        23526    23492     -34     
- Misses      12339    12375     +36     
+ Partials     1581     1579      -2

// A Snapshot is sent by the leader to catch up a follower that has fallen behind.
// The snapshot corresponds to the on-disk replicated state of the distributed
// state machine as of the log index described by the metadata's Index and Term.
// When raft asks a follower to apply a Snapshot (as part of Ready handling), it
// is informing the application that the contents of the Raft log are made obsolete
// by the Snapshot. 
//
// If all data is on the same storage engine and can be manipulated atomically,
// the procedure is easy: apply the snapshot and the remainder of the Ready (it
// will typically at least include an updated HardState to reflect a new Commit
// index) in one atomic operation.
//
// If atomicity between the two is not available, a more complicated approach
// has to be taken. Assume that the application state (i.e the Data field) can
// be written atomically along with the index it belongs to (Metadata.Index).
// Assume further that the Raft log and HardState are on different storage
// engines, respectively, and no atomicity is available between the three.
//
// The Raft log poses the smallest problem. We must not delete from it before
// the snapshot data is durably applied (otherwise we break all of the promises
// we made by accepting entries into it). Assuming that, it won't matter whether
// the log gets removed before or after the HardState is updated. Care must be
// taken when initializing the system after a crash to understand that the Raft
// log needs to be compacted synchronously if the applied index is ahead of the
// last index. To avoid watering down detection of problems in general, this
// should be considered only if the data partition holds the immediate result
// of a snapshot application following the procedure detailed later in this
// comment.
//
// The bigger
// constraint exists between the HardState and snapshot data because they
// interact via HardState.Commit: if that is updated but then the system
// crash-restarts before the snapshot data is persisted, we'll see a commit
// index ahead of the last index, which is an undefined state and will panic. On
// the other hand, if the snapshot data is persisted but not the commit index,
// we will see the applied index ahead of the commit index, which is similarly
// problematic. To avoid that, the app can adapt Storage.InitialState so that it
// takes the commit index not only from the persisted HardState, but also takes
// into account any larger applied index in its data engine. Naively this waters
// down Raft's internal detection against data loss (if the storage engine
// unexpectedly lost HardState updates, this mechanism could mask it) but the
// app can additionally make this behavior conditional: when snapshot data is
// written to the data engine, it (atomically) includes a marker. At startup,
// only when the marker is present is the applied index taken into account and
// the HardState transparently updated, and then the flag removed again.
//
// It remains to detail how the Raft log is removed. As discussed, this is the
// last step and requires essentially carrying out a log compaction
// (occasionally called truncation) that removes the entirety of the log
// entries.
message Snapshot {
	optional bytes            data     = 1;
	optional SnapshotMetadata metadata = 2 [(gogoproto.nullable) = false];
}