Sending a snapshot of a range currently involves generating a series of 256KB WriteBatches which are then applied on the recipient. Per the recent work to use the RocksDB IngestExternalFile functionality for restore, it would likely be significantly faster on the recipient to send an sstable instead. Using IngestExternalFile for applying snapshots should be straightforward in comparison to the usage within restore as we don't have any of the complexity of Raft involved. The sender can create the sstable locally and then stream it to the recipient. And because we send a header before streaming the snapshot, we can probe whether the recipient supports receiving an sstable snapshot which will make the migration story easier.

There is no particular urgency here as applying snapshots is not currently a bottleneck.

@danhhz Do you have benchmark numbers readily available showing the performance difference between IngestExternalFile and applying the equivalent WriteBatches?