-
Notifications
You must be signed in to change notification settings - Fork 4.1k
storage: Consider moving raft log out of rocksdb #7807
Copy link
Copy link
Closed
Labels
A-storageRelating to our storage engine (Pebble) on-disk storage.Relating to our storage engine (Pebble) on-disk storage.C-investigationFurther steps needed to qualify. C-label will change.Further steps needed to qualify. C-label will change.
Milestone
Description
Writing the raft log to rocksdb (which uses its own write-ahead log) is probably suboptimal; it may be more efficient to manage the raft log separately (although this is not clear: as long as raft log entries are GC'd aggressively then it's going to be hard to beat rocksdb).
For future reference:
- https://www.usenix.org/system/files/conference/inflow14/inflow14-yang.pdf discusses the downsides of layering log systems on one another
- Intel has demonstrated impressive performance increases by putting the raft log in non-volatile memory instead of disk: http://thenewstack.io/intel-gives-the-etcd-key-value-store-a-needed-boost/
- https://ayende.com/blog/174753/fast-transaction-log-linux?key=273aa566963445188a9c1c5ef3463311 and http://ayende.com/blog/174785/fast-transaction-log-windows discuss the fastest way to implement a write-ahead log.
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
A-storageRelating to our storage engine (Pebble) on-disk storage.Relating to our storage engine (Pebble) on-disk storage.C-investigationFurther steps needed to qualify. C-label will change.Further steps needed to qualify. C-label will change.