-
Notifications
You must be signed in to change notification settings - Fork 4.1k
kvserver: AddSSTable option to write at current timestamp #70422
Description
As described in #69380, the fact that AddSSTable can write keys with timestamps far in the past is problematic, since it violates invariants that other parts of the system rely on, such as MVCC immutability and closed timestamps. To avoid this, AddSSTable should in the typical case write at a current (given) HLC timestamp. However, rewriting the keys with a new timestamp is costly, and the whole point of AddSSTable is to efficiently write bulk data. To get around this, we should introduce support for synthesizing the timestamp of the added keys at read-time, and only rewrite them during Pebble compactions.
Implementation details are intentionally underspecified here, and need to be explored. See also the RFC in #69380 for further info. Functionality that relies on writing at historical timestamps will need to be updated separately with this new behavior.
Epic CRDB-2624