Skip to content

kvserver: AddSSTable option to write at current timestamp #70422

@erikgrinaker

Description

@erikgrinaker

As described in #69380, the fact that AddSSTable can write keys with timestamps far in the past is problematic, since it violates invariants that other parts of the system rely on, such as MVCC immutability and closed timestamps. To avoid this, AddSSTable should in the typical case write at a current (given) HLC timestamp. However, rewriting the keys with a new timestamp is costly, and the whole point of AddSSTable is to efficiently write bulk data. To get around this, we should introduce support for synthesizing the timestamp of the added keys at read-time, and only rewrite them during Pebble compactions.

Implementation details are intentionally underspecified here, and need to be explored. See also the RFC in #69380 for further info. Functionality that relies on writing at historical timestamps will need to be updated separately with this new behavior.

Epic CRDB-2624

Metadata

Metadata

Assignees

Labels

A-kv-transactionsRelating to MVCC and the transactional model.A-storageRelating to our storage engine (Pebble) on-disk storage.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)T-storageStorage Team

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions