-
Notifications
You must be signed in to change notification settings - Fork 4.1k
storage: revisit Writer range clear API and heuristics #83032
Copy link
Copy link
Closed
Labels
A-kv-replicationRelating to Raft, consensus, and coordination.Relating to Raft, consensus, and coordination.A-storageRelating to our storage engine (Pebble) on-disk storage.Relating to our storage engine (Pebble) on-disk storage.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)
Description
In #82041, we began clearing range keys in the Engine range clear methods, e.g. ClearRawRange. While this logic is correct, it can result in dropping unnecessary Pebble range tombstones across range key spans where there are no range keys. A heuristic was added to Pebble.ExperimentalClearAllRangeKeys() that would scan the span looking for any range keys before dropping a tombstone, but this logic is insufficient -- in particular, it only applies to Pebble clears, but not when writing to an SSTWriter who can't know anything about the existence of range keys in the key span it may be ingested into.
Several improvements may be needed, e.g.:
- Lift heuristics that look for existing range keys out from
ClearAllRangeKeys. ClearMVCCIteratorRangeshould scan and clear range keys separately.ClearRangeWithHeuristicshould scan and clear range keys separately.ClearRawRangeshould be able to clear point/range keys separately, either via parameter or separate methods, and do so unconditionally. Callers must implement their own heuristics.- KV's
ClearRangeRPC method needs improved heuristics that also take into account range keys.
And so on.
Jira issue: CRDB-16806
Epic CRDB-2624
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
A-kv-replicationRelating to Raft, consensus, and coordination.Relating to Raft, consensus, and coordination.A-storageRelating to our storage engine (Pebble) on-disk storage.Relating to our storage engine (Pebble) on-disk storage.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)