Skip to content

storage: revisit Writer range clear API and heuristics #83032

@erikgrinaker

Description

@erikgrinaker

In #82041, we began clearing range keys in the Engine range clear methods, e.g. ClearRawRange. While this logic is correct, it can result in dropping unnecessary Pebble range tombstones across range key spans where there are no range keys. A heuristic was added to Pebble.ExperimentalClearAllRangeKeys() that would scan the span looking for any range keys before dropping a tombstone, but this logic is insufficient -- in particular, it only applies to Pebble clears, but not when writing to an SSTWriter who can't know anything about the existence of range keys in the key span it may be ingested into.

Several improvements may be needed, e.g.:

  • Lift heuristics that look for existing range keys out from ClearAllRangeKeys.
  • ClearMVCCIteratorRange should scan and clear range keys separately.
  • ClearRangeWithHeuristic should scan and clear range keys separately.
  • ClearRawRange should be able to clear point/range keys separately, either via parameter or separate methods, and do so unconditionally. Callers must implement their own heuristics.
  • KV's ClearRange RPC method needs improved heuristics that also take into account range keys.

And so on.

Jira issue: CRDB-16806

Epic CRDB-2624

Metadata

Metadata

Assignees

Labels

A-kv-replicationRelating to Raft, consensus, and coordination.A-storageRelating to our storage engine (Pebble) on-disk storage.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions