kvserver: add garbage collection for range tombstones#76783
kvserver: add garbage collection for range tombstones#76783erikgrinaker wants to merge 4 commits intocockroachdb:masterfrom
Conversation
|
This change is |
|
Realized |
|
@jbowens One of the changes here is to add an |
4e74115 to
67d71b2
Compare
|
Ok, I think this should be ready for review now. I added a parameter This optimization could possibly also simply call |
67d71b2 to
33bdf5d
Compare
This patch adds an iteration key type `IterKeyTypePointsWithRanges` for `MVCCIterator` which iterates only over point keys, but surfaces range keys overlapping those points. Release note: None
This patch automatically clears all range keys (at any timestamp) in the `Engine` methods `ClearMVCCRangeAndIntents` and `ClearIterRange`. Range keys are not affected by `ClearRawRange` (which clears raw engine keys) and `ClearMVCCRange` (which is intended for clearing a subset of versions for a single key). This is implemented via a new `Engine.ExperimentalClearMVCCRangeKeys` method which calls through to Pebble's `RangeKeyDelete`, efficiently deleting all range keys in a key span. By extension, the `ClearRange` RPC method now also clears range keys, both when using point deletions and range deletions. Release note: None
This patch adds basic support for MVCC range tombstones in garbage collection. It garbage collects points below a range tombstone, as well as the range tombstones themselves. However, `MVCCStats` do not currently take range tombstones into account for garbage statistics -- this will be addressed separately. Garbage collection below range tombstones does not do anything fancy like dropping a Pebble range tombstone when there are no newer versions above the range tombstone -- it still uses point clears for every GCed key. This can be optimized later. Release note: None
This adds a parameter `ExperimentalRanges` to `GCRequest`, and a corresponding `ExperimentalGCRanges` version gate, which allows GCing large swathes of keys using Pebble range tombstones. This parameter is not yet in use, but is added preemptively to allow garbage collection to make use of it in the future when GCing below an MVCC range tombstone with no keys above it. Since MVCC range tombstones are experimental, this can possibly be added in a 22.1 patch release. Release note: None
33bdf5d to
4cdd8cd
Compare
storage: add
IterKeyTypePointsWithRangesforMVCCIteratorThis patch adds an iteration key type
IterKeyTypePointsWithRangesforMVCCIteratorwhich iterates only over point keys, but surfaces rangekeys overlapping those points.
Release note: None
storage: clear range keys in some
Engine.Clear*methodsThis patch automatically clears all range keys (at any timestamp) in the
EnginemethodsClearMVCCRangeAndIntentsandClearIterRange. Rangekeys are not affected by
ClearRawRange(which clears raw engine keys)and
ClearMVCCRange(which is intended for clearing a subset ofversions for a single key).
This is implemented via a new
Engine.ExperimentalClearMVCCRangeKeysmethod which calls through to Pebble's
RangeKeyDelete, efficientlydeleting all range keys in a key span.
By extension, the
ClearRangeRPC method now also clears range keys,both when using point deletions and range deletions.
Release note: None
kvserver: add garbage collection for range tombstones
This patch adds basic support for MVCC range tombstones in garbage
collection. It garbage collects points below a range tombstone, as well
as the range tombstones themselves. However,
MVCCStatsdo notcurrently take range tombstones into account for garbage statistics --
this will be addressed separately.
Garbage collection below range tombstones does not do anything fancy
like dropping a Pebble range tombstone when there are no newer versions
above the range tombstone -- it still uses point clears for every GCed
key. This can be optimized later.
Resolves #70414.
Release note: None
batcheval: add
ExperimentalRangesparameter forGCRequestThis adds a parameter
ExperimentalRangestoGCRequest, and acorresponding
ExperimentalGCRangesversion gate, which allows GCinglarge swathes of keys using Pebble range tombstones.
This parameter is not yet in use, but is added preemptively to allow
garbage collection to make use of it in the future when GCing below an
MVCC range tombstone with no keys above it. Since MVCC range tombstones
are experimental, this can possibly be added in a 22.1 patch release.
Release note: None