This patch set improves clearing of range keys, with the overall goal of avoiding dropping Pebble range tombstones unnecessarily, in particular across range key spans that will most often be empty. The main affected components are:

• ClearRange gRPC method: similarly to point keys, it now uses an iterator to clear individual range keys for small data sets. For larger data sets, it will use Pebble range tombstones to clear point or range keys separately, but only if any such keys exist.

• Raft snapshots

  • Unreplicated data: no change, we only write a Pebble range tombstone across the point key span, since we don't expect range keys here.
  • Subsumed replicas' range-local state: iterates and clears any point/range keys (individually or ranged).
  • Snapshot SSTs: every SST (for each key span in the Raft range) unconditionally has a Pebble range tombstone at the bottom, both across point keys and range keys.

• Raft splits: on RHS conflict, iterates and clears any RHS point/range keys (individually or ranged).

• Raft merges: for RHS range-local data, iterates and clears any RHS point/range keys (individually or ranged).

• Raft replica removal: iterates and clears any RHS point/range keys (individually or ranged).

• Raft log truncation: no change, only writes tombstones across point keys.

The iffiest bit is the unconditional Pebble range tombstones at the bottom of Raft snapshot SSTs, both because they're unconditional, and also because they will be fairly common. @jbowens Do you consider this problematic? Specifically, this bit here:

Resolves #83032.

storage: add point/range key params to Writer.ClearRawRange()

This patch adds boolean parameters for point/range keys to
Writer.ClearRawRange(). When true, it will unconditionally write
Pebble range tombstones across the point/range key spans. This gives the
caller better control over when to write tombstones, to avoid writing
them unnecessarily.

This is a behavioral change, because the previous Engine and Batch
implementations would only drop Pebble range key tombstones if there
were range keys present. Later commits will introduce higher-level
functionality to avoid writing these tombstones unnecessarily and adapt
callers.

Release note: None

storage: clear individual range keys in ClearMVCCIteratorRange

Previously, Writer.ClearMVCCIteratorRange would use a Pebble
RANGEKEYDEL to clear all range keys within the span. This went against
the intent of the method, which is to iterate across the the span and
clear individual range keys. This patch changes is to do the latter.

Parameters are also added to control whether to clear point and/or range
keys.

Release note: None

storage: remove Writer.ClearAllRangeKeys

This patch removes Writer.ClearAllRangeKeys(), which was used to clear
all range keys in a span using a single RANGEKEYDEL tombstone.
ClearRawRange should be used instead.

This also removes some clumsy Engine logic that attempted to detect
and avoid writing range key tombstones if there were no range keys,
which required additional metadata tracking in batches and such. The
responsibility for this is instead placed on callers. It was never
possible to handle this sufficiently inside the writers anyway, because
of e.g. SSTWriter which does not have access to a reader.

Release note: None

storage: add Writer.ClearEngineRangeKey()

This patch adds Writer.ClearEngineRangeKey(), and makes
ClearMVCCRangeKey() call through to it. It also takes the opportunity
to consolidate the put logic in a similar fashion, by calling through to
the engine method.

Release note: None

storage: improve ClearRangeWithHeuristic for range keys

This patch improves ClearRangeWithHeuristic() to take individual,
optional thresholds for point and/or range keys, controlling when to use
RANGEDEL or RANGEKEYDEL instead of individual tombstones. The Raft
replica clearing code has been adapted to take advantage of this, and in
particular, will no longer drop Pebble range key tombstones unless there
are actual range keys present.

Release note: None

kvserver/batcheval: improve range key clears in ClearRange

This patch improves range key clearing in ClearRange, by using
ClearRangeWithHeuristic to only write Pebble range key tombstones if
there are any range keys present.

Release note: None

storage: add point/range key params to Writer.ClearMVCCRange

This patch adds point/range key parameters to Writer.ClearMVCCRange()
controlling whether to clear the key type. This can be used by callers
to avoid dropping unnecessary tombstones, particularly across range key
spans that are expected to be empty most of the time.

Release note: None