roachtest: add KV/YCSB benchmarks with global MVCC range tombstones#85368
Conversation
|
This change is |
|
First kv95 run is looking pretty bad, with a ~30% regression. For comparison, the last nightly came in at 116,870 ops/s total (about 110k reads/s and 5,700 writes/s). |
ed9546a to
3149769
Compare
This patch adds the envvar `COCKROACH_GLOBAL_MVCC_RANGE_TOMBSTONE`. When enabled, it will write a global MVCC range tombstone across the entire table data keyspan during cluster bootstrapping. This can be used to test performance and correctness in the presence of MVCC range tombstones, by activating range key-specific code paths while not semantically affecting the data above it. Release note: None
This patch adds a set of benchmark variants that write a single MVCC range tombstone across the entire SQL keyspan at cluster start, via the `COCKROACH_GLOBAL_MVCC_RANGE_TOMBSTONE` env var. Even though this range tombstone will not affect the data written during the benchmarks, it activates range key-specific code paths in the storage layer which can have a significant impact on performance. The new benchmarks are: * `kv0/enc=false/nodes=3/cpu=32/mvcc-range-keys=global` * `kv95/enc=false/nodes=3/cpu=32/mvcc-range-keys=global` * `ycsb/A/nodes=3/cpu=32/mvcc-range-keys=global` * `ycsb/B/nodes=3/cpu=32/mvcc-range-keys=global` * `ycsb/C/nodes=3/cpu=32/mvcc-range-keys=global` * `ycsb/D/nodes=3/cpu=32/mvcc-range-keys=global` * `ycsb/E/nodes=3/cpu=32/mvcc-range-keys=global` * `ycsb/F/nodes=3/cpu=32/mvcc-range-keys=global` Release note: None
3149769 to
a1d87ac
Compare
jbowens
left a comment
There was a problem hiding this comment.
First kv95 run is looking pretty bad, with a ~30% regression.
😅
Reviewed 6 of 6 files at r1, 2 of 2 files at r2, all commit messages.
Reviewable status:complete! 1 of 0 LGTMs obtained (waiting on @aliher1911 and @pavelkalinnikov)
|
TFTR! bors r=jbowens |
Thanks for adding this, @erikgrinaker. Gotta start somewhere! |
nicktrav
left a comment
There was a problem hiding this comment.
Reviewed 6 of 6 files at r1, 2 of 2 files at r2, all commit messages.
Reviewable status:
|
Build succeeded: |
|
Oof, YCSB/B is looking at a 78% regression: |
|
I had a thought about this case where there's a single very broad range key. The range key will be split into 1 fragment for every sstable in L6. During iteration we defragment to ensure deterministic bounds, which means we need to load every range key block in L6, and compare bounds to defragment it. Respecting the iterator bounds when defragmenting should help. The worst case becomes defragmenting across all the range key blocks that overlap the KV range we're reading. Filed cockroachdb/pebble#1861. |
kvserver: add env var to write global MVCC range tombstone
This patch adds the envvar
COCKROACH_GLOBAL_MVCC_RANGE_TOMBSTONE.When enabled, it will write a global MVCC range tombstone across the
entire table data keyspan during cluster bootstrapping. This can be used
to test performance and correctness in the presence of MVCC range
tombstones, by activating range key-specific code paths while not
semantically affecting the data above it.
Touches #84384.
Release note: None
roachtest: add KV/YCSB benchmarks with global MVCC range tombstones
This patch adds a set of benchmark variants that write a single MVCC
range tombstone across the entire SQL keyspan at cluster start, via the
COCKROACH_GLOBAL_MVCC_RANGE_TOMBSTONEenv var. Even though this rangetombstone will not affect the data written during the benchmarks, it
activates range key-specific code paths in the storage layer which can
have a significant impact on performance.
The new benchmarks are:
kv0/enc=false/nodes=3/cpu=32/mvcc-range-keys=globalkv95/enc=false/nodes=3/cpu=32/mvcc-range-keys=globalycsb/A/nodes=3/cpu=32/mvcc-range-keys=globalycsb/B/nodes=3/cpu=32/mvcc-range-keys=globalycsb/C/nodes=3/cpu=32/mvcc-range-keys=globalycsb/D/nodes=3/cpu=32/mvcc-range-keys=globalycsb/E/nodes=3/cpu=32/mvcc-range-keys=globalycsb/F/nodes=3/cpu=32/mvcc-range-keys=globalResolves #84384.
Release note: None