So what I think we should start out with here is the following. The idea here is to have a fast-moving write front and put lots of data into a table while not having any contention (which would slow us down). And the point is to create a data set that has lots of RocksDB tombstones, which is achieved by running it for extended periods of time on a cluster with a low GC TTL (say 10min). We can use the created dataset to create a store dump (a backup would not contain the tombstones), and write targeted tests that make sure that we handle the tombstones in a way that doesn't create a huge disruption.

// Should also add a payload to the key, omitted here.
create table queue(ts int, client_id int, payload byte) // primary (ts, client_id)

for client i (i = 1..num_clients):
in iteration j (partitioned monotonically across all clients, i.e. no two clients will ever reuse a j, which is the "timestamp", but so that there's a well-defined write front)

INSERT INTO tbl VALUES (j, i, random_payload()), (j+1, i, random_payload()), ..., (j + 999, i, random_payload());
DELETE FROM tbl WHERE ts >= j && ts < j+1000;
j += 1000 // really need to do this before using any j's so that others don't use overlapping ones

roachtest: test a queue-type workload #24236

Description

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions