sql: zeroqps during schema change with reduced bulk_io_write.max_rate #36806
Description
Describe the problem
While testing out the new index backfill I SET CLUSTER SETTING kv.bulk_io_write.max_rate = '1mb'. Once the schema change started it dropped QPS to zero on the cluster:
To Reproduce
export CLUSTER=andy-3node
roachprod create $CLUSTER -n 4 --clouds=aws --aws-machine-type-ssd=c5d.4xlarge
roachprod run $CLUSTER -- "sudo umount /mnt/data1; sudo mount -o discard,defaults,nobarrier /mnt/data1/; mount | grep /mnt/data1"
roachprod stage $CLUSTER:1-3 cockroach 0c83360778c511ab79103aefd8f5e3a115990144
roachprod adminurl --open $CLUSTER:1
roachprod run $CLUSTER:1 -- "./cockroach workload fixtures import tpcc --warehouses=2500 --db=tpcc"
Then run the test:
roachprod run $CLUSTER:4 "./workload run tpcc --ramp=5m --warehouses=2500 --active-warehouses=2100 --duration=1h --split --scatter {pgurl:1-3}"
SET CLUSTER SETTING kv.bulk_io_write.max_rate = '1mb'
CREATE UNIQUE INDEX ON tpcc.order (o_entry_d, o_w_id, o_d_id, o_carrier_id, o_id);
Environment:
19.1 branch
v19.1.0-rc.2-40-g0c83360