-
Notifications
You must be signed in to change notification settings - Fork 4.1k
kv: collapse adjacent span configs to reduce number of necessary splits #72389
Description
After #67679 + enabling the infrastructure by default, we'll get to a place where we'll split on every schema object boundary for all tenants (including, as before, the host tenant). That's mighty expensive and comes with a lot of overhead and is a scalability bottleneck for how many tenants we can pack into the same cluster. #70555 proposes some rudimentary guardrails, but on the KV side we can do better. The spanconfig.Store introduced in #70287 can be augmented to recognize that certain span config entries are adjacent to one another (S1.EndKey == S2.Key) and have the same configs. We expect the majority of span configs to be applicable here, given almost everything inherits from RANGE DEFAULT. When recognizing such an adjacency, we can avoid unconditionally splitting on that boundary. This should let us claw back a low range count with secondary tenants while still enabling multi-tenant zone configs.
Some neighboring issues:
- config,kvserver: don't unconditionally split on table boundaries for the host tenant #66063 (this same optimization can be applied to the host tenant)
- sql: support manual splits on table boundaries for secondary tenants #65903 (might still be desirable to induce manual splits on table boundaries, for say, perf reasons)
- kv,*: non-contiguous ranges #65726 (orthogonal issue describing a scheme to reduce the total number of ranges in the system)
Zendesk Ticket IDs
12099
Jira issue: CRDB-11130