-
Notifications
You must be signed in to change notification settings - Fork 4.1k
kvserver: avoid lease transfers to restarted nodes that could become overloaded #96521
Copy link
Copy link
Closed
Closed
Copy link
Labels
A-kv-distributionRelating to rebalancing and leasing.Relating to rebalancing and leasing.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)GA-blockerT-kvKV TeamKV Teambranch-release-23.1Used to mark GA and release blockers, technical advisories, and bugs for 23.1Used to mark GA and release blockers, technical advisories, and bugs for 23.1
Milestone
Description
We have seen that a node restarting in the cluster can have very negative effects on request latency.
A reproduction test was added in #95161
Allowing Lease Transfers
Extremely High P99 SQL Latency


Blocking Lease Transfers
No impact on P99 SQL Latency


This issue tracks the improvements to crdb in 23.1 to prevent a restarted node impacting end latency. The improvements are narrowly scoped but have been proven to have a significant effect in keeping request latency normal.
- storepool: consider stores suspect on join #97532
- kvserver: consider io overload for lease transfers #96508
Jira issue: CRDB-24164
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
A-kv-distributionRelating to rebalancing and leasing.Relating to rebalancing and leasing.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)GA-blockerT-kvKV TeamKV Teambranch-release-23.1Used to mark GA and release blockers, technical advisories, and bugs for 23.1Used to mark GA and release blockers, technical advisories, and bugs for 23.1