-
Notifications
You must be signed in to change notification settings - Fork 4.1k
kv: don't commit-wait on each columnBackfiller chunk #61444
Copy link
Copy link
Closed
Labels
A-disaster-recoveryA-kvAnything in KV that doesn't belong in a more specific category.Anything in KV that doesn't belong in a more specific category.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)GA-blocker
Description
See #61386 (comment) and discussion in https://cockroachlabs.slack.com/archives/C2C5FKPPB/p1614790893188800.
Currently, each transaction issued by columnBackfiller.runChunk will perform a commit-wait stage of about 800ms when writing to a GLOBAL table. This adds up, as the ColumnBackfiller can issue many small transactions. The commit-wait step is meant to ensure linearizability with future reads of the values written by the column backfiller. However, there's no need to pay this cost on each chunk - we only need reads after the backfiller is completely finished to be guaranteed to observe its writes. So we should be able to pay the commit-wait cost once.
Proposal to lift commit-wait into the columnBackfiller:
- add
SkipCommitWaitAPI tokv.Txn - call
SkipCommitWaiton txn incolumnBackfiller.runChunk - record transaction commit time in
columnBackfiller.runChunk hlc.SleepUntilthe maximum of all chunk commit timestamps incolumnBackfiller.flush
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
A-disaster-recoveryA-kvAnything in KV that doesn't belong in a more specific category.Anything in KV that doesn't belong in a more specific category.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)GA-blocker