Skip to content

kv: don't commit-wait on each columnBackfiller chunk #61444

@nvb

Description

@nvb

See #61386 (comment) and discussion in https://cockroachlabs.slack.com/archives/C2C5FKPPB/p1614790893188800.

Currently, each transaction issued by columnBackfiller.runChunk will perform a commit-wait stage of about 800ms when writing to a GLOBAL table. This adds up, as the ColumnBackfiller can issue many small transactions. The commit-wait step is meant to ensure linearizability with future reads of the values written by the column backfiller. However, there's no need to pay this cost on each chunk - we only need reads after the backfiller is completely finished to be guaranteed to observe its writes. So we should be able to pay the commit-wait cost once.

Proposal to lift commit-wait into the columnBackfiller:

  1. add SkipCommitWait API to kv.Txn
  2. call SkipCommitWait on txn in columnBackfiller.runChunk
  3. record transaction commit time in columnBackfiller.runChunk
  4. hlc.SleepUntil the maximum of all chunk commit timestamps in columnBackfiller.flush

Metadata

Metadata

Assignees

Labels

A-disaster-recoveryA-kvAnything in KV that doesn't belong in a more specific category.C-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)GA-blocker

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions