-
Notifications
You must be signed in to change notification settings - Fork 4.1k
changefeedccl: improve commit-to-emit row latency #36289
Copy link
Copy link
Closed
Labels
A-cdcChange Data CaptureChange Data CaptureC-performancePerf of queries or internals. Solution not expected to change functional behavior.Perf of queries or internals. Solution not expected to change functional behavior.T-cdcX-staleno-issue-activity
Description
The promise of RangeFeed is that we will push row changes from raft all the way out to the sink, which should result in very low end-to-end latencies for this. (Note: resolved timestamp latency, which is the only thing we test as of 2019-03-28, will always be bounded by closed timestamps and so is expected to be larger). However, this promise of low latency doesn't work in practice because of 2 small-ish issues.
- We currently wait for the TableHistory to be above the row timestamp, so that we're always guaranteed to emit each row with the very latest possible TableDescriptor. Specifically, during the second half of a schema change (e.g. adding a row), we always want to interpret a changed kv with the descriptor where the mutation is live so that the new column is in the output. An alternate is to switch TableHistory to use RangeFeed and stop blocking on the latest descriptor, instead using any descriptor that's valid to lease for the kv's updated timestamp. Then, if we see a descriptor where a mutation goes live, restart the poller from that descriptor's first valid timestamp (the modification timestamp) and re-emit any changes. This trades a small window of duplicates during a schema change (rare) for dramatically lower latency in the common case.
- We also need to make this retry with backoff into something based on signals. https://github.com/cockroachdb/cockroach/blob/master/pkg/ccl/changefeedccl/buffer.go#L218-L238
Jira issue: CRDB-4509
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
A-cdcChange Data CaptureChange Data CaptureC-performancePerf of queries or internals. Solution not expected to change functional behavior.Perf of queries or internals. Solution not expected to change functional behavior.T-cdcX-staleno-issue-activity