-
Notifications
You must be signed in to change notification settings - Fork 4.1k
changefeedccl: Reduce backfill impact on foreground latency #84582
Copy link
Copy link
Closed
Labels
A-cdcChange Data CaptureChange Data CaptureC-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)T-cdc
Description
Investigation conducted by @nvanbenschoten shows significant impact on foreground SQL latency when executing backfills; summary here (internal). The solution will be to reduce memory usage to reduce GC (assist) pressure.
- Reduce default size of blocking buffer. Suggested default of 1/8 of current default (125MB).
- kvevent: implement chunked blocking buffer #86421
- changefeedccl: Make kvevent.Event memory efficient #87718
- changefeedccl: reduce size or # of pointers of event struct #84709
- changefeedccl: investigate initial SQL latency spike on initial scan #86469
Jira issue: CRDB-17745
Epic CRDB-24463
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
A-cdcChange Data CaptureChange Data CaptureC-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)T-cdc