-
Notifications
You must be signed in to change notification settings - Fork 4.1k
cdc: support Debezium (via Postgres WAL) #68333
Copy link
Copy link
Closed
Labels
A-cdcChange Data CaptureChange Data CaptureC-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)E-starterMight be suitable for a starter project for new employees or team members.Might be suitable for a starter project for new employees or team members.T-cdc
Metadata
Metadata
Assignees
Labels
A-cdcChange Data CaptureChange Data CaptureC-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)Solution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)E-starterMight be suitable for a starter project for new employees or team members.Might be suitable for a starter project for new employees or team members.T-cdc
Debezium is a platform-agnostic CDC framework, allowing consumers of CDC feeds to plug into any database via a generic adaptor, instead of individually supporting mysql binlogs, mongodb oplogs, etc. This has been requested frequently per @dbist.
Here’s the architecture from the Debezium site:
Approaches
I see two options (caveat that I am not an expert on Debezium):
Update, WAL will likely require functionality that CRDB does not currently offer; WAL is not simply a changefeed “format” or “sink”. Rather, it’s a fully ordered log of events across a whole cluster, of which a changefeed is only a subset. Internal conversation
Epic CRDB-1776
Jira issue: CRDB-8991