Skip to content

cdc: support Debezium (via Postgres WAL) #68333

@shermanCRL

Description

@shermanCRL

Debezium is a platform-agnostic CDC framework, allowing consumers of CDC feeds to plug into any database via a generic adaptor, instead of individually supporting mysql binlogs, mongodb oplogs, etc. This has been requested frequently per @dbist.

Here’s the architecture from the Debezium site:

Approaches

I see two options (caveat that I am not an expert on Debezium):

  1. Establish compatibility with the existing Postgres Debezium connector. I believe this means emitting CDC events as WAL events? This would be a feature of CRDB.
  2. Create a new connector. This would be a plugin/component for Debezium, not in CRDB, and presumably written in Java.

Update, WAL will likely require functionality that CRDB does not currently offer; WAL is not simply a changefeed “format” or “sink”. Rather, it’s a fully ordered log of events across a whole cluster, of which a changefeed is only a subset. Internal conversation

Epic CRDB-1776

Jira issue: CRDB-8991

Metadata

Metadata

Assignees

No one assigned

    Labels

    A-cdcChange Data CaptureC-enhancementSolution expected to add code/behavior + preserve backward-compat (pg compat issues are exception)E-starterMight be suitable for a starter project for new employees or team members.T-cdc

    Type

    No type

    Projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions