[improve][io] Upgrade to Debezium 3.2.2 #24712
Conversation
|
@efcasado Please add the following content to your PR description and select a checkbox: |
b4bf7a9 to
f67cdc5
Compare
|
@AlvaroStream, @lhotari - I was able to spend some time on this and it seems all tests are now passing! When you get a chance, I would appreciate your review to make sure I didn’t miss anything. The upgrade from If we move forward with merging this, we will likely need to update some documentation as well (for example, this page). I believe the updated configuration in the integration tests contain most of the information needed to update the docs / examples. |
|
Also, I believe this pull requests renders this other pull request redundant? |
Great work @efcasado ! I really appreciate the effort you have put into solving this. Thank you! |
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## master #24712 +/- ##
============================================
+ Coverage 74.12% 74.21% +0.09%
- Complexity 33068 33108 +40
============================================
Files 1895 1895
Lines 147979 147989 +10
Branches 17137 17137
============================================
+ Hits 109693 109837 +144
+ Misses 29524 29386 -138
- Partials 8762 8766 +4
Flags with carried forward coverage won't be shown. Click here to find out more.
🚀 New features to boost your workflow:
|
lhotari
left a comment
There was a problem hiding this comment.
A few minor comments about dependency versions, other than that LGTM. Outstanding work!
Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com>
Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com>
|
@efcasado I've suggested to cherry-pick this to branch-4.1 so that we could include this in 4.1.1 release. I made the proposal on the dev mailing list: https://lists.apache.org/thread/jrb8n25k266x3o2y4bjfhom6btc21tr9 . |
|
@lhotari, thank you for the review and for helping us push this over the finish line! 🙏 I can certainly help with updating the docs / examples. I will try to at least put a first draft in the coming days. |
Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com>
Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com>
Co-authored-by: Lari Hotari <lhotari@users.noreply.github.com>
Fixes #24333
Motivation
The version of Debezium supported by the project is very old. Debezium 1.9.7 was released in October 25th 2022 and it is lagging two major versions behind the latest stable version, ie. Debezium 3.2.2, which was released in September 4th 2025.
The main motivation behind this change is to bridge this gap and offer the Pulsar community a more recent version of Debezium that can run natively on Pulsar without resorting to external dependencies such as debezium-server.
Modifications
2.x, Debezium renameddatabase.historytoschema.history(see here)2.xchanged a lot of connector property names (see here). Tests have been updated accordingly to reflect this change while preserving the previous semantics as much as possible.mongocommand line is no longer available in newer MongoDB versions and has been replaced bymongosh. Tests have been updated accordingly.database.dbnamehas been replaced with a new option calleddatabase.names(see here). Tests have been updated accordingly.database.encryptproperty in connector configuration to false. (see here).Verifying this change
This change is already covered by existing tests, such as:
Does this pull request potentially affect one of the following parts:
If the box was checked, please highlight the changes
Documentation
docdoc-requireddoc-not-neededdoc-completeMatching PR in forked repository
PR in forked repository: bluelabs-eu#15