Skip to content

ctk load table: Add support for MongoDB Change Streams#196

Merged
amotl merged 1 commit intomainfrom
mongodb-cdc
Jul 23, 2024
Merged

ctk load table: Add support for MongoDB Change Streams#196
amotl merged 1 commit intomainfrom
mongodb-cdc

Conversation

@amotl
Copy link
Copy Markdown
Member

@amotl amotl commented Jul 17, 2024 •
edited
Loading

About

Converge MongoDB CDC events into CrateDB table, using the universal ctk load table interface.

Documentation

Preview: https://cratedb-toolkit--196.org.readthedocs.build/io/mongodb/cdc.html

References

Backlog

  • Documentation
  • Software tests

@codecov
Copy link
Copy Markdown

codecov Bot commented Jul 17, 2024 •
edited
Loading

Codecov Report

Attention: Patch coverage is 36.53846% with 33 lines in your changes missing coverage. Please review.

Project coverage is 81.11%. Comparing base (302c44f) to head (b148a12).
Report is 3 commits behind head on main.

Files Patch % Lines
cratedb_toolkit/io/mongodb/cdc.py 38.46% 16 Missing ⚠️
cratedb_toolkit/io/mongodb/api.py 18.18% 9 Missing ⚠️
cratedb_toolkit/api/main.py 33.33% 6 Missing ⚠️
cratedb_toolkit/model.py 66.66% 2 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main     #196      +/-   ##
==========================================
- Coverage   81.76%   81.11%   -0.65%     
==========================================
  Files          79       80       +1     
  Lines        3213     3257      +44     
==========================================
+ Hits         2627     2642      +15     
- Misses        586      615      +29
Flag Coverage Δ
influxdb 30.42% <0.00%> (-0.42%) ⬇️
main 59.22% <7.69%> (-0.75%) ⬇️
mongodb 39.17% <36.53%> (-0.07%) ⬇️
pymongo 41.78% <0.00%> (-0.58%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@amotl amotl mentioned this pull request Jul 17, 2024
Merged
amotl
amotl commented Jul 18, 2024
View reviewed changes
Comment thread doc/io/mongodb/cdc.md Outdated
@amotl amotl requested a review from widmogrod July 19, 2024 15:39
amotl
amotl commented Jul 19, 2024
View reviewed changes
Comment thread pyproject.toml Outdated
@amotl amotl requested review from ckurze and simonprickett July 19, 2024 15:40
@amotl amotl marked this pull request as ready for review July 19, 2024 15:40
amotl
amotl commented Jul 19, 2024
View reviewed changes
Copy link
Copy Markdown
Member Author

@amotl amotl left a comment •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi. We are intending to merge this soon, in this case without software tests, and eventually acknowledged by timeout, in order to slot it into that other PR as early as possible, in order to ship it timely.

It still makes sense to review in retrospective if you have a few spare cycles, because any suggestions will be happily taken into consideration.

amotl
amotl commented Jul 19, 2024
View reviewed changes
Comment thread cratedb_toolkit/io/mongodb/cdc.py Outdated
Comment thread cratedb_toolkit/io/mongodb/cdc.py
Comment thread cratedb_toolkit/model.py Outdated
@amotl amotl force-pushed the mongodb-cdc branch 2 times, most recently from da5fb37 to 12c07b1 Compare July 19, 2024 22:01
This was referenced Jul 20, 2024
Merged
Merged
amotl
amotl commented Jul 20, 2024
View reviewed changes
Comment thread doc/io/mongodb/cdc.md Outdated
Comment thread doc/io/mongodb/cdc.md Outdated
@amotl amotl merged commit 8c1acd4 into main Jul 23, 2024
@amotl amotl deleted the mongodb-cdc branch July 23, 2024 15:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hammerhead hammerhead Awaiting requested review from hammerhead

@hlcianfagna hlcianfagna Awaiting requested review from hlcianfagna

@widmogrod widmogrod Awaiting requested review from widmogrod

@simonprickett simonprickett Awaiting requested review from simonprickett

@ckurze ckurze Awaiting requested review from ckurze

1 more reviewer

@surister surister surister approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@amotl @surister