Skip to content

custom transformation implementation#2040

Merged
shreyakhajanchi merged 3 commits intoGoogleCloudPlatform:mainfrom
shreyakhajanchi:ct-new-rr
Dec 4, 2024
Merged

custom transformation implementation#2040
shreyakhajanchi merged 3 commits intoGoogleCloudPlatform:mainfrom
shreyakhajanchi:ct-new-rr

Conversation

@shreyakhajanchi
Copy link
Copy Markdown
Contributor

@shreyakhajanchi shreyakhajanchi commented Dec 1, 2024

  1. Added support for custom transformations in the spanner-to-sourcedb reverse replication template
  2. Added support for filtered events, they will be written to a new directory FilterEventsDirectoryName
  3. Added metrics for calculating transformation latency and counting invalid transformation exceptions
  4. Updated unit tests

@codecov
Copy link
Copy Markdown

codecov bot commented Dec 1, 2024

Codecov Report

Attention: Patch coverage is 74.32432% with 19 lines in your changes missing coverage. Please review.

Project coverage is 45.95%. Comparing base (3455819) to head (73bc785).
Report is 9 commits behind head on main.

Files with missing lines Patch % Lines
...v2/templates/transforms/SourceWriterTransform.java 0.00% 8 Missing ⚠️
...cloud/teleport/v2/templates/SpannerToSourceDb.java 0.00% 6 Missing ⚠️
...plates/dbutils/processor/InputRecordProcessor.java 92.00% 1 Missing and 1 partial ⚠️
...leport/v2/templates/transforms/SourceWriterFn.java 87.50% 2 Missing ⚠️
...rt/v2/templates/dbutils/dml/MySQLDMLGenerator.java 94.73% 0 Missing and 1 partial ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##               main    #2040      +/-   ##
============================================
+ Coverage     45.60%   45.95%   +0.34%     
- Complexity     3729     4114     +385     
============================================
  Files           847      850       +3     
  Lines         50188    50759     +571     
  Branches       5275     5338      +63     
============================================
+ Hits          22887    23324     +437     
- Misses        25629    25741     +112     
- Partials       1672     1694      +22     
Components Coverage Δ
spanner-templates 67.55% <74.32%> (+0.48%) ⬆️
spanner-import-export 65.42% <ø> (+1.17%) ⬆️
spanner-live-forward-migration 76.30% <ø> (+0.39%) ⬆️
spanner-live-reverse-replication 77.25% <74.32%> (+0.38%) ⬆️
spanner-bulk-migration 86.87% <ø> (+0.26%) ⬆️
Files with missing lines Coverage Δ
...rt/v2/templates/dbutils/dml/MySQLDMLGenerator.java 97.63% <94.73%> (-0.31%) ⬇️
...plates/dbutils/processor/InputRecordProcessor.java 92.45% <92.00%> (+2.13%) ⬆️
...leport/v2/templates/transforms/SourceWriterFn.java 85.21% <87.50%> (-0.08%) ⬇️
...cloud/teleport/v2/templates/SpannerToSourceDb.java 0.00% <0.00%> (ø)
...v2/templates/transforms/SourceWriterTransform.java 0.00% <0.00%> (ø)

... and 19 files with indirect coverage changes

@shreyakhajanchi shreyakhajanchi marked this pull request as ready for review December 2, 2024 03:37
@shreyakhajanchi shreyakhajanchi requested a review from a team as a code owner December 2, 2024 03:37
@Deep1998
Copy link
Copy Markdown
Contributor

Deep1998 commented Dec 2, 2024

We also need some sort of exhaustive units tests covering all supported response java types and ensure the right DML gets generated, besides end-to-end ITs.

@shreyakhajanchi shreyakhajanchi added the improvement Making existing code better label Dec 3, 2024
@shreyakhajanchi shreyakhajanchi merged commit 09f68d2 into GoogleCloudPlatform:main Dec 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

improvement Making existing code better size/XL

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants