Skip to content

Deserialize proto based (mutable key range) change stream return records#35408

Merged
Abacn merged 1 commit intoapache:masterfrom
changliiu:v2-deserialization
Jun 27, 2025
Merged

Deserialize proto based (mutable key range) change stream return records#35408
Abacn merged 1 commit intoapache:masterfrom
changliiu:v2-deserialization

Conversation

@changliiu
Copy link
Copy Markdown
Contributor

@changliiu changliiu commented Jun 24, 2025 •
edited
Loading

Parse proto type change stream return records into connector models.

Including:

  1. DataChangeRecord
  2. HeartbeatRecord
  3. PartitionStartRecord
  4. PartitionEndRecord
  5. PartitionEventRecord

Also:

  1. increase grpc java client lib version so that change stream new records e.g. PartitionStartRecords can be used.
  2. add unit tests for each type or records.

@changliiu changliiu force-pushed the v2-deserialization branch from 10e07a1 to 254762f Compare June 24, 2025 00:14
@changliiu changliiu marked this pull request as ready for review June 24, 2025 00:19
@changliiu changliiu force-pushed the v2-deserialization branch from 254762f to 71ddfe4 Compare June 24, 2025 00:36
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Jun 24, 2025

Checks are failing. Will not request review until checks are succeeding. If you'd like to override that behavior, comment assign set of reviewers

@changliiu changliiu force-pushed the v2-deserialization branch 2 times, most recently from 53081b9 to 458398f Compare June 24, 2025 20:45
dedocibula
dedocibula reviewed Jun 25, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor

@dedocibula dedocibula left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First batch of comments

@changliiu changliiu force-pushed the v2-deserialization branch 2 times, most recently from 684c368 to 52ebaac Compare June 25, 2025 18:16
@changliiu
Parse v2 record.
eff8905 
pick 4d2a5a7 # Fix API surface test (apache#35028)

Deserialize change stream proto based (mutable key range) return types.
@changliiu changliiu force-pushed the v2-deserialization branch from 52ebaac to eff8905 Compare June 25, 2025 18:28
@changliiu
Copy link
Copy Markdown
Contributor Author

changliiu commented Jun 25, 2025

Friendly ping :)

@Abacn Abacn merged commit 3f3f214 into apache:master Jun 27, 2025
22 of 23 checks passed
@changliiu changliiu deleted the v2-deserialization branch June 27, 2025 17:20
Abacn added a commit that referenced this pull request Jun 28, 2025
@Abacn
Revert "Parse v2 record. (#35408)"
6dc3c27 
This reverts commit 3f3f214.
@Abacn Abacn mentioned this pull request Jun 28, 2025
Merged
liferoad pushed a commit that referenced this pull request Jun 28, 2025
@Abacn
Revert "Parse v2 record. (#35408)" (#35469)
f9b7c74 
This reverts commit 3f3f214.
changliiu added a commit to changliiu/beam that referenced this pull request Jun 30, 2025
@changliiu
Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"
7523e30 
This reverts commit f9b7c74.
@changliiu changliiu mentioned this pull request Jun 30, 2025
Closed
changliiu added a commit to changliiu/beam that referenced this pull request Jun 30, 2025
@changliiu
Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"
dbf98da 
This reverts commit f9b7c74.
changliiu added a commit to changliiu/beam that referenced this pull request Jul 1, 2025
@changliiu
Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"
196c0dd 
This reverts commit f9b7c74.
changliiu added a commit to changliiu/beam that referenced this pull request Jul 1, 2025
@changliiu
Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"
f7761e9 
This reverts commit f9b7c74.
changliiu added a commit to changliiu/beam that referenced this pull request Jul 1, 2025
@changliiu
Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"
cb7e67a 
This reverts commit f9b7c74.
changliiu added a commit to changliiu/beam that referenced this pull request Jul 1, 2025
@changliiu
Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"
9262476 
This reverts commit f9b7c74.
changliiu added a commit to changliiu/beam that referenced this pull request Jul 1, 2025
@changliiu
Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"
75ba9cb 
This reverts commit f9b7c74.
changliiu added a commit to changliiu/beam that referenced this pull request Jul 1, 2025
@changliiu
Revert "Revert "Parse v2 record. (apache#35408)" (apache#35469)"
517f589 
This reverts commit f9b7c74.
@changliiu changliiu mentioned this pull request Jul 1, 2025
Merged
jrmccluskey pushed a commit to jrmccluskey/beam that referenced this pull request Jul 1, 2025
@Abacn @jrmccluskey
Revert "Parse v2 record. (apache#35408)" (apache#35469)
101a4e5 
This reverts commit 3f3f214.
Abacn pushed a commit that referenced this pull request Jul 1, 2025
@changliiu
Revert "Revert "Parse v2 record. (#35408)" (#35469)" (#35497)
9487042 
This reverts commit f9b7c74.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

2 more reviewers

@dedocibula dedocibula dedocibula left review comments

@thiagotnunes thiagotnunes thiagotnunes approved these changes

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

build gcp io java spanner

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@changliiu @thiagotnunes @dedocibula @Abacn