Skip to content

[Do not merge] Beam 2.55.0RC1 validation#1361

Closed
Abacn wants to merge 3 commits intoGoogleCloudPlatform:mainfrom
Abacn:validate_2_55_rc1
Closed

[Do not merge] Beam 2.55.0RC1 validation#1361
Abacn wants to merge 3 commits intoGoogleCloudPlatform:mainfrom
Abacn:validate_2_55_rc1

Conversation

@Abacn
Copy link
Copy Markdown
Contributor

@Abacn Abacn commented Mar 12, 2024

No description provided.

@Abacn
Copy link
Copy Markdown
Contributor Author

Abacn commented Mar 12, 2024

Many templates failed to launch with error

java.lang.NoClassDefFoundError: org/hamcrest/Matcher
...
at org.apache.beam.sdk.options.PipelineOptionsFactory$Cache.validateWellFormed(PipelineOptionsFactory.java:2161)
...
at org.apache.beam.sdk.options.PipelineOptionsFactory.<clinit>(PipelineOptionsFactory.java:543)
...
at com.google.cloud.teleport.v2.templates.BigQueryToBigtable.main(BigQueryToBigtable.java:100)

@Abacn
Copy link
Copy Markdown
Contributor Author

Abacn commented Mar 13, 2024 •
edited
Loading

Spanner tests failed with

Caused by: java.lang.NoSuchMethodError: 'org.apache.beam.sdk.options.ValueProvider org.apache.beam.sdk.io.gcp.spanner.SpannerConfig.getCredentials()'
	at org.apache.beam.sdk.io.gcp.spanner.SpannerAccessor.buildSpannerOptions(SpannerAccessor.java:228)
	at org.apache.beam.sdk.io.gcp.spanner.SpannerAccessor.createAndConnect(SpannerAccessor.java:237)
	at org.apache.beam.sdk.io.gcp.spanner.SpannerAccessor.getOrCreate(SpannerAccessor.java:94)
	at org.apache.beam.sdk.io.gcp.spanner.LocalSpannerAccessor.getOrCreate(LocalSpannerAccessor.java:57)
	at com.google.cloud.teleport.templates.common.SpannerConverters$CreateTransactionFnWithTimestamp.setup(SpannerConverters.java:766

The method was added in apache/beam#30361

SpannerConfig.getCredentials should be present, sounds like maven cache issue on the GHA runner

Update:

Confirmed the method present in 2.55.0 artifact:

in AutoValue_SpannerConfig,

public @Nullable ValueProvider<Credentials> getCredentials() {
    return this.credentials;
  }

@Abacn Abacn mentioned this pull request Mar 13, 2024
Closed
@Abacn Abacn force-pushed the validate_2_55_rc1 branch from 7d3cdab to 31552ca Compare March 13, 2024 16:22
@Abacn Abacn force-pushed the validate_2_55_rc1 branch from 31552ca to 447cfaa Compare March 13, 2024 19:57
@Abacn
Copy link
Copy Markdown
Contributor Author

Abacn commented Mar 14, 2024

Ran twice and different tests failed:

Attempt#1:

CdcToBigQueryChangeApplierPipelineIT.testDebeziumCdcToBigQuerySingleTopic

CdcToBigQueryChangeApplierPipelineIT.testDebeziumCdcToBigQueryMultiTopic

launcher log:

com.google.api.pathtemplate.ValidationException: ProjectTopicName.parse: formattedString not in valid format: Parameter "_deleted-topic_" must be in the form "projects/{project=*}/topics/{topic=*}"

Attempt#2:

JmsToPubsubIT.testJmsToPubsubQueue

launcher log:

code: "Unauthenticated", message: "Request is missing required authentication credential. Expected OAuth 2 access token, login cookie or other valid authentication credential. See https://developers.google.com/identity/sign-in/web/devconsole-project.", details: []

looks like some auth issue

@Abacn Abacn closed this Mar 15, 2024
@Abacn Abacn mentioned this pull request Mar 15, 2024
Closed
@Abacn Abacn mentioned this pull request Apr 10, 2024
Merged
@Abacn Abacn deleted the validate_2_55_rc1 branch January 6, 2026 23:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

size/M

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@Abacn