fix: improve 503 handling for json resumable uploads#2987
Merged
Conversation
Port `NullPointerException` fix of `JsonResumableSessionPutTask` in 9b4bb82 to `JsonResumableSessionQueryTask`, which has similar code.
BenWhitehead
approved these changes
Mar 14, 2025
1 task
MartijnVisser
added a commit
to MartijnVisser/flink
that referenced
this pull request
Jun 1, 2026
…nk-gs-fs-hadoop The GCS file system bundled google-cloud-storage 2.29.1, which throws a NullPointerException instead of retrying certain GCS 503 Service Unavailable errors during resumable uploads, breaking checkpointing for jobs writing to gs:// via a RecoverableWriter. The upstream fix is in googleapis/java-storage#2987. Bump google-cloud-storage 2.29.1 -> 2.68.0 and the matching grpc artifacts 1.59.1 -> 1.81.0, regenerate the bundled-dependency NOTICE accordingly, add the bundled license file for the newly bundled stax2-api, and update the version links in the GCS filesystem documentation. The newly pulled-in jspecify dependency provides only static-analysis nullness annotations that are not needed at runtime, so it is excluded from the shaded jar like the other annotation-only dependencies (checker-qual, error_prone_annotations, j2objc-annotations). Generated-by: Claude Code (Opus 4.8)
MartijnVisser
added a commit
to apache/flink
that referenced
this pull request
Jun 3, 2026
…nk-gs-fs-hadoop (#28286) * [FLINK-39136][filesystems] Bump google-cloud-storage to 2.68.0 in flink-gs-fs-hadoop The GCS file system bundled google-cloud-storage 2.29.1, which throws a NullPointerException instead of retrying certain GCS 503 Service Unavailable errors during resumable uploads, breaking checkpointing for jobs writing to gs:// via a RecoverableWriter. The upstream fix is in googleapis/java-storage#2987. Bump google-cloud-storage 2.29.1 -> 2.68.0 and the matching grpc artifacts 1.59.1 -> 1.81.0, regenerate the bundled-dependency NOTICE accordingly, add the bundled license file for the newly bundled stax2-api, and update the version links in the GCS filesystem documentation. Add integration tests that run against a real GCS bucket, mirroring the existing S3 filesystem integration tests. They are skipped unless a bucket is configured via the IT_CASE_GCS_BUCKET environment variable; authentication uses Application Default Credentials (GOOGLE_APPLICATION_CREDENTIALS). Generated-by: Claude Code (Opus 4.8)
MartijnVisser
added a commit
to apache/flink
that referenced
this pull request
Jun 3, 2026
…nk-gs-fs-hadoop (#28286) * [FLINK-39136][filesystems] Bump google-cloud-storage to 2.68.0 in flink-gs-fs-hadoop The GCS file system bundled google-cloud-storage 2.29.1, which throws a NullPointerException instead of retrying certain GCS 503 Service Unavailable errors during resumable uploads, breaking checkpointing for jobs writing to gs:// via a RecoverableWriter. The upstream fix is in googleapis/java-storage#2987. Bump google-cloud-storage 2.29.1 -> 2.68.0 and the matching grpc artifacts 1.59.1 -> 1.81.0, regenerate the bundled-dependency NOTICE accordingly, add the bundled license file for the newly bundled stax2-api, and update the version links in the GCS filesystem documentation. Add integration tests that run against a real GCS bucket, mirroring the existing S3 filesystem integration tests. They are skipped unless a bucket is configured via the IT_CASE_GCS_BUCKET environment variable; authentication uses Application Default Credentials (GOOGLE_APPLICATION_CREDENTIALS). Generated-by: Claude Code (Opus 4.8)
MartijnVisser
added a commit
to apache/flink
that referenced
this pull request
Jun 3, 2026
…nk-gs-fs-hadoop (#28286) * [FLINK-39136][filesystems] Bump google-cloud-storage to 2.68.0 in flink-gs-fs-hadoop The GCS file system bundled google-cloud-storage 2.29.1, which throws a NullPointerException instead of retrying certain GCS 503 Service Unavailable errors during resumable uploads, breaking checkpointing for jobs writing to gs:// via a RecoverableWriter. The upstream fix is in googleapis/java-storage#2987. Bump google-cloud-storage 2.29.1 -> 2.68.0 and the matching grpc artifacts 1.59.1 -> 1.81.0, regenerate the bundled-dependency NOTICE accordingly, add the bundled license file for the newly bundled stax2-api, and update the version links in the GCS filesystem documentation. Add integration tests that run against a real GCS bucket, mirroring the existing S3 filesystem integration tests. They are skipped unless a bucket is configured via the IT_CASE_GCS_BUCKET environment variable; authentication uses Application Default Credentials (GOOGLE_APPLICATION_CREDENTIALS). Generated-by: Claude Code (Opus 4.8)
MartijnVisser
added a commit
to MartijnVisser/flink
that referenced
this pull request
Jun 3, 2026
…nk-gs-fs-hadoop (apache#28286) * [FLINK-39136][filesystems] Bump google-cloud-storage to 2.68.0 in flink-gs-fs-hadoop The GCS file system bundled google-cloud-storage 2.29.1, which throws a NullPointerException instead of retrying certain GCS 503 Service Unavailable errors during resumable uploads, breaking checkpointing for jobs writing to gs:// via a RecoverableWriter. The upstream fix is in googleapis/java-storage#2987. Bump google-cloud-storage 2.29.1 -> 2.68.0 and the matching grpc artifacts 1.59.1 -> 1.81.0, regenerate the bundled-dependency NOTICE accordingly, add the bundled license file for the newly bundled stax2-api, and update the version links in the GCS filesystem documentation. Add integration tests that run against a real GCS bucket, mirroring the existing S3 filesystem integration tests. They are skipped unless a bucket is configured via the IT_CASE_GCS_BUCKET environment variable; authentication uses Application Default Credentials (GOOGLE_APPLICATION_CREDENTIALS). Generated-by: Claude Code (Opus 4.8)
MartijnVisser
added a commit
to apache/flink
that referenced
this pull request
Jun 4, 2026
…nk-gs-fs-hadoop (#28286) * [FLINK-39136][filesystems] Bump google-cloud-storage to 2.68.0 in flink-gs-fs-hadoop The GCS file system bundled google-cloud-storage 2.29.1, which throws a NullPointerException instead of retrying certain GCS 503 Service Unavailable errors during resumable uploads, breaking checkpointing for jobs writing to gs:// via a RecoverableWriter. The upstream fix is in googleapis/java-storage#2987. Bump google-cloud-storage 2.29.1 -> 2.68.0 and the matching grpc artifacts 1.59.1 -> 1.81.0, regenerate the bundled-dependency NOTICE accordingly, add the bundled license file for the newly bundled stax2-api, and update the version links in the GCS filesystem documentation. Add integration tests that run against a real GCS bucket, mirroring the existing S3 filesystem integration tests. They are skipped unless a bucket is configured via the IT_CASE_GCS_BUCKET environment variable; authentication uses Application Default Credentials (GOOGLE_APPLICATION_CREDENTIALS). Generated-by: Claude Code (Opus 4.8)
MartijnVisser
added a commit
to apache/flink
that referenced
this pull request
Jun 4, 2026
…nk-gs-fs-hadoop (#28286) * [FLINK-39136][filesystems] Bump google-cloud-storage to 2.68.0 in flink-gs-fs-hadoop The GCS file system bundled google-cloud-storage 2.29.1, which throws a NullPointerException instead of retrying certain GCS 503 Service Unavailable errors during resumable uploads, breaking checkpointing for jobs writing to gs:// via a RecoverableWriter. The upstream fix is in googleapis/java-storage#2987. Bump google-cloud-storage 2.29.1 -> 2.68.0 and the matching grpc artifacts 1.59.1 -> 1.81.0, regenerate the bundled-dependency NOTICE accordingly, add the bundled license file for the newly bundled stax2-api, and update the version links in the GCS filesystem documentation. Add integration tests that run against a real GCS bucket, mirroring the existing S3 filesystem integration tests. They are skipped unless a bucket is configured via the IT_CASE_GCS_BUCKET environment variable; authentication uses Application Default Credentials (GOOGLE_APPLICATION_CREDENTIALS). Generated-by: Claude Code (Opus 4.8)
MartijnVisser
added a commit
to apache/flink
that referenced
this pull request
Jun 4, 2026
…nk-gs-fs-hadoop (#28286) * [FLINK-39136][filesystems] Bump google-cloud-storage to 2.68.0 in flink-gs-fs-hadoop The GCS file system bundled google-cloud-storage 2.29.1, which throws a NullPointerException instead of retrying certain GCS 503 Service Unavailable errors during resumable uploads, breaking checkpointing for jobs writing to gs:// via a RecoverableWriter. The upstream fix is in googleapis/java-storage#2987. Bump google-cloud-storage 2.29.1 -> 2.68.0 and the matching grpc artifacts 1.59.1 -> 1.81.0, regenerate the bundled-dependency NOTICE accordingly, add the bundled license file for the newly bundled stax2-api, and update the version links in the GCS filesystem documentation. Add integration tests that run against a real GCS bucket, mirroring the existing S3 filesystem integration tests. They are skipped unless a bucket is configured via the IT_CASE_GCS_BUCKET environment variable; authentication uses Application Default Credentials (GOOGLE_APPLICATION_CREDENTIALS). Generated-by: Claude Code (Opus 4.8)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Port
NullPointerExceptionfix ofJsonResumableSessionPutTaskin 9b4bb82 toJsonResumableSessionQueryTask, which has similar code.