Skip to content

fix(bigtable): retry unexpected EOF errors#13157

Merged
bhshkh merged 3 commits into
googleapis:mainfrom
kiwicom:ms/retry-unexpected-eof-upstream-v2
May 30, 2026
Merged

fix(bigtable): retry unexpected EOF errors#13157
bhshkh merged 3 commits into
googleapis:mainfrom
kiwicom:ms/retry-unexpected-eof-upstream-v2

Conversation

@martin-sucha

Copy link
Copy Markdown
Contributor

This has been previously discussed at #11276.
I am resubmitting the change again, since we still see the UnexpectedEOF and we are running with this patch in production to retry it.

Apparently there is a flaky test that shows the same issue as well: #10909

There are more cases that use the comparison of the error message now, and the grpc-go has some places that return a gRPC status error with the error message of the io.UnexpectedEOF.
So I kept the handling by error message text instead of comparing directly to io.UnexpectedEOF.

@martin-sucha martin-sucha requested review from a team October 16, 2025 13:34
@product-auto-label product-auto-label Bot added the api: bigtable Issues related to the Bigtable API. label Oct 16, 2025
@bhshkh bhshkh enabled auto-merge (squash) April 29, 2026 05:48
@bhshkh

bhshkh commented Apr 29, 2026

Copy link
Copy Markdown
Contributor

Please merge main / rebase to main

We see the unexpected EOF error message in logs for large
scans.

Apparently there is a flaky test that shows the same issue as well:
googleapis#10909

Keeping the comparison by message text, because grpc-go returns a gRPC
status error with the same message as io.UnexpectedEOF in some places.

There are more cases that use the comparison of the error message now,
and the grpc-go has some places
(see https://github.com/grpc/grpc-go/blob/ae62635cb0e21ae37e7ae5f0f18aaac5e8139834/rpc_util.go#L1033)
that return a gRPC status error with the error message of the io.UnexpectedEOF.
So I kept the handling by error message text instead of comparing directly to io.UnexpectedEOF.
auto-merge was automatically disabled April 29, 2026 13:18

Head branch was pushed to by a user without write access

@martin-sucha martin-sucha force-pushed the ms/retry-unexpected-eof-upstream-v2 branch from 3492fb7 to 123001a Compare April 29, 2026 13:18
@martin-sucha martin-sucha requested review from a team as code owners April 29, 2026 13:18
@martin-sucha

Copy link
Copy Markdown
Contributor Author

@bhshkh rebased

@bhshkh bhshkh enabled auto-merge (squash) May 1, 2026 17:31
@martin-sucha

Copy link
Copy Markdown
Contributor Author

@bhshkh it seems that pull request approval is needed here before the merge succeeds.

@bhshkh

bhshkh commented May 16, 2026

Copy link
Copy Markdown
Contributor

@martin-sucha
Please push an empty commit:

git commit --allow-empty -m "re-run multi-approvers check"

@martin-sucha

Copy link
Copy Markdown
Contributor Author

@bhshkh I've pushed the empty commit.

The multi-approvers action failed with:

Multi-approvers action failed: this pull request has 1 of 2 required internal approvals.

@bhshkh

bhshkh commented May 26, 2026

Copy link
Copy Markdown
Contributor

Fixes b/369728110

@martin-sucha

Copy link
Copy Markdown
Contributor Author

@hongalex Thanks for the approval!

@bhshkh I've pushed another empty commit and now the multi-approvers check suceeded. Hope that helps.

@bhshkh bhshkh added kokoro:force-run Add this label to force Kokoro to re-run the tests. kokoro:run Add this label to force Kokoro to re-run the tests. labels May 30, 2026
@kokoro-team kokoro-team removed kokoro:run Add this label to force Kokoro to re-run the tests. kokoro:force-run Add this label to force Kokoro to re-run the tests. labels May 30, 2026
@bhshkh bhshkh merged commit 2b4ac42 into googleapis:main May 30, 2026
17 checks passed
bhshkh added a commit that referenced this pull request Jun 10, 2026
PR created by the Librarian CLI to initialize a release. Merging this PR
will auto trigger a release.

Librarian Version: v0.20.0
Language Image:
us-central1-docker.pkg.dev/cloud-sdk-librarian-prod/images-prod/librarian-go@sha256:b7e38e80d677dc6b7266896bb99a33fa62348d3e39d7b8f2f6423e7dc4b35a60
<details><summary>bigtable: v1.49.0</summary>

##
[v1.49.0](bigtable/v1.48.0...bigtable/v1.49.0)
(2026-06-10)

### Features

* enable new auth library and remove async refresh (#19943)
([0d3697b](0d3697b2))

* add traffic diverter and table shim for bigtable (#14633)
([2f3c918](2f3c9184))

* add virtual RPC descriptors (#14648)
([5eedda1](5eedda18))

* add opensession descriptors for bigtable (#14647)
([8968f4f](8968f4fd))

* update API sources and regenerate (#14701)
([a9b7921](a9b79215))

### Bug Fixes

* retry unexpected EOF errors (#13157)
([2b4ac42](2b4ac429))

* disable dynamic channel pool by default and raise default pool size to
10 (#19945)
([5ddf8d4](5ddf8d46))

* raise default connection-recycler MaxAge to 7 days (#19940)
([a7959ef](a7959efc))

</details>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

api: bigtable Issues related to the Bigtable API.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants