Cherry pick #9055, #9032 to v1.81.x#9095
Merged
Merged
Conversation
This PR introduces a buffered `io.Reader` that automatically releases its read buffer when empty. To optimize memory usage, the reader defers buffer reallocation until data is available in the underlying `readyreader.Reader` (achieved by calling `ReadOnReady`). In a follow-up PR (grpc#9032), the HTTP/2 framer will be updated to utilize this new buffered reader whenever the underlying reader implements the `readyreader.Reader` interface. The implementation and associated tests are based on the [standard library's](https://cs.opensource.google/go/go/+/refs/tags/go1.26.2:src/bufio/bufio.go;l=35). RELEASE NOTES: N/A --------- Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
The HTTP/2 framer in gRPC uses a `bufio.Reader` with a 32KB buffer by default. When there are a large number of transports, these buffers consume significant memory, even when the transport is idle. reads. This PR replaces the standard `bufio.Reader` with a custom `io.Reader` implementation that uses pooled buffers and releases the buffer once all data is consumed. To defer the re-allocation of the read buffer, the reader calls `ReadOnReady` on the underlying `io.Reader`. For this to work, the underlying `io.Reader` must implement either the `ReadyReader` interface or `syscall.RawConn`. If neither condition is met, the framer gracefully falls back to using the regular `bufio.Reader`. Additional Changes: * The ALTS connection has been refactored to implement the `ReadyReader` interface. * The write buffer pools used by the framer are updated to use the `mem.BufferPool` interface, allowing the pools to be shared across both read and write operations. * Use `syscall.Read` instead of `unix.Read` to avoid triggering the race detector, see comment for details. * Add environment variable protection for the changes to allow fast rollback. In a [real-world benchmark](https://github.com/arjan-bal/custom-go-client-benchmark/tree/retry-dp), where a GCS directpath client downloads a file in a loop, the average "in use" memory falls from 28.3MB to 21.3MB (-24%). Local Benchmarks show no significant difference ``` ❯ go run benchmark/benchresult/main.go streaming-before streaming-after streaming-networkMode_Local-bufConn_false-keepalive_false-benchTime_2m0s-trace_false-latency_0s-kbps_0-MTU_0-maxConcurrentCa lls_120-reqSize_1024B-respSize_1024B-compressor_off-channelz_false-preloader_false-clientReadBufferSize_-1-clientWriteBuffer Size_-1-serverReadBufferSize_-1-serverWriteBufferSize_-1-sleepBetweenRPCs_0s-connections_1-recvBufferPool_simple-sharedWrite Buffer_true Title Before After Percentage TotalOps 29981273 29966908 -0.05% SendOps 0 0 NaN% RecvOps 0 0 NaN% Bytes/op 4971.06 4971.41 0.00% Allocs/op 19.79 19.79 0.00% ReqT/op 2046721570.13 2045740919.47 -0.05% RespT/op 2046721570.13 2045740919.47 -0.05% 50th-Lat 461.523µs 460.906µs -0.13% 90th-Lat 654.435µs 655.327µs 0.14% 99th-Lat 1.225856ms 1.240984ms 1.23% Avg-Lat 478.845µs 479.553µs 0.15% GoVersion go1.25.0 go1.25.0 GrpcVersion 1.81.0-dev 1.81.0-dev ``` RELEASE NOTES: * transport: Pool HTTP/2 framer read buffers to reduce idle memory consumption. Currently limited to Linux for ALTS and non-encrypted transports (TCP, Unix). To disable, set `GRPC_GO_EXPERIMENTAL_HTTP_FRAMER_READ_BUFFER_POOLING=false` and report any issues.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## v1.81.x #9095 +/- ##
===========================================
+ Coverage 82.84% 83.03% +0.18%
===========================================
Files 413 413
Lines 33404 33443 +39
===========================================
+ Hits 27675 27770 +95
- Misses 4242 4245 +3
+ Partials 1487 1428 -59
🚀 New features to boost your workflow:
|
Pranjali-2501
approved these changes
Apr 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Original PRs: #9055, #9032
RELEASE NOTES:
GRPC_GO_EXPERIMENTAL_HTTP_FRAMER_READ_BUFFER_POOLING=falseand report any issues.