Skip to content

Cherry pick #9055, #9032 to v1.81.x#9095

Merged
arjan-bal merged 2 commits into
grpc:v1.81.xfrom
arjan-bal:cherry-pick-9055-v1.81.x
Apr 27, 2026
Merged

Cherry pick #9055, #9032 to v1.81.x#9095
arjan-bal merged 2 commits into
grpc:v1.81.xfrom
arjan-bal:cherry-pick-9055-v1.81.x

Conversation

@arjan-bal

Copy link
Copy Markdown
Contributor

Original PRs: #9055, #9032

RELEASE NOTES:

  • transport: Pool HTTP/2 framer read buffers to reduce idle memory consumption. Currently limited to Linux for ALTS and non-encrypted transports (TCP, Unix). To disable, set GRPC_GO_EXPERIMENTAL_HTTP_FRAMER_READ_BUFFER_POOLING=false and report any issues.

arjan-bal and others added 2 commits April 27, 2026 11:28
This PR introduces a buffered `io.Reader` that automatically releases
its read buffer when empty. To optimize memory usage, the reader defers
buffer reallocation until data is available in the underlying
`readyreader.Reader` (achieved by calling `ReadOnReady`).

In a follow-up PR (grpc#9032), the HTTP/2 framer will be updated to utilize
this new buffered reader whenever the underlying reader implements the
`readyreader.Reader` interface.

The implementation and associated tests are based on the [standard
library's](https://cs.opensource.google/go/go/+/refs/tags/go1.26.2:src/bufio/bufio.go;l=35).

RELEASE NOTES: N/A

---------

Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
The HTTP/2 framer in gRPC uses a `bufio.Reader` with a 32KB buffer by
default. When there are a large number of transports, these buffers
consume significant memory, even when the transport is idle.

reads. This PR replaces the standard `bufio.Reader` with a custom
`io.Reader` implementation that uses pooled buffers and releases the
buffer once all data is consumed.

To defer the re-allocation of the read buffer, the reader calls
`ReadOnReady` on the underlying `io.Reader`. For this to work, the
underlying `io.Reader` must implement either the `ReadyReader` interface
or `syscall.RawConn`. If neither condition is met, the framer gracefully
falls back to using the regular `bufio.Reader`.

Additional Changes:
* The ALTS connection has been refactored to implement the `ReadyReader`
interface.
* The write buffer pools used by the framer are updated to use the
`mem.BufferPool` interface, allowing the pools to be shared across both
read and write operations.
* Use `syscall.Read` instead of `unix.Read` to avoid triggering the race
detector, see comment for details.
* Add environment variable protection for the changes to allow fast
rollback.

In a [real-world
benchmark](https://github.com/arjan-bal/custom-go-client-benchmark/tree/retry-dp),
where a GCS directpath client downloads a file in a loop, the average
"in use" memory falls from 28.3MB to 21.3MB (-24%).

Local Benchmarks show no significant difference
```
❯ go run benchmark/benchresult/main.go streaming-before streaming-after
streaming-networkMode_Local-bufConn_false-keepalive_false-benchTime_2m0s-trace_false-latency_0s-kbps_0-MTU_0-maxConcurrentCa
lls_120-reqSize_1024B-respSize_1024B-compressor_off-channelz_false-preloader_false-clientReadBufferSize_-1-clientWriteBuffer
Size_-1-serverReadBufferSize_-1-serverWriteBufferSize_-1-sleepBetweenRPCs_0s-connections_1-recvBufferPool_simple-sharedWrite
Buffer_true
               Title       Before        After Percentage
            TotalOps     29981273     29966908    -0.05%
             SendOps            0            0      NaN%
             RecvOps            0            0      NaN%
            Bytes/op      4971.06      4971.41     0.00%
           Allocs/op        19.79        19.79     0.00%
             ReqT/op 2046721570.13 2045740919.47    -0.05%
            RespT/op 2046721570.13 2045740919.47    -0.05%
            50th-Lat    461.523µs    460.906µs    -0.13%
            90th-Lat    654.435µs    655.327µs     0.14%
            99th-Lat   1.225856ms   1.240984ms     1.23%
             Avg-Lat    478.845µs    479.553µs     0.15%
           GoVersion     go1.25.0     go1.25.0
         GrpcVersion   1.81.0-dev   1.81.0-dev
```

RELEASE NOTES:
* transport: Pool HTTP/2 framer read buffers to reduce idle memory
consumption. Currently limited to Linux for ALTS and non-encrypted
transports (TCP, Unix). To disable, set
`GRPC_GO_EXPERIMENTAL_HTTP_FRAMER_READ_BUFFER_POOLING=false` and report
any issues.
@arjan-bal arjan-bal added this to the 1.81 Release milestone Apr 27, 2026
@arjan-bal arjan-bal added Type: Performance Performance improvements (CPU, network, memory, etc) Area: Transport Includes HTTP/2 client/server and HTTP server handler transports and advanced transport features. labels Apr 27, 2026
@codecov

codecov Bot commented Apr 27, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.88889% with 10 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.03%. Comparing base (5cba6da) to head (67a60b5).
⚠️ Report is 1 commits behind head on v1.81.x.

Files with missing lines Patch % Lines
credentials/alts/internal/conn/record.go 75.00% 6 Missing ⚠️
internal/transport/readyreader/ready_reader.go 90.90% 2 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           v1.81.x    #9095      +/-   ##
===========================================
+ Coverage    82.84%   83.03%   +0.18%     
===========================================
  Files          413      413              
  Lines        33404    33443      +39     
===========================================
+ Hits         27675    27770      +95     
- Misses        4242     4245       +3     
+ Partials      1487     1428      -59     
Files with missing lines Coverage Δ
internal/envconfig/envconfig.go 100.00% <ø> (ø)
internal/transport/http_util.go 96.58% <100.00%> (+0.02%) ⬆️
internal/transport/readyreader/raw_conn_linux.go 100.00% <100.00%> (ø)
internal/transport/readyreader/ready_reader.go 87.62% <90.90%> (+35.16%) ⬆️
credentials/alts/internal/conn/record.go 83.44% <75.00%> (-0.13%) ⬇️

... and 27 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Pranjali-2501 Pranjali-2501 self-requested a review April 27, 2026 07:30
@arjan-bal arjan-bal merged commit 9183222 into grpc:v1.81.x Apr 27, 2026
17 checks passed
@arjan-bal arjan-bal deleted the cherry-pick-9055-v1.81.x branch April 27, 2026 07:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area: Transport Includes HTTP/2 client/server and HTTP server handler transports and advanced transport features. Type: Performance Performance improvements (CPU, network, memory, etc)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants