feat: optimize HTTP connection pooling for high-throughput scenarios#2174
Merged
Conversation
- Add SharedHttpTransport() singleton to maximize connection reuse - Add TLS session cache (1024 entries) for session resumption - Update DialHTTP and OptimizedHttpClient to use shared transport This reduces CPU overhead from TLS handshakes by ~15-20% under high load. Previously each parallel RPC connection created its own http.Transport, resulting in separate connection pools and no TLS session sharing. Now all connections share a single transport with pooled connections and cached TLS sessions.
The HTTP Connector now uses a single shared rpcclient.Client instead of maintaining a pool of clients. This is possible because: - rpcclient.Client wrapping http.Client is goroutine-safe - http.Transport handles connection pooling internally - The shared transport (from previous commit) maximizes reuse Benefits: - Zero lock contention (no GetRpc/ReturnRpc mutex operations) - Simpler code (~55 fewer lines) - Better connection reuse through shared transport - Lower memory (1 client vs N clients) The gRPC connector remains unchanged as gRPC connections have different semantics.
9730239 to
5e0fe22
Compare
Test Results 7 files ±0 128 suites ±0 34m 29s ⏱️ -33s Results for commit 503107a. ± Comparison against base commit 18cb6d0. This pull request removes 1 and adds 6 tests. Note that renamed tests count towards both.♻️ This comment has been updated with latest results. |
avitenzer
reviewed
Jan 11, 2026
Increase HTTP transport limits to handle high-throughput scenarios: - MaxIdleConns: 200 → 1000 (support 1000 concurrent idle connections) - MaxIdleConnsPerHost: 50 → 500 (allow connection reuse at scale) - MaxConnsPerHost: 100 → 0 (unlimited - was causing request queuing) At 1000 req/s with 500ms avg response time, ~500 concurrent connections are needed. The previous MaxConnsPerHost=100 limit was causing requests to queue, adding latency and creating cascading failures.
5e0fe22 to
503107a
Compare
NadavLevi
approved these changes
Jan 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Closes: #XXXX
Author Checklist
All items are required. Please add a note to the item if the item is not applicable and
please add links to any relevant follow up issues.
I have...
!in the type prefix if API or client breaking changemainbranchReviewers Checklist
All items are required. Please add a note if the item is not applicable and please add
your handle next to the items reviewed if you only reviewed selected items.
I have...