rpc: compression with libdeflate (#20665) close #17112 This PR replaces the existing single-path compress/gzip middleware in node/rpcstack.go with a dual-path implementation that distinguishes between streaming and non-streaming responses. a) Non-streaming responses (standard JSON-RPC calls such as eth_getBlockByNumber): - are now compressed in one shot using go-libdeflate, a lightweight CGo wrapper around ebiggers/libdeflate. Benchmarks show ~1.75x speedup vs stdlib gzip on ~30 KB JSON payloads (4.6 ms vs 8.0 ms, 122 MB/s vs 70 MB/s); under high concurrency the advantage is larger due to lower CPU usage per request. - Since the compressed size is known before writing, Content-Length is now set to the exact compressed size, avoiding unnecessary Transfer-Encoding: chunked. - Responses smaller than 1 KB are sent uncompressed: at that size the CPU overhead of initialising the compressor outweighs the benefit, and for very small payloads the compressed output can exceed the input size due to gzip framing overhead. b) Streaming responses (e.g. debug_traceTransaction, trace_filter) are detected via http.Flusher: when the RPC handler calls Flush() before writing, the middleware switches to stdlib compress/gzip in streaming mode, compressing trace data incrementally without buffering the full response. Changes: - node/rpcstack.go: new gzipResponseWriter with buffer/stream dual mode; 4 sync.Pool instances for zero-allocation hot path - rpc/http.go: injects http.Flusher into request context via httpFlusherContextKey - rpc/handler.go: streaming methods call flush before writing to activate streaming mode - go.mod: adds github.com/erigontech/go-libdeflate v0.1.0 # 🚀 Performance Benchmarks: Gzip Optimization ---

1. Isolated Compression Benchmarks (libdeflate vs stdlib)

Diff Benchmark (Gzip isolation) Latency Throughput Mem Alloc - BenchmarkStdlibGzip 8.0ms/req 70 MB/s 66KB + BenchmarkLibdeflateGzip 4.6ms/req 122 MB/s 63KB (2x faster) Note: We observed a ~1.75x speedup on a single thread. Under high concurrency, the advantage is even greater due to reduced CPU overhead.

2. eth_getBlockByNumber with txs (Old SW vs Main SW)

Old SW - Results with instability and errors: Diff - [2. 1] qps: 2000 -> [R=97.74% max=9.103s error=503 Service Unavailable] - [3. 3] qps: 2300 -> [R=98.50% max=5.841s error=503 Service Unavailable] - [3. 4] qps: 2300 -> [R=99.80% max=5.634s error=503 Service Unavailable] - [3. 5] qps: 2300 -> [R=98.25% max=5.877s error=503 Service Unavailable] Main SW - Stable results with 100% success rate: Diff + [2. 1] qps: 2000 -> [R=100.00% max=141.22ms] + [2. 5] qps: 2000 -> [R=100.00% max=157.7ms] + [3. 1] qps: 2300 -> [R=100.00% max=217.23ms] + [3. 5] qps: 2300 -> [R=100.00% max=224.174ms]

3. trace_block

| Metric | Old SW | Main SW | Speedup | | :--- | :--- | :--- | :--- | | **Throughput** | 375.9 r/s | **455.0 r/s** | **+21%** | | **p50 Latency** | 114.4 ms | **96.9 ms** | **1.18x** | | **p90 Latency** | 232.0 ms | **181.2 ms** | **1.28x** | | **Mean Latency** | 132.4 ms | **109.5 ms** | **1.21x** |

4. Executes all RPC using http, http-compressed and websockets

./run_all.sh -T http,http_comp,websocket Run tests in parallel on localhost:8545/localhost:8551 Result directory: /home/simon/silkworm/tests/rpc-tests3/integration/results Time: 2026-04-19 08:58:05.257624 Total round_trip time: 3:40:01.359044 Total marshalling time: 0:00:00.063308 Total unmarshalling time: 0:01:40.509521 No of json Diffs: 0 Test time-elapsed: 0:08:37.467460 Available tests: 1436 Available tested api: 112 Number of loop: 1 Number of executed tests: 4152 Number of NOT executed tests: 156 Number of success tests: 4152 Number of failed tests: 0

--------- Co-authored-by: Claude Sonnet 4.6

rpc: evaluate go-libdeflate for non-streaming endpoints #17112

Description

Metadata

Metadata

Assignees

Labels

Type

Fields

Projects

Milestone

Relationships

Development

Issue actions