perf(legacy): stream-decode incoming block messages off the socket#885
Conversation
Register a go-wire SetExternalHandler for the "block" command in services/legacy/peer that calls MsgBlock.Bsvdecode directly against the network reader, replacing the default ReadMessageWithEncodingN path which allocates a fresh []byte the full size of the declared payload before parsing. Heap profile pulled from bsva-ovh-teranode-ttn-eu-4 during a 25.4 GiB RSS peak on 2026-05-16 showed 2.86 GB of inuse heap held by ReadMessageWithEncodingN's payload buffer alone, the second-largest contributor after per-tx scratch in go-wire (probe bundle archived at probe/eu4-rss-peak-2026-05-16). On multi-GB blocks that buffer is allocated once, copied from once, and then released, but it dominates peak inuse during the decode window and aggravates GC pressure. Tradeoffs: - Payload checksum is no longer verified for the "block" command. The external-handler API gives the handler only the payload reader (the header is consumed before dispatch), so the DoubleHash comparison the default path performs cannot run here. This matches the existing extended-format path which already skips checksum verification because the cost outweighs the benefit at multi-GB sizes. - The third return value of ReadMessageWithEncodingN (the raw payload bytes) is now nil for block messages. Only consumer in the legacy path is peer_server.OnBlock -> bsvutil.NewBlockFromBlockAndBytes, which stores the slice but reads it only if block.Bytes() is later called; no legacy netsync code path calls Bytes() on a block coming in over the wire (verified by grep), and bsvutil falls back to re-serializing from MsgBlock if it ever is, so correctness is preserved. The handler wraps the reader in an io.LimitedReader so a malformed varint cannot read past the declared payload boundary and desync the next ReadMessage call, and drains any unread suffix before returning so the stream stays aligned regardless of which error path Bsvdecode takes. Tests added: roundtrip equality vs the buffered path, drain-on-error boundary check, idempotent registration.
|
🤖 Claude Code Review Status: Complete Current Review: No critical issues found. The implementation is well-designed and addresses the documented memory optimization goal effectively. Observations:
Minor note: The trade-off of skipping wire-level checksum verification is well-documented and justified given downstream validation guarantees. |
There was a problem hiding this comment.
Pull request overview
This PR reduces legacy peer memory usage when receiving large block messages by stream-decoding block payloads directly from the socket instead of buffering the full payload first.
Changes:
- Registers a legacy peer streaming handler during server initialization after wire limits are configured.
- Adds a
blockexternal handler that usesio.LimitedReader, drains unread payload bytes, and returns no retained raw payload. - Adds tests for direct handler round-trip decoding, drain-on-error behavior, and repeated registration calls.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.
| File | Description |
|---|---|
services/legacy/Server.go |
Wires the streaming block handler into legacy server startup. |
services/legacy/peer/wire_streaming.go |
Implements and registers the go-wire external block handler. |
services/legacy/peer/wire_streaming_test.go |
Adds tests for streaming decode behavior and registration idempotency. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Reframe checksum doc: not a tradeoff. Downstream HandleBlockDirect enforces PoW, merkle reconstruction, and per-tx parse — any wire- level corruption a DoubleHash would have caught fails one of those. Per Siggi review on bsv-blockchain#885. - Strengthen registration test: assert wire.ReadMessageWithEncodingN takes the streaming path (raw == nil, block decodes correctly) post- registration, and survives repeat Register calls. Replaces the prior no-op idempotent test. Per Copilot review on bsv-blockchain#885.
Benchmark Comparison ReportBaseline: Current: Summary
All benchmark results (sec/op)
Threshold: >10% with p < 0.05 | Generated: 2026-05-18 10:30 UTC |
- Detect undersized payload: io.Copy on a LimitedReader returns nil if the underlying reader EOFs before N reaches 0, so a successful Bsvdecode followed by a truncated stream would silently report success and desync the next ReadMessage. Check lr.N after drain and return an error when the declared payload was not fully consumed. Per Copilot review on bsv-blockchain#885. - Add TestStreamingBlockHandler_ShortStreamReturnsError covering it. - Drop incorrect TCP CRC claim from the rationale comment (TCP uses a 16-bit additive checksum, not a CRC). Reframe what is actually given up: the early-rejection signal, not integrity itself. Per Copilot review on bsv-blockchain#885.
|



Summary
wire.SetExternalHandlerfor theblockcommand inservices/legacy/peerthat callsMsgBlock.Bsvdecodedirectly against the network reader instead of lettingReadMessageWithEncodingNallocate a full-payload[]bytefirst.legacy.Server.Initright afterwire.SetLimits.payload []bytefrom theOnBlocklistener for block messages; the legacy netsync path never reads it.Why
Heap profile pulled from a testnet host during a 25.4 GiB RSS peak on 2026-05-16:
go-wire.(*MsgTx).bsvdecodeWithScratchgo-wire.ReadMessageWithEncodingNpayload buffergo-wire.(*MsgBlock).Bsvdecodeper-tx contiguous scriptsThe 2.86 GB buffer is the lowest-effort win in that profile and it is caller-side — go-wire already exposes
SetExternalHandlerfor exactly this case ("useful, for instance, for very large blocks that may not fit in memory and need to be processed differently" —go-wire/message.go:42).Tradeoffs
blockmessages. The external-handler API gives only the payload reader (header consumed byReadMessageWithEncodingN), so theDoubleHashcomparison the default path performs cannot run here. Matches the existing extended-format path which already skips it at multi-GB sizes.OnBlockbufis nownilfor block messages. Only consumer (peer_server.OnBlock→bsvutil.NewBlockFromBlockAndBytes) stores the slice but reads it only ifblock.Bytes()is later called. No legacy netsync code path callsBytes()on an incoming block (grep -rn 'block.Bytes()' services/legacy/), andbsvutil.Block.Bytes()re-serializes fromMsgBlockif anything ever does.Safety
io.LimitedReadercaps the inner decoder so a malformed varint cannot read past the declared payload boundary and desync the nextReadMessage.sync.Once; safe to call multiple times.Test plan
go build ./services/legacy/...go vet ./services/legacy/...go test -run 'TestStreamingBlockHandler|TestRegisterStreamingBlockHandler' ./services/legacy/peer/— 3 passFollow-ups (go-wire-side, separate PRs)
sync.Poolthe per-block scratch buffer inMsgBlock.Bsvdecode.scripts := make([]byte, totalScriptSize)final-copy buffer with a per-block arena.payload []bytereference insideReadMessageWithEncodingNafterBsvdecodereturns.