Summary

blockvalidation OOMs during testnet catch-up sync around block ~5,000. Captured heap profile at 13.17 GiB RSS shows ~70% of inuse heap in go-bt tx/output decode hot paths. Container restarted (RestartCount=1) seconds after the peak snapshot, consistent with OOM kill.

Environment

• Host: bsva-ovh-teranode-ttn-eu-3
• Teranode: v0.15.1-beta-3 (commit 857a1b18a — includes perf(legacy): stream-decode incoming block messages off the socket #885 legacy streaming fix)
• Container limit: 31.26 GiB
• Workload: testnet catch-up sync, restarted ~2026-05-21 ~13:00 UTC

Evidence

Heap snapshot captured at 2026-05-21T15:07:57Z, RSS = 13.17 GiB at sample time. Container restart timestamp: 2026-05-21T15:09:53Z.

Top of 8.29 GB inuse:

(go-bt/v2@v2.6.3)

RSS trajectory (sawtooth)

Sawtooth (climbing → GC-reclaiming → climbing) → allocation churn, not a leak. But the peak crosses cgroup limit before GC can intervene.

Suspected cause

The Output.ReadFrom / bscript.NewFromBytes / Tx.ReadFrom triad allocates fresh []byte per script per output per tx. On historical mainnet blocks containing very large outputs (OP_RETURN data, large script bodies), the per-decode allocation outpaces GC.

Looks structurally similar to the wire-side issue addressed by #885 (per-payload []byte allocation in go-wire.ReadMessageWithEncodingN), but on the validation side via go-bt. The streaming fix in #885 helps legacy ingestion; this hot path is hit during blockvalidation's parse of subtrees / txs received via gRPC + the catchup pipeline, so #885's win does not apply here.

Potential fix directions

Likely needs work in go-bt (similar to the go-wire follow-ups planned):

1. Pool the per-output script []byte buffers across a block's tx decode.
2. Replace the per-tx toBytesHelper re-serialization (524 MB at peak — needed only for hashing?) with a streaming hash or a reusable buffer.
3. Audit bscript.NewFromBytes for unnecessary copies — if the source buffer is already owned, wrap rather than copy.

Repro

1. Point a Teranode node at testnet, fresh state.
2. Let it catch up through block ~5,000.
3. Watch blockvalidation RSS — expect a sawtooth pattern crossing 13 GiB.

Captured artifacts (local, not attached)

Heap-raw + goroutines + CPU profile for the 13 GiB peak and 14 other snapshots across the sawtooth available on request (probe/eu3-bv-2026-05-21/watcher/ in my local checkout).

Related

• perf(legacy): stream-decode incoming block messages off the socket #885 — legacy streaming block decoder (same class of issue, wire side)