Feature/MvP-4632: validator parity + RPC absurd-fee guard#1013
Conversation
|
🤖 Claude Code Review Status: Complete Current Review: This PR implements three distinct features with strong code quality and comprehensive test coverage: T13 - Consensus rejection of unconfirmed parents:
T11 - RPC absurd-fee guard:
T1 - BIP68 documentation:
Documentation Accuracy:
Code Quality:
No issues found. All three tasks are correctly implemented with strong engineering discipline. |
Benchmark Comparison ReportBaseline: Current: Summary
All benchmark results (sec/op)
Threshold: >10% with p < 0.05 | Generated: 2026-06-05 08:46 UTC |
86cda10 to
c9462fd
Compare
T13 — consensus-critical: in-block parents can be wrongly stamped with the unconfirmed sentinelI think T13 will reject valid blocks in consensus mode. There are two independent paths by which an earlier-in-block parent (which is perfectly legal to spend) gets the Worth confirming against svnode parity: in block connection, outputs created by an earlier tx in the same block are added to the block's Path 1 —
|
c9462fd to
4cf90c3
Compare
Re-review — both consensus paths now closed ✅Re-reviewed after the three fix commits. The root cause is fixed at the right layer, not just patched — and the in-block-parent metadata is now wired through the validator service boundary, which Path 2 genuinely required. Reviewed from the PR diff; I did not run Path 1 — fixed as recommended
Path 2 — fixed structurallyThe level-1-only
The concurrency design matches its comments: per-tx goroutines only ever see a pre-filtered private map ( gRPC/HTTP wiring — good defensive posture
Test coverageStrong at the component level: the two regression tests above, plus accumulator semantics (filter, first-writer-wins, Phase-2 delta merge, failed-Phase-2-doesn't-contribute), wire round-trip, fail-closed (malformed length + nil entry), non-symmetric hash byte order, defensive hash copy, content-type parsing, and both HTTP body paths end-to-end. Residual notes (non-blocking)
VerdictThe consensus-split regression is resolved correctly. Good to merge once the end-to-end block-acceptance integration test (note 1) is added — that closes the gap between "the pieces are proven" and "the assembled path is proven." Thanks for the thorough fix. |
ordishs
left a comment
There was a problem hiding this comment.
Re-review — note 1 resolved ✅ — approving
The new commit adds check_block_subtrees_assembled_path_test.go, the end-to-end block-DAG acceptance test from the prior note — and it's well-constructed.
What it does well:
- Drives the real
CheckBlockSubtreespipeline (parse → batch → level-order → per-tx filter → validator) with a purpose-built DAG:G(level 0),P(level 1, spends G),C(level 2, spends P + G skip-level + Gext external). This is exactly the Path 2 shape —C's skip-level edge to grandparentGis what the old level-1-onlybuildParentMetadatadropped. - Pre-flight invariant asserts the level builder actually places C at level 2 before the real assertions run, so a future leveling refactor can't make the test silently vacuous.
- Captures Options at the
ValidateWithOptionsboundary and asserts on every recorded call — both the batch-loop pass and the ordered-retry pass — so a regression that breaks only one pipeline still fails. - Asserts the full invariant set: G has no ParentMetadata; P carries G@candidate; C carries G@candidate across the level skip (the core guard); C does not carry Gext (negative assertion against accidental over-seeding of confirmed external parents, which would corrupt CSV/locktime); and
resp.Blessed == true.
Layering is sound: the validator is mocked here, so this proves "the accumulator threads correct per-tx ParentMetadata through the assembled pipeline." The complementary links — ParentMetadata → no sentinel → correct height, sentinel → BDK rejection, and the wire round-trip — are each covered by their own unit tests. In aggregate the path is fully covered.
Status of prior notes
- Note 1 (E2E block-DAG acceptance test): resolved. ✅
- Note 2 (intern the per-tx
*ParentTxMetadata/ accumulator memory on very large blocks): still open — non-blocking perf follow-up. Fine to defer.
Verdict
The consensus-split regression (both paths), the gRPC/HTTP wiring with fail-closed semantics, and the assembled-path regression guard are all in place. T11 (RPC absurd-fee guard) and T1 (BIP68 doc) remain correct. Approving — the only remaining item is the optional allocation/memory optimization in note 2, which can land in a follow-up.
Reviewed from the diff with symbol existence spot-checked against the tree; I did not run go test locally.
fbbc6c1 to
6ddc7c9
Compare
Consensus-mode parity fix for the case where a tx's parent UTXO exists in the teranode UTXO store but has no recorded BlockHeights (parent not yet confirmed). The validator previously substituted `blockState.Height + 1` for these slots, silently accepting txs that svnode rejects as `bad-txns-unconfirmed-input-in-block`. BDK already enforces the rejection when it sees MEMPOOL_HEIGHT (bdk/core/txvalidator.cpp:779) — the gap was teranode never passing the sentinel. - Mark the unconfirmed-parent fallback at the source with a teranode-internal sentinel `unconfirmedParentHeight = 0xFFFFFFFF` (distinct from MEMPOOL_HEIGHT, impossible as a real height). - Translate at the BDK adapter boundary into MEMPOOL_HEIGHT in consensus mode or the candidate height in policy mode (matching svnode's GetInputScriptBlockHeight at validation.cpp:2668-2675). MEMPOOL_HEIGHT stays confined to the BDK-adapter file. - Guard the SkipScriptValidation early-return so legacy-catchup callers also reject the sentinel in consensus mode — without it, the sentinel would propagate to BIP68 height arithmetic (int32(0xFFFFFFFF) = -1) and clamped MTP lookups, silently accepting. - NullStore.Get now returns a non-empty BlockHeights so tests using the no-op store represent confirmed parents by default; three pre-existing tests that relied on the old silent acceptance updated.
User-protection guard against a fat-fingered fee on the
sendrawtransaction JSON-RPC. This is NOT consensus or policy
enforcement — an absurd-fee tx is a perfectly valid transaction that
miners would accept; the only reason to reject it is operator safety.
Deliberately divergent from bitcoin-sv's placement. svnode enforces
the ceiling inside its validator's TxnValidation pipeline because it
passes a per-call nAbsurdFee from RPC down to the validator
(allowhighfees=true → Amount(0) → disabled for that call). We do not
mirror that placement: svnode's P2P path uses DEFAULT_TRANSACTION_MAXFEE
(0.1 BSV) which doesn't reject anything realistic, so the
validator-level placement gives no real protection beyond the RPC
entry point. Putting it in our validator would pollute the policy
path with a check that exists purely to catch a fat-fingered RPC
call. Operators needing strict svnode parity can later add an
Options.AbsurdFeeCeiling to the validator; the seam stays clean.
- New Policy.MaxRawTxFee (uint64 sats); default 10M = 0.1 BSV,
matching bitcoin-sv DEFAULT_TRANSACTION_MAXFEE = COIN/10. Wired
through settings.NewSettings via getUint64("maxrawtxfee", …) so
the runtime default is non-zero — the struct tag alone is
decorative. Settings-wiring tests pin default, env-override, and
env-zero opt-out.
- handleSendRawTransaction extends the tx (RPC arrives as wire
bytes), computes fee = inputSats - outputSats, and rejects with
`absurdly-high-fee <fee> > <ceiling>` when over the cap.
allowhighfees=true bypasses; MaxRawTxFee=0 disables globally.
- After PreviousOutputsDecorate succeeds, call tx.SetExtended(true)
so the downstream validator does not re-decorate and the gRPC
client serialises extended bytes instead of wire bytes.
- rpcserverhelp.go updated (was: "has no effect").
Tests: 5 handler-level (below ceiling accept; above ceiling reject
with full error-string assertions; allowhighfees bypass; ceiling=0
disables; help-text drift pin) + 3 settings-wiring.
Pt.1 of a 3-part fix for the consensus bug PR review identified in the original T13 commit (f56e5cd): in-block parents can be wrongly stamped with unconfirmedParentHeight, surfacing bad-txns-unconfirmed-input-in-block on blocks svnode accepts. The root cause has two paths, but a prerequisite for fixing either is that Options.ParentMetadata reaches the validator in distributed mode (UseLocalValidator=false, production default). Today it doesn't: the proto has no parent_metadata field and the HTTP fallback sends raw tx bytes plus lossy query-string scalars only. The metadata is silently dropped on every wire crossing. This commit closes that wire gap. No behavioural change in isolation — pt.2 (validator-core preservation across the extend==true fallthrough) and pt.3 (block-scoped accumulator + per-tx filtered metadata in subtreevalidation) need this wire to be in place first. - Proto: new ParentTxMetadata message (bytes parent_hash, uint32 block_height) and repeated parent_metadata on ValidateTransactionRequest. bytes parent_hash carries the chainhash.Hash internal byte order directly — no hex encoding, no display-order conversion, no risk of round-tripping to the wrong key. - Client: buildValidateTxRequest now marshals Options.ParentMetadata via a new parentMetadataToWire helper, with a defensive 32-byte copy per entry so the wire form does not alias the source map. - Server: optionsFromValidateRequest reconstructs the in-memory map via parentMetadataFromWire. Fails closed on malformed entries (nil or wrong-length parent_hash) with a request-level error rather than silently dropping — silent drop would resurrect exactly the consensus rejection shape this PR series is meant to remove. Error is propagated through both the gRPC handler and the HTTP-fallback batch retry path in Client.go. - HTTP fallback: validateTransactionViaHTTP now sends application/x-protobuf with a marshalled ValidateTransactionRequest in the body. Server's handleSingleTx discriminates on Content-Type and falls back to the legacy raw-body + query-string path for non-protobuf callers (backward compatible with any existing non-protobuf client). Tests in services/validator/parent_metadata_api_test.go cover: proto round-trip; empty/missing field; full Client → Server round-trip; non-symmetric-hash byte-order pin (catches any future drift to a hex-encoded string key); defensive-copy invariant; isProtobufContentType across 10 Content-Type variants; HTTP protobuf-body end-to-end; legacy octet-stream backward-compat; fail-closed on nil entry; fail-closed on 5 malformed-length variants.
…3 fix pt.2] Pt.2 of 3 (continues b472731b3 / pt.1 wire prep). Closes Path 1 of the consensus bug PR review identified in the original T13 commit (e03ad5c): the extend==true fallthrough in getUtxoBlockHeightAndExtendForParentTx silently overwrites the candidate height that ParentMetadata supplied for an in-block parent. When extend==true and the parent appears in ParentMetadata, the function correctly stamps the candidate height up front but must also fetch the parent tx body from the UTXO store for input extension. The post-Get height-stamping block was unconditional: in-block parents have empty BlockHeights (Create writes the tx row but the blocks_transactions join row is only added by SetMinedMulti) so the "empty → sentinel" branch fires and clobbers the candidate height with unconfirmedParentHeight. The BDK adapter then translates that to MEMPOOL_HEIGHT and rejects with bad-txns-unconfirmed-input-in-block on a perfectly legitimate block — chain-split risk in consensus mode. This commit adds a cameFromParentMetadata flag and gates the post-Get stamping on !cameFromParentMetadata. The Get call still happens (input extension needs the parent tx body); only the height stamping is skipped for the affected indices. Tests in services/validator/Validator_test.go: - _ParentMetadataPreservedThroughExtendFallthrough — positive pin: candidate height survives extend==true; PreviousTxSatoshis / PreviousTxScript populated from the parent body with deliberately distinctive values (placeholder vs parent value are different, so the assertion cannot pass by accident if extension never ran). - _UnconfirmedSentinelOnlyAppliedWhenNoParentMetadata — negative drift pin, sharp != sentinel assertion to catch any future refactor that reintroduces an unconditional overwrite. - _FallbackWritesUnconfirmedSentinel reinforced with an explicit Options.ParentMetadata == nil assertion so its intent (the no-ParentMetadata code path) is pinned, not implicit. Sanity-checked: temporarily disabling the gate fails the preservation pin at the height assertion (expected 12345, got 0xffffffff), confirming the test catches the regression shape. Pt.3 still needed for the rest of the consensus fix: - Path 2 (skip-level parents) — a tx spending an in-block grandparent is not in ParentMetadata at all because the current builder only fills level-1. - Cross-batch in-block parents in CheckBlockSubtrees — batch 1's parent is not in batch 2's local ParentMetadata. - Per-tx filtered metadata to avoid O(N²) wire bandwidth in distributed validator mode (every per-tx gRPC call currently serialises the full block-scoped accumulator).
…[pt.3] Plumbs ParentMetadata through block-validation so the validator can answer "is this in-block parent confirmed?" for every tx in a block — not just the level-1 children that have a direct UTXO-store hit AND not just the subtree-being-validated, but every parent the candidate block contains. Without this, in-block parents fall through to the UTXO-store BlockHeights path, find them empty (the parent's blocks_transactions row is only written by SetMinedMulti after the block is accepted), the validator stamps unconfirmedParentHeight, and BDK rejects legitimate blocks with bad-txns-unconfirmed-input-in-block. Block-scoped accumulator spans all batches and the ordered-retry phases: - CheckBlockSubtrees hoists a single map[chainhash.Hash]*ParentTxMetadata for the whole block, wrapped in a parentMetadataAccumulator composite (snapshot + delta — see below). - processTransactionsInLevels / processMissingTransactions: per-tx wire bandwidth is bounded by filterParentMetadataForInputs, which returns only the accumulator entries referenced by the tx's input prevouts (nil when there are no matches, so the proto field is absent on the wire). After each level's g.Wait(), successful-tx hashes are merged into the accumulator under first-writer-wins semantics. - Already-known txs that hit the cache/store pre-check (e.g. validated earlier via the peer-announced subtree path) are seeded into the accumulator at the candidate block's height BEFORE the missed==0 early return — closing the gap where a child of such a parent in the same block would otherwise fall through to the UTXO-store BlockHeights path and trigger bad-txns-unconfirmed-input-in-block. - validateMissingSubtreesWithOrderedRetryAccumulated splits Phase 2 (parallel) from Phase 3 (sequential) for the block-validation ordered retry. Phase 2 hands each goroutine a parentMetadataAccumulator with a SHARED read-only snapshot of the live accumulator + a small fresh per-subtree delta — no per-subtree full-copy. After g.Wait(), only fully-successful subtrees' (small) deltas merge into the live accumulator in block-subtree order. Failed-Phase-2 deltas are dropped; those subtrees retry in Phase 3 against the live accumulator sequentially. This preserves consensus determinism: failed-Phase-2 partial writes never enter the accumulator. - ValidateSubtreeInternal stays as a public shim over a private validateSubtreeInternalImpl that accepts the block accumulator; the public signature is unchanged. Tests: - TestProcessTransactionsInLevels/SeedsAlreadyKnownTxsIntoAccumulator — regression for the "parent already in store/cache, child missing in same block" scenario. Sanity-verified by temporarily disabling the seeding code; test correctly fails. - TestParentMetadataAccumulatorAdd — first-writer-wins semantics (subsequent adds for the same hash are no-ops, regardless of whether the prior entry came from snapshot or delta). - TestFilterParentMetadataForInputs — match/miss/empty/nil-tx/duplicate cases plus snapshot+delta-read-merge and delta-wins-over-snapshot. - TestValidateMissingSubtreesWithOrderedRetryAccumulated_Phase2DeltasMergeAfterWait — pre-seeds the live accumulator and asserts every Phase 2 goroutine sees the seed via the shared snapshot AND enters with an empty delta (proves both snapshot sharing and delta isolation). - TestValidateMissingSubtreesWithOrderedRetryAccumulated_FailedPhase2DoesNotContribute — pins that failed-Phase-2 partial contributions are dropped; Phase 3 retry's contributions reach the live accumulator.
… wiring
Adds TestCheckBlockSubtrees_AssembledPath_SkipLevelAndMixedParent, an
end-to-end test that drives the full CheckBlockSubtrees pipeline (parse →
batch → level-order → per-tx filter → validator) against a candidate
block whose DAG exercises both shapes the consensus-split bug surfaced
through:
- a skip-level grandparent dependency (C at level 2 spends both P AND
G directly — the C → G edge crosses one level);
- a mixed in-block + confirmed external parent (C also spends Gext,
which lives in the UTXO store with BlockHeights=[99] and must NOT
enter the block-scoped accumulator).
DAG: G has 2 outputs; P spends G.vout 0 (level 1); C spends P.vout 0 AND
G.vout 1 (level 2 — the G edge is the skip-level case) AND Gext.vout 0
(the confirmed external case). C also referencing P is load-bearing for
the skip-level claim — without that edge the level builder would place
C at level 1 alongside P, and a previous-level-only buildParentMetadata
implementation would still feed G to C correctly, making the test
vacuous.
The test guards that structural invariant directly: a pre-flight check
calls server.selectPrepareTxsPerLevel and asserts maxLevel=2 with
G@level 0, P@level 1, C@level 2. A future refactor of the level builder
or accidental DAG edit that re-collapses the skip-level shape is caught
before the per-tx assertions run.
A recording wrapper around MockValidatorClient captures the resolved
Options per-tx as a slice — BOTH processTransactionsInLevels (batch
loop) and the ordered-retry processMissingTransactions call
ValidateWithOptions for each tx, and the test asserts the accumulator
wiring is correct on BOTH pipelines, not just whichever one happened to
be called last. The wrapper also returns Data with TxInpoints populated
from NewTxInpointsFromTx(tx) so the downstream subtreeMeta serialisation
in validateSubtreeInternalImpl succeeds.
Assertions walk every recorded call:
- G's ParentMetadata empty in every call (no in-block parent).
- P's ParentMetadata[G] present at candidate height in every call.
- C's ParentMetadata[G] present at candidate height in every call —
the regression assertion that pins skip-level survival.
- C's ParentMetadata does NOT contain Gext in any call — guards
against an over-eager refactor that seeds confirmed externals at
candidate height (which would break CSV/locktime).
- response.Blessed == true.
Sanity-verified two ways:
1. Blanked ParentMetadata in each of the two pipelines in turn — test
fails for both, naming the at-fault pipeline by call index.
2. Simulated a previous-level-only buildParentMetadata(txsPerLevel[level-1])
behaviour in processTransactionsInLevels — test fails at the
skip-level C → G assertion, the precise bug shape.
Closes the gap left after the accumulator commit: every component was
tested individually, but nothing drove the assembled orchestration
end-to-end. Guards against a future refactor silently re-opening the
consensus-split path even when component tests stay green.
Also strips reviewer-attribution wording from comments in
check_block_subtrees_test.go and parent_metadata_accumulator_test.go —
documentation describes behavior, not provenance.
Verified: go build, go vet, gofmt clean. Test passes; both sanity
teardowns correctly fail and name the at-fault pipeline.
6ddc7c9 to
07417af
Compare
|
freemans13
left a comment
There was a problem hiding this comment.
Approving after a deep review of all three items (T13 consensus, T11 RPC absurd-fee, T1 doc).
T13 — sound. The consensus-rejection mechanism (internal unconfirmedParentHeight sentinel → BDK MEMPOOL_HEIGHT at the adapter boundary) is the right design over a post-hoc height scan. I traced the one path that could falsely reject a valid block — a mixed-parent child (in-block parent + confirmed parent, arriving non-extended) where the ParentMetadata candidate height could be clobbered by the sentinel in the extend=true fall-through. The PR already handles this correctly:
cameFromParentMetadatagate preserves the candidate height through extension (extension still runs for the tx body).- Full gRPC + HTTP wiring of
parent_metadata(field 10) so a remote validator receives it; HTTP fallback now uses anapplication/x-protobufbody for field parity with gRPC. parentMetadataFromWirefails closed on malformed/nil/wrong-length entries rather than silently dropping them — exactly right, since a dropped entry would re-trigger the falsebad-txns-unconfirmed-input-in-block.- Block-scoped accumulator with cross-batch seeding (
check_block_subtrees.go:991) closes the "parent validated in an earlier batch" hole; deterministic Phase-2 snapshot+delta merge ordering. Backed by substantial new tests.
T11 / T1 — fine as RPC-only / doc-only, well justified in the comments and commit bodies.
Non-blocking follow-ups:
- (LOW) The cross-batch seeding stamps every already-validated in-batch tx with the candidate
blockHeight. Worth a one-line confirmation thattxMetaSlice[i].isSetthere can never be a prior-block-mined tx with realBlockHeights(benign post-Genesis regardless, since BIP68 short-circuits). - (LOW)
sendrawtransactionstores the tx blob before the absurd-fee gate, so a rejected absurd-fee tx leaves a dangling blob; and the default-on guard adds one UTXO round-trip + pre-extends the tx handed to the validator (functionally sound via theIsExtended()guards).
Please confirm CI is green on the current head given the proto was regenerated.
…r with candidate-height resolution Replaces the per-block ParentMetadata accumulator (bsv-blockchain#1013) on the block-validation path with the WithUnconfirmedParentsAtCandidateHeight validator option (introduced for the legacy path in bsv-blockchain#1065), then deletes the accumulator machinery. In-block parents resolve the unconfirmedParentHeight sentinel to the candidate block height inside validateTransaction — identical outcome to the accumulator for genuine in-block parents, without the hot-path seeding, per-tx metadata filtering, per-level mutex merges, or per-tx ParentMetadata wire payload. On a 5.24M-tx chained block this removes ~34% end-to-end overhead. Safety: - PoW (nBits + HasMetTargetDifficulty) now runs before subtree blessing in ValidateBlock, so the fail-open option cannot be triggered by a no-PoW header. - Floater backstop: a parent that is unconfirmed and not in the block fails open at tx level, then block.Valid surfaces ErrBlockIncomplete. When caught up (FSM not CATCHINGBLOCKS/LEGACYSYNCING) that is a genuine floater (SetMinedMulti->MinedSet invariant), so the block is invalidated/rolled back at every acceptance sink (optimistic background, non-optimistic, revalidation worker); while syncing it stays incomplete and is retried, preserving bsv-blockchain#1031. - Recorded parent BlockHeights are untouched (sentinel exact-match only), so fork/revalidation height resolution is unchanged. Deprecates proto field 10 (parent_metadata) via reserved; deletes ParentTxMetadata. Preserves the in_block provenance option (bsv-blockchain#1073, field 11); unconfirmed-parents resolution stays field 12. Closes bsv-blockchain#1066



Final PR for tracking issue #4632.
Summary
bad-txns-unconfirmed-input-in-blockwhen a tx's parent UTXO has no recordedBlockHeights. Uses an internal sentinel at the fallback source + boundary translation toMEMPOOL_HEIGHTinScriptVerifierGoBDK.SkipScriptValidationbypass also guarded.Policy.MaxRawTxFee(default 10M sats = 0.1 BSV);handleSendRawTransactionrejects withabsurdly-high-feewhen over the cap;allowhighfees=truebypasses. RPC-only by design, not validator-level — see commit message.Validator.goexplaining why it's deliberate (post-Genesis no-op + pre-Genesis policy mode unreachable on mainnet).Notes for reviewers
0xFFFFFFFF) distinct fromMEMPOOL_HEIGHT(0x7FFFFFFF). Reason: in mainline block validation the in-block-parent path and the unconfirmed-fallback path both produce the candidate height — a post-hoc height scan can't distinguish them. Sentinel at source is the only safe identifier.NullStore.Getnow returnsBlockHeights: [1]instead of empty. This aligns with NullStore's "everything succeeds" purpose and prevents the new consensus rejection from spuriously firing in tests that don't model parent state.Test plan
go test ./services/validator/... ./services/rpc/... ./settings/... ./stores/utxo/nullstore/...passesgo vet ./...andgo build ./...cleansendrawtransactionwith fee > 0.1 BSV → expectabsurdly-high-feerejection; retry withallowhighfees=true→ expect acceptancebitcoin-cli help sendrawtransaction→allowhighfeesdescription no longer says "has no effect"Out of scope
Other items in #4632 are either already merged in prior sub-PRs, marked skip with documented reasoning, BDK-side (tracked separately), or delegated externally. See the tracking issue for the full status.