Skip to content

fix(legacy): resolve in-block parents at candidate height during legacy subtree validation#1065

Merged
oskarszoon merged 9 commits into
bsv-blockchain:mainfrom
oskarszoon:fix/legacy-testnet-unconfirmed
Jun 11, 2026
Merged

fix(legacy): resolve in-block parents at candidate height during legacy subtree validation#1065
oskarszoon merged 9 commits into
bsv-blockchain:mainfrom
oskarszoon:fix/legacy-testnet-unconfirmed

Conversation

@oskarszoon

@oskarszoon oskarszoon commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Problem

Legacy sync wedges on testnet at block 1730003 with:

bad-txns-unconfirmed-input-in-block

1730003 is the first block past the highest testnet checkpoint (1730000) that contains an in-block tx chain (0dc2a467… spends an output of 35b7b758… in the same block). Past the checkpoint, netsync leaves quickValidationMode and validates subtrees through CheckSubtreeFromBlock.

A tx spending a same-block parent finds that parent in the UTXO store with empty BlockHeightsSetMinedMulti only runs after block acceptance — so the validator stamps the unconfirmedParentHeight sentinel. In consensus mode the sentinel translates to MEMPOOL_HEIGHT at the BDK boundary and BDK rejects it (txvalidator.cpp:779): BDK needs the real UTXO height to select per-input script-era flags (pre/post-Genesis) and refuses to guess in consensus context. Every post-checkpoint block with an in-block chain fails the same way; sync is stuck at 1730002.

Fix

New validator option WithUnconfirmedParentsAtCandidateHeight(true), set only by the legacy branch of checkSubtreeFromBlock. When set, validateTransaction resolves the sentinel to the candidate block height before any consumer sees it — both the BDK era-flag selection and the BIP68/MTP lookups. On this path the candidate height is the parent's true height (the parent is in the block being validated), so the substitution is exact, including cross-subtree parents, and needs no shared state — safe with multiple subtreevalidation instances behind load balancing.

Plumbing: one optional bool on ValidateTransactionRequest (field 11) so the option survives the gRPC hop to a remote validator; client build + server reconstruction mapped and pinned by a round-trip test.

Consensus safety

This is fail-open at tx level, deliberately scoped:

  • A parent that is unconfirmed and NOT in the block (mempool floater) is no longer rejected at tx level. The membership backstop is block validation's checkParentsExistOnChain (model/Block.go), which legacy netsync runs on every block before acceptance — an unmined external parent fails the block there.
  • The flag is only sound when the tx comes from a locally-held, PoW-checked block, the block-level parent-membership check runs before acceptance, and block assembly is disabled (all true during legacy sync). The option doc and proto comment state this contract; the peer-facing branch does not set it, pinned by test.
  • An earlier revision carried explicit in-block parent txid lists over gRPC. Dropped: the list is unbounded (a heavily-chained 1M-leaf subtree approaches ~32MB of hashes, past gRPC message limits), and the membership knowledge it encodes is already enforced block-level. The bool conveys the same trust decision at zero marginal payload.

Tests

  • TestValidate_ConsensusRejectsUnconfirmedParent (existing) and TestValidate_ConsensusAcceptsUnconfirmedParentAtCandidateHeight (new) — real TxValidator + GoBDK, same fixtures: sentinel rejects without the option, validates with it. The pair pins the consensus outcome in both directions.
  • TestCheckSubtreeFromBlockLegacyUnconfirmedParents — handler-level: legacy branch sets the flag on per-tx validation Options; cross-subtree case (parent subtree 0, child subtree 1, two sequential calls) validates; peer-facing path does NOT set the flag.
  • TestUnconfirmedParentsAtCandidateHeight_WireRoundTrip — flag survives client build → proto marshal → server reconstruction; default stays false.
  • go test -race green on validator, subtreevalidation, legacy/netsync, blockvalidation. vet / staticcheck / golangci-lint / gosec: no new findings.

Deploy note

Rolling-upgrade skew: new subtreevalidation + old validator service → unknown proto field ignored → old (wedging) behaviour until the validator is upgraded too. Single-binary deployments (quickstart compose) are unaffected. Deploy services together.

…cy subtree validation

Legacy sync wedges on testnet at block 1730003 — the first block past the
highest checkpoint (1730000) containing an in-block tx chain. Past the
checkpoint, netsync leaves quickValidationMode and validates subtrees via
CheckSubtreeFromBlock, which called ValidateSubtreeInternal with a nil
block accumulator. A child spending a same-block parent then resolved the
parent through the UTXO-store fallback, whose BlockHeights stay empty
until SetMinedMulti runs after block acceptance, so the validator stamped
the unconfirmedParentHeight sentinel. In consensus mode the sentinel
translates to MEMPOOL_HEIGHT at the BDK boundary and BDK rejects the
block with bad-txns-unconfirmed-input-in-block.

Pass the knowledge the caller already has: netsync collects, per subtree,
the input-parent txids that live in the same block (from the block txMap)
and sends them as a new in_block_parent_hashes field on
CheckSubtreeFromBlockRequest. The server seeds a block-scoped
parentMetadataAccumulator from the list at the candidate block height, so
children resolve in-block parents through ParentMetadata instead of the
sentinel.

Stateless per request: no server-side per-block state, safe with multiple
subtreevalidation instances behind load balancing, and covers
cross-subtree parents (the child's request carries the parent hash
regardless of which instance validated the parent's subtree). An empty
list preserves the prior behaviour exactly, so the proto change is
backward compatible.
@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

🤖 Claude Code Review

Status: Complete

Current Review: No blocking issues found.

The final design is narrow and well-reasoned: a single validator option (WithUnconfirmedParentsAtCandidateHeight) that resolves the unconfirmedParentHeight sentinel to the candidate block height before BDK era-flag selection and the BIP68/MTP lookups, set unconditionally only by the legacy checkSubtreeFromBlock branch (BaseUrl "legacy"). Wiring (option → *bool ptr helper → proto field 12 → server reconstruction) is consistent, and the proto field renumber (11 in_block from #1073, 12 here) is internally consistent in both .proto and .pb.go.

Things I verified:

  • The earlier in_block_parent_hashes gRPC-list approach is fully revertedCheckSubtreeFromBlock signatures restored, helper/test files removed, no dead code left behind.
  • Double-substitution is harmless: validateTransaction rewrites sentinels to blockHeight before phase 1, so the BDK adapters consensus-branch substituteUnconfirmedHeights` finds none and passes through. The resolved slice flows into phase 2 (BIP68/MTP) consistently.
  • Consensus fail-open is sound in practice: this path runs only past the highest checkpoint (structurally post-Genesis on both networks), so the era-flag boundary cannot be crossed by the height substitution; a genuine mempool floater is backstopped by block validations checkParentsExistOnChain (BlockIncompleteError`, block never accepted).
  • Test coverage is strong: real-BDK accept/reject pair, cross-subtree handler test, wire round-trip, and per-FSM-state assertions (flag on in every state; assembly-off only during LEGACYSYNCING/CATCHINGBLOCKS).

Main operational risk (already documented by the author, restating for visibility): wire skew. The beta deployment sent this flag as field 11, which post-merge is in_block — a beta subtreevalidation talking to a post-merge validator would silently set the wrong option. Beta deployments must be replaced wholesale, and subtreevalidation+validator upgraded together; a new sender against an old validator simply keeps the wedge until the validator is upgraded.

Existing inline threads (handle_block.go:536, Server.go:853) reference superseded revisions and have author replies explaining the evolution; no action needed.

@github-actions

github-actions Bot commented Jun 10, 2026

Copy link
Copy Markdown
Contributor

Benchmark Comparison Report

Baseline: main (unknown)

Current: PR-1065 (5941b3a)

Summary

  • Regressions: 0
  • Improvements: 0
  • Unchanged: 132
  • Significance level: p < 0.05
All benchmark results (sec/op)
Benchmark Baseline Current Change p-value
_NewBlockFromBytes-4 1.759µ 1.604µ ~ 0.700
SplitSyncedParentMap_SetIfNotExists/256_buckets-4 71.12n 71.13n ~ 0.700
SplitSyncedParentMap_SetIfNotExists/16_buckets-4 71.16n 71.84n ~ 0.400
SplitSyncedParentMap_SetIfNotExists/1_bucket-4 71.13n 71.05n ~ 0.700
SplitSyncedParentMap_ConcurrentSetIfNotExists/256_buckets... 32.73n 32.46n ~ 0.400
SplitSyncedParentMap_ConcurrentSetIfNotExists/16_buckets_... 55.42n 55.07n ~ 0.100
SplitSyncedParentMap_ConcurrentSetIfNotExists/1_bucket_pa... 128.4n 129.8n ~ 0.200
MiningCandidate_Stringify_Short-4 219.1n 220.1n ~ 0.700
MiningCandidate_Stringify_Long-4 1.633µ 1.630µ ~ 0.600
MiningSolution_Stringify-4 858.1n 859.3n ~ 1.000
BlockInfo_MarshalJSON-4 1.730µ 1.721µ ~ 0.700
NewFromBytes-4 131.1n 129.7n ~ 0.100
AddTxBatchColumnar_Validation-4 2.495µ 2.499µ ~ 0.700
OffsetValidationLoop-4 635.9n 639.8n ~ 0.400
Mine_EasyDifficulty-4 65.24µ 64.76µ ~ 0.200
Mine_WithAddress-4 6.832µ 6.813µ ~ 0.600
BlockAssembler_AddTx-4 0.01967n 0.01858n ~ 1.000
AddNode-4 8.370 8.558 ~ 0.400
AddNodeWithMap-4 8.517 8.704 ~ 0.100
DirectSubtreeAdd/4_per_subtree-4 76.41n 74.80n ~ 0.400
DirectSubtreeAdd/64_per_subtree-4 40.88n 41.17n ~ 0.200
DirectSubtreeAdd/256_per_subtree-4 40.63n 40.05n ~ 0.700
DirectSubtreeAdd/1024_per_subtree-4 38.40n 38.47n ~ 0.400
DirectSubtreeAdd/2048_per_subtree-4 37.96n 38.21n ~ 0.400
SubtreeProcessorAdd/4_per_subtree-4 343.0n 344.8n ~ 1.000
SubtreeProcessorAdd/64_per_subtree-4 322.6n 320.4n ~ 1.000
SubtreeProcessorAdd/256_per_subtree-4 318.6n 321.8n ~ 0.700
SubtreeProcessorAdd/1024_per_subtree-4 327.8n 325.2n ~ 0.700
SubtreeProcessorAdd/2048_per_subtree-4 327.1n 325.7n ~ 1.000
SubtreeProcessorRotate/4_per_subtree-4 323.2n 323.8n ~ 0.400
SubtreeProcessorRotate/64_per_subtree-4 326.4n 324.4n ~ 0.400
SubtreeProcessorRotate/256_per_subtree-4 329.7n 335.3n ~ 0.100
SubtreeProcessorRotate/1024_per_subtree-4 331.2n 337.7n ~ 0.200
SubtreeNodeAddOnly/4_per_subtree-4 87.71n 86.87n ~ 0.100
SubtreeNodeAddOnly/64_per_subtree-4 64.35n 63.93n ~ 0.100
SubtreeNodeAddOnly/256_per_subtree-4 63.36n 63.16n ~ 1.000
SubtreeNodeAddOnly/1024_per_subtree-4 63.10n 62.89n ~ 0.100
SubtreeCreationOnly/4_per_subtree-4 131.1n 133.7n ~ 0.400
SubtreeCreationOnly/64_per_subtree-4 565.2n 524.9n ~ 0.100
SubtreeCreationOnly/256_per_subtree-4 1.965µ 1.964µ ~ 1.000
SubtreeCreationOnly/1024_per_subtree-4 6.118µ 5.898µ ~ 0.100
SubtreeCreationOnly/2048_per_subtree-4 10.65µ 10.70µ ~ 1.000
SubtreeProcessorOverheadBreakdown/64_per_subtree-4 336.1n 334.7n ~ 0.700
SubtreeProcessorOverheadBreakdown/1024_per_subtree-4 344.6n 336.0n ~ 0.100
ParallelGetAndSetIfNotExists/1k_nodes-4 15.49m 15.84m ~ 0.700
ParallelGetAndSetIfNotExists/10k_nodes-4 18.31m 21.69m ~ 0.100
ParallelGetAndSetIfNotExists/50k_nodes-4 20.87m 18.11m ~ 0.100
ParallelGetAndSetIfNotExists/100k_nodes-4 21.29m 20.57m ~ 0.200
SequentialGetAndSetIfNotExists/1k_nodes-4 14.56m 15.41m ~ 0.400
SequentialGetAndSetIfNotExists/10k_nodes-4 16.40m 18.66m ~ 0.200
SequentialGetAndSetIfNotExists/50k_nodes-4 21.13m 20.25m ~ 0.400
SequentialGetAndSetIfNotExists/100k_nodes-4 28.65m 28.10m ~ 0.700
ProcessOwnBlockSubtreeNodesParallel/1k_nodes-4 11.64m 11.69m ~ 0.700
ProcessOwnBlockSubtreeNodesParallel/10k_nodes-4 18.66m 18.06m ~ 1.000
ProcessOwnBlockSubtreeNodesParallel/100k_nodes-4 22.99m 22.94m ~ 1.000
ProcessOwnBlockSubtreeNodesSequential/1k_nodes-4 12.02m 14.35m ~ 0.400
ProcessOwnBlockSubtreeNodesSequential/10k_nodes-4 18.37m 17.69m ~ 1.000
ProcessOwnBlockSubtreeNodesSequential/100k_nodes-4 51.18m 53.86m ~ 0.100
DiskTxMap_SetIfNotExists-4 4.061µ 4.088µ ~ 1.000
DiskTxMap_SetIfNotExists_Parallel-4 3.778µ 3.809µ ~ 1.000
DiskTxMap_ExistenceOnly-4 474.3n 339.3n ~ 0.400
Queue-4 199.7n 200.4n ~ 0.700
AtomicPointer-4 8.139n 8.157n ~ 0.200
TxMapSetIfNotExists-4 53.19n 54.78n ~ 0.400
TxMapSetIfNotExistsDuplicate-4 48.39n 48.49n ~ 0.200
ChannelSendReceive-4 652.4n 653.4n ~ 0.700
CalcBlockWork-4 485.5n 470.5n ~ 0.100
CalculateWork-4 652.3n 647.7n ~ 0.700
CheckOldBlockIDs/on-chain-prefetch/1000-4 59.34µ 60.79µ ~ 0.100
CheckOldBlockIDs/off-chain-prefetch/1000-4 55.91µ 50.38µ ~ 0.700
CheckOldBlockIDs/on-chain-prefetch/10000-4 422.3µ 424.1µ ~ 0.100
CheckOldBlockIDs/off-chain-prefetch/10000-4 345.8µ 348.4µ ~ 0.700
BuildBlockLocatorString_Helpers/Size_10-4 1.351µ 1.357µ ~ 0.700
BuildBlockLocatorString_Helpers/Size_100-4 12.92µ 12.88µ ~ 1.000
BuildBlockLocatorString_Helpers/Size_1000-4 126.2µ 128.0µ ~ 0.100
CatchupWithHeaderCache-4 104.5m 104.5m ~ 0.400
_BufferPoolAllocation/16KB-4 4.195µ 4.120µ ~ 0.700
_BufferPoolAllocation/32KB-4 9.185µ 11.841µ ~ 0.200
_BufferPoolAllocation/64KB-4 20.03µ 19.64µ ~ 0.700
_BufferPoolAllocation/128KB-4 37.79µ 39.43µ ~ 0.400
_BufferPoolAllocation/512KB-4 137.2µ 130.9µ ~ 0.100
_BufferPoolConcurrent/32KB-4 24.52µ 20.69µ ~ 0.100
_BufferPoolConcurrent/64KB-4 36.18µ 32.88µ ~ 0.200
_BufferPoolConcurrent/512KB-4 154.2µ 154.6µ ~ 1.000
_SubtreeDeserializationWithBufferSizes/16KB-4 687.3µ 648.6µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/32KB-4 699.0µ 645.4µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/64KB-4 687.6µ 624.5µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/128KB-4 687.3µ 644.3µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/512KB-4 690.2µ 647.1µ ~ 0.100
_SubtreeDataDeserializationWithBufferSizes/16KB-4 37.30m 37.61m ~ 0.100
_SubtreeDataDeserializationWithBufferSizes/32KB-4 37.47m 37.40m ~ 1.000
_SubtreeDataDeserializationWithBufferSizes/64KB-4 37.16m 37.71m ~ 0.200
_SubtreeDataDeserializationWithBufferSizes/128KB-4 37.22m 37.77m ~ 0.700
_SubtreeDataDeserializationWithBufferSizes/512KB-4 36.84m 37.51m ~ 0.100
_PooledVsNonPooled/Pooled-4 742.3n 744.9n ~ 0.400
_PooledVsNonPooled/NonPooled-4 9.234µ 9.442µ ~ 0.700
_MemoryFootprint/Current_512KB_32concurrent-4 7.052µ 7.054µ ~ 1.000
_MemoryFootprint/Proposed_32KB_32concurrent-4 10.63µ 10.04µ ~ 0.100
_MemoryFootprint/Alternative_64KB_32concurrent-4 10.788µ 9.452µ ~ 0.100
_prepareTxsPerLevel-4 417.8m 412.7m ~ 0.200
_prepareTxsPerLevelOrdered-4 3.749m 3.827m ~ 0.700
_prepareTxsPerLevel_Comparison/Original-4 415.0m 417.1m ~ 0.400
_prepareTxsPerLevel_Comparison/Optimized-4 3.875m 3.568m ~ 0.200
SubtreeSizes/10k_tx_4_per_subtree-4 1.267m 1.265m ~ 1.000
SubtreeSizes/10k_tx_16_per_subtree-4 297.3µ 297.0µ ~ 1.000
SubtreeSizes/10k_tx_64_per_subtree-4 70.07µ 72.13µ ~ 0.200
SubtreeSizes/10k_tx_256_per_subtree-4 17.70µ 17.72µ ~ 0.700
SubtreeSizes/10k_tx_512_per_subtree-4 8.744µ 8.859µ ~ 0.100
SubtreeSizes/10k_tx_1024_per_subtree-4 4.347µ 4.433µ ~ 0.400
SubtreeSizes/10k_tx_2k_per_subtree-4 2.187µ 2.155µ ~ 0.100
BlockSizeScaling/10k_tx_64_per_subtree-4 69.88µ 69.65µ ~ 0.400
BlockSizeScaling/10k_tx_256_per_subtree-4 17.59µ 17.52µ ~ 0.200
BlockSizeScaling/10k_tx_1024_per_subtree-4 4.363µ 4.357µ ~ 0.400
BlockSizeScaling/50k_tx_64_per_subtree-4 368.6µ 374.0µ ~ 0.200
BlockSizeScaling/50k_tx_256_per_subtree-4 87.70µ 87.68µ ~ 0.700
BlockSizeScaling/50k_tx_1024_per_subtree-4 21.33µ 21.45µ ~ 0.100
SubtreeAllocations/small_subtrees_exists_check-4 149.4µ 150.7µ ~ 0.200
SubtreeAllocations/small_subtrees_data_fetch-4 157.6µ 158.4µ ~ 1.000
SubtreeAllocations/small_subtrees_full_validation-4 306.6µ 310.1µ ~ 0.400
SubtreeAllocations/medium_subtrees_exists_check-4 8.844µ 8.840µ ~ 0.700
SubtreeAllocations/medium_subtrees_data_fetch-4 9.187µ 9.294µ ~ 0.200
SubtreeAllocations/medium_subtrees_full_validation-4 17.52µ 17.56µ ~ 1.000
SubtreeAllocations/large_subtrees_exists_check-4 2.085µ 2.082µ ~ 1.000
SubtreeAllocations/large_subtrees_data_fetch-4 2.202µ 2.231µ ~ 0.700
SubtreeAllocations/large_subtrees_full_validation-4 4.357µ 4.340µ ~ 1.000
StoreBlock_Sequential/BelowCSVHeight-4 307.9µ 315.1µ ~ 0.200
StoreBlock_Sequential/AboveCSVHeight-4 319.9µ 309.0µ ~ 0.200
GetUtxoHashes-4 300.1n 290.4n ~ 0.400
GetUtxoHashes_ManyOutputs-4 50.32µ 48.26µ ~ 0.700
_NewMetaDataFromBytes-4 230.4n 216.0n ~ 0.100
_Bytes-4 402.0n 403.9n ~ 0.100
_MetaBytes-4 138.4n 139.7n ~ 0.700

Threshold: >10% with p < 0.05 | Generated: 2026-06-11 11:09 UTC

…rshalling

Sonar flagged the new lines in Server.go (legacy gRPC handler branch) and
Client.go (hash marshalling loop) as uncovered. Adds a handler-level test
driving CheckSubtreeFromBlock end-to-end through the legacy branch —
request hint reaching per-tx validation Options, malformed hash rejected
as invalid argument — and a client test asserting the in-block parent
hashes round-trip into the request.
ctnguyen

This comment was marked as outdated.

… cross-subtree e2e

Addresses ctnguyen's review on bsv-blockchain#1065:

- R4: the "merkle root enforces the hint" rationale was wrong — the hinted
  height feeds BDK's per-input protocol-era flag selection, so a wrong
  assertion can change script validity without touching block content.
  Rewrote the safety rationale everywhere (proto, helper, interface,
  netsync collector): the hint is trusted, not verified; it is safe only
  because the sole producer derives it locally from the node's own txMap
  of a PoW-checked block, and the field MUST never be populated from
  peer-forwarded or untrusted data.
- R5: added TestValidate_ConsensusAcceptsHintedInBlockParent — the
  accept-counterpart of TestValidate_ConsensusRejectsUnconfirmedParent,
  running the real TxValidator + GoBDK with the same fixtures: same
  unconfirmed parent, ParentMetadata at the candidate height, validation
  succeeds. Together the pair pins the consensus outcome, not just the
  options wiring.
- R6: added a cross-subtree handler test — parent in subtree 0, child in
  subtree 1, two sequential CheckSubtreeFromBlock calls; the child's
  request hint resolves the parent.
- R7: renamed the shadowed err to accErr in the legacy branch.
- R9: documented the pre-extension coupling (hint supplies height, not
  body; legacy txs are pre-extended so no UTXO-store body fetch occurs;
  no Phase-3 backstop on this path).
@oskarszoon

This comment was marked as outdated.

@ctnguyen ctnguyen self-requested a review June 10, 2026 10:16
@oskarszoon oskarszoon removed the request for review from ordishs June 10, 2026 11:10
…solution option

The in_block_parent_hashes request list had an unbounded-payload flaw:
per-subtree parent hashes approach one hash per input on heavily-chained
blocks (a 1M-leaf subtree → ~32MB), past gRPC message limits exactly
where legacy sync needs it most — mainnet IBD through big chained
blocks.

Replace it with validator.WithUnconfirmedParentsAtCandidateHeight(true),
set only by the legacy branch of checkSubtreeFromBlock. The validator
resolves the unconfirmedParentHeight sentinel to the candidate block
height before the BDK call and the BIP68/MTP lookups. On this path the
candidate height is the parent's true height, so the substitution is
exact — cross-subtree parents included, no shared state, no payload,
safe with multiple subtreevalidation instances.

Consensus-safety trade, documented on the option and proto field: this
is fail-open at tx level for unconfirmed parents that are NOT in the
block; the membership backstop is block validation's
checkParentsExistOnChain, which legacy netsync runs on every block
before acceptance. The flag's contract (locally-held PoW-checked block,
membership check before acceptance, block assembly disabled) holds on
the legacy path; the peer-facing branch never sets it, pinned by test.

The option travels as an optional bool on ValidateTransactionRequest
(field 11) so it survives the gRPC hop to a remote validator; the
client-build → server-reconstruction mapping is pinned by a round-trip
test. Real-BDK accept/reject pair and the cross-subtree sequential
handler test carry over from the previous design.
@oskarszoon

Copy link
Copy Markdown
Contributor Author

Design reworked — the in_block_parent_hashes list is gone.

The list had an unbounded-payload flaw we caught after the last round: per-subtree in-block parent hashes approach one hash per input on heavily-chained blocks (a 1M-leaf subtree → ~32MB of hashes), past gRPC message limits exactly where legacy sync needs it most — mainnet IBD through big chained blocks.

Replacement: a single validator option, WithUnconfirmedParentsAtCandidateHeight(true), set only by the legacy branch of checkSubtreeFromBlock. The validator resolves the unconfirmedParentHeight sentinel to the candidate block height before the BDK call and the BIP68/MTP lookups. On this path the candidate height is the parent's true height, so the substitution is exact — cross-subtree parents included, no shared state, no payload.

Consensus-safety trade, stated explicitly in the option/proto docs: this is fail-open at tx level for mempool floaters; the membership backstop is block validation's checkParentsExistOnChain, which legacy netsync runs on every block before acceptance. The flag's contract (locally-held PoW-checked block, block-level membership check before acceptance, block assembly disabled) holds on the legacy path and the peer-facing branch never sets it — pinned by test. Your R4 concern shaped this: rather than documenting why a trusted hint list mustn't be misused, the trust decision is now a single bool whose misuse surface is one option contract.

Carry-overs from the previous round: the real-BDK accept/reject pair (TestValidate_ConsensusRejectsUnconfirmedParent / TestValidate_ConsensusAcceptsUnconfirmedParentAtCandidateHeight, same fixtures, R5), the cross-subtree two-sequential-calls handler test (R6), and a wire round-trip test for the new proto bool (field 11 on ValidateTransactionRequest — needed so the option survives the hop to a remote validator).

Comment thread services/legacy/netsync/handle_block.go Outdated
… lag invariant

Pass-2 review findings on the reworked design:

- P2 (validator/Server.go:485): the fail-open contract was enforced only
  by convention — the validator honored the flag even with
  AddTXToBlockAssembly=true (the default). Now validateTransaction
  hard-errors on that combination, covering both the in-process and
  gRPC/HTTP validator paths from one enforcement point. The legacy
  branch pairs the flag with an unconditional
  WithAddTXToBlockAssembly(false), replacing the FSM-conditional append
  (a per-subtree gRPC round-trip to derive a value that is always false
  for this branch's only caller).

- P2 (options.go:113): "unconfirmed parent IS a same-block parent" was
  missing its justification — the sentinel also covers a parent from
  block N-1 whose async SetMinedMulti has not landed. That window is
  closed by netsync's waitForPreviousBlockMined running before
  prepareSubtrees (all ancestors mined-set by induction); documented on
  the option, with a warning that other callers must establish the same
  invariant.

- Nits: backstop comment now says checkParentsExistOnChain fails the
  block in validOrderAndBlessed (not "rejects" with an implied invalid
  marking); proto field 11 is only put on the wire when the flag is set.

Tests: guard case added to the real-BDK accept test (flag without
assembly-off errors, runs before the accepting call so the store stays
clean); handler test asserts the legacy branch pairs flag with
assembly-off; wire round-trip asserts unset flag stays off the wire.
@ctnguyen

Copy link
Copy Markdown
Collaborator

Verification Scope

Citation-verified conclusions:

  • The refreshed diff targets head sha 4c60d382a2d7bec7bc8a9899d21a6a60779a6931 in the PR description.
  • The legacy CheckSubtreeFromBlock branch now sets both WithUnconfirmedParentsAtCandidateHeight(true) and WithAddTXToBlockAssembly(false) unconditionally (PR diff lines 37-54), and removes the per-subtree GetFSMCurrentState round trip (PR diff lines 42-51).
  • The validator now hard-errors when UnconfirmedParentsAtCandidateHeight is combined with AddTXToBlockAssembly=true (PR diff lines 328-338).
  • The new validator test covers both the hard-error default path and the accepting path when block assembly is explicitly disabled (PR diff lines 407-424).
  • The gRPC client only puts unconfirmed_parents_at_candidate_height on the wire when the option is true (PR diff lines 251-262 and 281-292); the server reconstructs it only when present (PR diff lines 299-305).
  • The wire round-trip test covers both set/true and unset/default-false cases (PR diff lines 514-556).
  • Field 11 is additive in the proto (PR diff lines 644-648), and the generated getter defaults nil/absent to false (PR diff lines 602-607).
  • Mainnet CSVHeight is 419328 in go-chaincfg v1.5.8 (go.mod:15; module cache params.go:268). The new validator accept test uses mainnet height 257727 (PR diff lines 379-386), so it is below CSV activation.
  • Existing source tests cover the generic sentinel and ParentMetadata behavior cited below.

Conclusions that would need a live patch application and build/test run:

  • Whether the new tests compile and pass under the repo's exact generated-code/toolchain setup.
  • Whether the generated protobuf files are byte-for-byte what the repo's pinned generator would produce.
  • Whether a full legacy-sync or multi-service integration run leaves no unexpected UTXO/txmeta side effects in the "external unconfirmed parent" case.
  • Whether mixed old/new validator deployments produce only the expected fail-closed liveness stalls under real load balancing.

The consensus/parity conclusions are citation-verified. I do not claim the PR's test suite is green.

Calibration Rule

Severity and author-work happiness measure different things:

  • Severity is the maximum risk if the item is wrong or ignored. critical would mean likely invalid-block acceptance, chain split, or silent consensus divergence. high means a consensus-boundary fail-open, persistent validator-side effect risk, or deployment mode that can halt legacy sync. low means the implementation is directionally correct and the remaining issue is mainly coverage, clarity, or operational polish.
  • Author-work happiness is how satisfied I am with the author's handling of that item, including code enforcement, tests, comments, and deployment mitigation. Within the same severity, lower happiness means mitigation is mostly assumed or not directly tested; higher happiness means there is executable enforcement, default-safe behavior, or clear deployment handling.

Current open high-severity ordering:

  1. Current P1, prior P2: validator-side effects before the block-level backstop. This is highest now because it is a single-node correctness/cleanup question at the consensus boundary. The new commits fixed block-assembly contamination, but they did not prove UTXO/txmeta side effects are absent or cleaned up for an external unconfirmed parent.
  2. Current P2, prior P3: rollback and mixed-version liveness. This is still high because it can halt legacy sync, but it gets higher happiness than current P1 because the wire default is fail-closed, the field is request-only, the new helper reduces needless field emission, and the PR description has a deploy note.

Prior P1 is now resolved for the concrete merge-blocking concern: the unsafe block-assembly combination is enforced in both the legacy caller and the validator. I still list it first because it was the previous top finding, but its current severity is low.

Overall Judgment

The author materially improved the PR after the prior review. The old highest-priority concern, "fail-open scope is documented but not code-enforced", is now resolved for block assembly: the legacy branch always disables assembly with the new flag, and the validator hard-errors if any caller combines the flag with assembly enabled.

I do not see a direct BDK/bitcoin-sv validation-parity flaw in the core fix. Resolving the teranode sentinel to the candidate height before BDK and before BIP68/MTP consumers is the right shape for a same-block parent in a block-validation context. The remaining concerns are around invalid/floater edge-case side effects, deployment skew, and PR-specific coverage of post-CSV BIP68 behavior.

I would be close to comfortable merging this version. The most valuable remaining change would be a focused external-floater regression test that proves final block non-acceptance and documents any validator-store side effects.

Detailed Concerns and Prior-Finding Status

1. Prior P1 resolved: fail-open block-assembly scope is now code-enforced

Status: resolved for the previous high-severity concern; residual severity is low.

The refreshed code fixes the earlier gap. The legacy branch now constructs validatorOptions with both WithUnconfirmedParentsAtCandidateHeight(true) and WithAddTXToBlockAssembly(false) in the same option list (PR diff lines 37-54). It also removes the prior FSM-conditional block-assembly disable and the GetFSMCurrentState call from this hot path (PR diff lines 42-51).

More importantly, the validator now enforces the dangerous combination centrally: if validationOptions.UnconfirmedParentsAtCandidateHeight is true and validationOptions.AddTXToBlockAssembly is also true, validateTransaction returns a processing error before it substitutes heights or reaches BDK (PR diff lines 328-338). That covers both in-process and gRPC/HTTP callers because they all reconstruct Options and land in this same function. The new validator test asserts the default hard-error and then the accepting call with WithAddTXToBlockAssembly(false) (PR diff lines 407-424), and the subtreevalidation test asserts the legacy branch records both the resolution flag and assembly disabled (PR diff lines 173-178).

This is a substantial improvement. The only residual low-severity nit is that the guard is option-level, not computed from the global BlockAssembly.Disabled setting. That is probably the right conservative API contract, but callers in globally-disabled assembly deployments still must pass WithAddTXToBlockAssembly(false) when using this flag.

2. Prior P2 partially addressed: external-floater side effects are still not pinned

Status: partially addressed; current highest open issue.

The new commits address the largest part of this concern by preventing block-assembly contamination. A transaction accepted through the fail-open height resolution can no longer be placed into our own block template when the PR's safety contract is followed, and the validator hard-errors if a caller forgets the pairing.

The remaining issue is UTXO/txmeta side effects before the block-level parent-membership backstop. With this option set, a parent that exists in the UTXO store with empty BlockHeights but is not in the candidate block can pass tx-level validation after the sentinel is resolved to the candidate height. After tx validation succeeds, current source can spend inputs at services/validator/Validator.go:655 and create tx metadata/UTXOs at services/validator/Validator.go:754-756. The later block backstop checks same-block ordering/membership in model/Block.go:1088-1102 and checks external parents through txmeta in model/Block.go:1125-1145; for missing parents or parents with no BlockIDs, current source returns block-incomplete style failures, not a successful block acceptance.

That means final block acceptance still appears protected, but the PR does not pin what happens to validator-store mutations from an invalid external-floater case. The comments now say the backstop "fails the block" (PR diff lines 20-27 and 467-479), but the precise behavior is more nuanced: it may be incomplete/retry rather than invalid, and side effects may already exist by that point.

Recommended test: construct a legacy block/subtree case where the child spends a parent that exists in the UTXO store with empty BlockHeights but the parent is not in the block. Assert that block acceptance does not succeed, and explicitly assert the expected UTXO/txmeta state after the failed attempt. If the side effects are intentional and harmless, document that contract in the test.

3. Prior P3 partially addressed: rollback and mixed-version deployment remain liveness-sensitive

Status: partially addressed; second-highest open issue.

The refreshed client helper is a good improvement. unconfirmedParentsAtCandidateHeightPtr returns nil unless the option is true, so the optional proto field is only emitted for the rare legacy-sync requests that need it (PR diff lines 251-262 and 281-292). The round-trip test now asserts set/true survives marshal and reconstruction, and default/unset stays false with a nil request field (PR diff lines 520-555). The field is additive at proto tag 11 (PR diff lines 644-648), so absent/unknown field behavior is wire-compatible.

Deployment skew still matters:

  • New subtreevalidation to old validator: the old service ignores field 11 as unknown, so the behavior is old/fail-closed and legacy sync can still wedge on same-block parents.
  • Old subtreevalidation to new validator: the field is absent, so the new server reconstructs false and keeps old/fail-closed behavior.
  • Mixed validator fleets behind load balancing: identical legacy subtree requests can pass or fail depending on which validator instance receives the request.
  • Downgrading after processing some blocks does not require data migration because the option is request-only, but it reintroduces the same-block-parent wedge for future legacy catchup blocks.

The PR description includes the right deploy note: deploy services together. I would still prefer that note in a durable release/deployment document or near the proto field comment, because this is operationally consensus-adjacent even though it is primarily a liveness failure.

4. Low item resolved/affirmed: MEMPOOL_HEIGHT and BDK era-flag reasoning is correct

Status: resolved/affirmed.

The core parity argument checks out. In BDK, consensus script verification rejects MEMPOOL_HEIGHT (bdk/core/txvalidator.cpp:779-780), and BDK computes policy era at blockHeight + 1 but consensus era at blockHeight (bdk/core/txvalidator.cpp:960-980). bitcoin-sv's policy helper maps MEMPOOL_HEIGHT to chainActive.Height()+1 (bitcoin-sv/src/validation.cpp:2668-2675), while block validation gets real coin heights from the shard view before SequenceLocks (bitcoin-sv/src/validation.cpp:3337-3345).

For a same-block parent in a block being validated at height H, the real coin height is H. The author's placement in validateTransaction substitutes before BDK validation and before the later BIP68/MTP phase (PR diff lines 318-342), which is the correct data-flow point.

5. Low item open: BIP68/MTP behavior is still not PR-specifically pinned

Status: open, but lower risk than before because the code placement and comments are good.

The implementation substitutes the sentinel before the BIP68/MTP phase, so the intended data flow is correct (PR diff lines 318-342; current source returns into BIP68 only after the CSV gate at services/validator/Validator.go:1603-1638). The option comment now also explains why "unconfirmed implies same-block" is supposed to hold on the legacy path: netsync waits for the previous block's mined-set before validating the next block (services/legacy/netsync/handle_block.go:132-137 and :231-234; PR diff lines 456-465).

The new validator accept test does not exercise BIP68/MTP. It uses mainnet height 257727 (PR diff lines 379-386), and mainnet CSVHeight is 419328 (go.mod:15; module cache params.go:268). Existing source has generic post-CSV MTP-path coverage in TestValidateTransaction_BIP68PathReadsMTPStore, which intentionally uses mainnet height 500000 (services/validator/Validator_test.go:2252-2264), but that test does not combine UnconfirmedParentsAtCandidateHeight with the substituted same-block parent.

Recommended coverage: add one post-CSV, pre-Genesis test using WithUnconfirmedParentsAtCandidateHeight(true) where a same-block parent with sequence 0 passes and a same-block parent with a non-zero relative lock fails until the later block. That would pin the PR's BIP68/MTP claim directly.

6. Low item partially addressed: test coverage improved, but the legacy handler test is still mostly option propagation

Status: partially addressed.

Coverage is meaningfully better now:

  • The validator-level accept/reject pair uses real TxValidator + GoBDK and proves the unconfirmed-parent sentinel rejects without the option and accepts with the option plus assembly disabled (existing source services/validator/Validator_test.go:493-548; PR diff lines 355-428).
  • The subtreevalidation test pins that the legacy branch sets the flag and now also pins the required AddTXToBlockAssembly=false pairing (PR diff lines 160-178).
  • The same test pins the peer-facing path does not set the fail-open flag (PR diff lines 212-240).
  • The wire test pins client build, proto marshal/unmarshal, server reconstruction, and default-false behavior (PR diff lines 514-556).

The remaining gap is narrower: the new cross-subtree handler test uses recordingValidatorClient, whose ValidateWithOptions records options and returns synthetic metadata rather than delegating through the real validator or creating parent UTXOs (services/subtreevalidation/check_block_subtrees_assembled_path_test.go:52-66). So it does not itself prove that the first subtree creates a parent with empty BlockHeights and the second subtree validates through the real UTXO fallback. Existing tests cover the generic sentinel and ParentMetadata pieces, including empty BlockHeights fallback (services/validator/Validator_test.go:1756-1801), preservation of ParentMetadata through extend (:1803-1894), and no-sentinel behavior when metadata exists (:1896-1940). The PR-specific missing piece is one real UTXO integration test for the legacy bool path.

7. Low item resolved: gRPC/proto plumbing and default-false compatibility are now solid

Status: resolved.

The new helper avoids putting an explicit false on the wire (PR diff lines 251-262), the server still reconstructs absent/nil as false (PR diff lines 299-305), and the round-trip test covers both set and default cases (PR diff lines 520-555). This makes nil and explicit-false behavior equivalent on the server while reducing unnecessary unknown-field traffic to old validators.

The proto change is additive at tag 11 (PR diff lines 644-648), so old receivers ignore it and new receivers default absent to false. That is the safe/fail-closed compatibility stance for a consensus-adjacent flag. The only remaining nit is generated-code churn: the generated comments show protoc changing from v6.33.2 to v7.34.1 (PR diff lines 561-566 and 656-661). I do not see a wire-compatibility issue from that, but generated files should ideally come from the repo-pinned generator.

8. Low item partially addressed: maintainability and clarity are much better, with one API sharp edge

Status: partially addressed.

The option comment is unusually detailed, and the new validator-side guard makes the main safety contract executable. That is appropriate for consensus-boundary code. The new helper name also makes the wire-default behavior clearer.

The remaining clarity nit is the call shape in the validator hunk: substituteUnconfirmedHeights(utxoHeights, blockHeight, false) is intentionally used on a consensus/block-validation path because false selects candidate-height substitution rather than BDK's consensus MEMPOOL_HEIGHT substitution (PR diff lines 340-341; existing helper at services/validator/ScriptVerifierGoBDK.go:54-71). The inline comment explains it, but this is easy to misread later. A purpose-named wrapper such as resolveUnconfirmedParentsAtCandidateHeight would make this special case easier to grep and harder to refactor incorrectly.

Summary Table

Priority Prior finding Current status Item Severity Author-work happiness
resolved prior P1 P1 resolved Fail-open block-assembly scope is now enforced in the legacy caller and validator low 90%
current P1 P2 partially addressed External unconfirmed-parent backstop still runs after possible UTXO/txmeta side effects high 64%
current P2 P3 partially addressed Rollback and mixed-version deployments can still halt legacy sync until services are upgraded together high 78%
P3 low item resolved/affirmed MEMPOOL_HEIGHT / BDK consensus-vs-policy era reasoning and substitution placement low 94%
P4 low item open BIP68/MTP path is plausible but not PR-specifically pinned post-CSV low 76%
P5 low item partially addressed Test coverage is much stronger, but legacy bool path lacks one real UTXO integration case low 82%
P6 low item resolved gRPC/proto field 11, nil/default-false behavior, and wire round-trip low 93%
P7 low item partially addressed Maintainability/clarity of the special sentinel substitution and strict option-level guard low 84%

GLOBAL HAPPINESS: 86%

@oskarszoon oskarszoon self-assigned this Jun 10, 2026
…the substitution

Pass-3 review items:

- Floater side effects (current P1): new real-validator integration test
  (TestLegacyUnconfirmedParent_RealValidatorIntegration) runs the legacy
  gRPC handler against the real validator + GoBDK + sqlitememory store —
  unmined parent, child through the genuine UTXO fallback and option —
  and asserts the post-blessing store state: child meta unmined, parent
  output spent by the child. The validator-level accept test pins the
  same contract. Backstop comments corrected: checkParentsExistOnChain
  returns BlockIncompleteError (retry/catchup-ordering semantics, issue
  bsv-blockchain#1031, pinned in model/Block_test.go), not an invalid-block marking;
  the store state equals a policy-mode mempool admission of the same txs
  and is handled by the same unmined-tx cleanup machinery.

- Deployment skew (current P2): full skew matrix documented on the proto
  field comment — new→old ignores the field (wedge persists), old→new
  reconstructs false, mixed validator fleets pass-or-wedge per instance,
  downgrade reintroduces the wedge but corrupts nothing.

- Call-shape clarity (P7): resolveUnconfirmedParentsAtCandidateHeight
  wraps substituteUnconfirmedHeights(…, false) so the consensus-context
  call site no longer reads as a policy-mode translation.

Deferred deliberately: PR-specific post-CSV BIP68 pinning (needs
post-CSV pre-Genesis sequence-locked fixtures; the substitution-before-
BIP68 data flow and generic MTP-path coverage already exist).
@oskarszoon

Copy link
Copy Markdown
Contributor Author

Pass-3 items addressed in a0ac819.

Current P1 (floater side effects) — pinned twice, plus comment corrections:

  • New TestLegacyUnconfirmedParent_RealValidatorIntegration: nothing mocked between the gRPC handler and GoBDK (subtreevalidation Server → real validator → real TxValidator/GoBDK → real sqlitememory store). Parent sits in the store with empty BlockHeights (the test comment argues byte-equivalence of Create without mined info to a prior blessing call — the parent's own input tx isn't constructible from fixtures), the child validates through the real UTXO fallback + option, and the store side effects are asserted: child meta unmined, parent output spent by the child.
  • The real-BDK accept test now asserts the same side-effect contract at validator level, with the contract spelled out: the state is identical to a policy-mode mempool admission of the same txs and is handled by the same unmined-tx cleanup machinery.
  • You were right about the backstop nuance: checkParentsExistOnChain returns BlockIncompleteError (retry/catchup-ordering semantics, issue blockvalidation: ErrTxMissingParent/ErrTxNotFound during subtree validation wrongly persists block as invalid (permanent sync stall) #1031), not an invalid marking — already pinned by the "parent has no block ID" case in model/Block_test.go. All comments now say so instead of "rejects the block".

Current P2 (deployment skew) — the full skew matrix now lives on the proto field comment (new→old ignores the field and keeps the wedge, old→new reconstructs false, mixed fleets behind LB pass-or-wedge per instance, downgrade reintroduces the wedge but corrupts nothing; upgrade together). Durable and next to the thing that ships.

P4 (post-CSV BIP68 pin) — deferred, deliberately. The data flow is covered (substitution happens before the BIP68/MTP phase; generic post-CSV MTP coverage exists in TestValidateTransaction_BIP68PathReadsMTPStore), and pinning it PR-specifically needs post-CSV pre-Genesis sequence-locked fixtures that don't exist yet. Will pick it up as a follow-up rather than hold the wedge fix on it.

P5 (real UTXO integration) — covered by the new integration test above.

P7 (call-shape sharp edge) — added resolveUnconfirmedParentsAtCandidateHeight(utxoHeights, blockHeight): purpose-named wrapper whose doc states it reuses the consensus=false branch because that branch IS the wanted blockHeight substitution, not because the caller is in policy mode.

Residual nit from your item 1 (option-level guard vs global BlockAssembly.Disabled): keeping option-level — as you note, the conservative API contract is the point; a globally-disabled-assembly caller passing the flag without WithAddTXToBlockAssembly(false) should still be told explicitly.

protoc version note: regenerated locally (v7.34.1 vs the repo files' v6.33.2). Wire-identical apart from the version comment; if the repo pins a generator in CI I'll regenerate from that — pointer welcome.

Comment thread services/subtreevalidation/Server.go Outdated
…s keep pre-change behaviour

The previous commit's justification for the unconditional
WithAddTXToBlockAssembly(false) was wrong: the legacy branch is not
exclusive to legacy sync. handleBlockMsg calls HandleBlockDirect
unconditionally and block listeners are enabled whenever the FSM is not
LEGACYSYNCING, so tip blocks arriving over the legacy bridge while
RUNNING also reach this branch — where the old code left assembly
enabled. Forcing it false there was an unintended behaviour change
(blessed txs from a reorged-out block would vanish from our template).

Restore the FSM gate: WithUnconfirmedParentsAtCandidateHeight(true) and
WithAddTXToBlockAssembly(false) are paired inside the
LEGACYSYNCING/CATCHINGBLOCKS conditional. RUNNING preserves the
pre-change behaviour exactly — fail-closed sentinel, assembly enabled.
An in-block tx chain in a RUNNING-state legacy-bridge block still fails
this branch (same as before this PR); the authoritative tip path is
blockvalidation's CheckBlockSubtrees with its accumulator.

Tests: fsmStateOverrideClient pins the FSM state (LocalClient always
reports RUNNING); new RUNNING-state subtest asserts no flag + assembly
enabled; legacy-sync subtests and the real-validator integration test
run under LEGACYSYNCING.
@sonarqubecloud

Copy link
Copy Markdown

…ING catch-up wedged

Field test on testnet (beta deploy) wedged immediately at 1740437: a
restarted node has its FSM restored to RUNNING and catches up the gap
over the legacy bridge (handleBlockMsg → HandleBlockDirect runs in
every FSM state), so the FSM-gated flag was inactive for exactly the
blocks it exists to fix. On a node whose only peers are legacy nodes
there is no CheckBlockSubtrees fallback path either.

FSM state was the wrong predicate. The consensus-safety conditions
(locally-held PoW-checked block; block-level membership check before
acceptance) hold for every legacy-branch call, so
WithUnconfirmedParentsAtCandidateHeight(true) is now unconditional on
the legacy branch. The assembly option returns to exactly upstream's
FSM-conditional behaviour: disabled during LEGACYSYNCING/CATCHINGBLOCKS,
enabled in RUNNING (reorg resilience).

The validator's flag+assembly hard-error is removed — it made the two
requirements above mutually exclusive in RUNNING. The combination is
safe: a floater child blessed at the candidate height is the same tx
policy-mode admission would put into assembly anyway (policy
substitutes tip+1 for unconfirmed parents, equal to the candidate
height at the tip; era flags cannot differ post-Genesis), and
accepted-block txs are mined-removed from assembly as always.

Tests: RUNNING subtest now pins flag=on + assembly=on; legacy-sync
subtests pin flag=on + assembly=off; guard subtest removed; docs on
the option and proto field updated to the compatibility stance.

@ordishs ordishs left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve — correct, minimal, and a good architectural choice: a stateless per-request hint that reuses the existing parentMetadataAccumulator / filterParentMetadataForInputs machinery from the CheckBlockSubtrees path rather than introducing server-side per-block state. Backward compatible (empty list = prior behaviour), and the trust model is unchanged (a lying hint yields a block that fails acceptance, never a wrongly-accepted one).

Verified locally: affected packages build; TestCollectInBlockParentHashes, TestBuildInBlockParentAccumulator, and TestLegacySubtreeInBlockParentHint pass under -race. The hand-edited .pb.go rawDesc is internally consistent (message length +53 bytes matches the appended field-6 descriptor).

Minor follow-ups (non-blocking):

  1. Wiring test gap — the tests exercise buildInBlockParentAccumulator + validateSubtreeInternalImpl directly, but not the Server.checkSubtreeFromBlock glue (request.InBlockParentHashes → accumulator → impl). That path is only covered by compilation; consider one end-to-end test through the gRPC server method with a populated InBlockParentHashes.

  2. Perf sanity checkcollectInBlockParentHashes does a mutex-guarded txMap.Get per input per tx, and on a typical post-checkpoint block most inputs are prior-block (mostly-miss) lookups. It only runs in non-quickValidationMode, so likely fine, but worth confirming the pre-pass adds no meaningful sync latency on a large real block.

  3. Naming nit — the function collects both intra- and cross-subtree in-block parents (a superset); the comment could say so rather than implying cross-subtree only.

Make sure the rolling-upgrade-skew note (deploy netsync + subtreevalidation together) makes it into the deploy runbook, not just the PR description.

…mber resolution flag to proto field 12

Conflicts resolved:
- validator_api.proto: bsv-blockchain#1073 took field 11 for in_block (merged upstream
  first, owns the number); unconfirmed_parents_at_candidate_height moves
  to field 12. Both fields kept.
- validator_api.pb.go: regenerated from the resolved proto.
- validator/Client.go: buildValidateTxRequest carries both InBlock
  (upstream) and UnconfirmedParentsAtCandidateHeight (this branch).

WIRE SKEW WARNING for the existing beta deployment: the beta build sends
the resolution flag as field 11, which post-merge means in_block. A beta
subtreevalidation talking to a post-merge validator would silently set
the wrong option (wedge returns, wrong provenance). Replace beta
deployments wholesale; do not mix beta and post-merge services.
@sonarqubecloud

Copy link
Copy Markdown

@oskarszoon oskarszoon merged commit e21ce10 into bsv-blockchain:main Jun 11, 2026
34 checks passed
oskarszoon added a commit that referenced this pull request Jun 11, 2026
oskarszoon added a commit to oskarszoon/teranode that referenced this pull request Jun 11, 2026
…r with candidate-height resolution

Replaces the per-block ParentMetadata accumulator (bsv-blockchain#1013) on the
block-validation path with the WithUnconfirmedParentsAtCandidateHeight
validator option (introduced for the legacy path in bsv-blockchain#1065), then deletes the
accumulator machinery.

In-block parents resolve the unconfirmedParentHeight sentinel to the candidate
block height inside validateTransaction — identical outcome to the accumulator
for genuine in-block parents, without the hot-path seeding, per-tx metadata
filtering, per-level mutex merges, or per-tx ParentMetadata wire payload. On a
5.24M-tx chained block this removes ~34% end-to-end overhead.

Safety:
- PoW (nBits + HasMetTargetDifficulty) now runs before subtree blessing in
  ValidateBlock, so the fail-open option cannot be triggered by a no-PoW header.
- Floater backstop: a parent that is unconfirmed and not in the block fails open
  at tx level, then block.Valid surfaces ErrBlockIncomplete. When caught up
  (FSM not CATCHINGBLOCKS/LEGACYSYNCING) that is a genuine floater
  (SetMinedMulti->MinedSet invariant), so the block is invalidated/rolled back at
  every acceptance sink (optimistic background, non-optimistic, revalidation
  worker); while syncing it stays incomplete and is retried, preserving bsv-blockchain#1031.
- Recorded parent BlockHeights are untouched (sentinel exact-match only), so
  fork/revalidation height resolution is unchanged.

Deprecates proto field 10 (parent_metadata) via reserved; deletes ParentTxMetadata.
Preserves the in_block provenance option (bsv-blockchain#1073, field 11); unconfirmed-parents
resolution stays field 12.

Closes bsv-blockchain#1066
oskarszoon added a commit to oskarszoon/teranode that referenced this pull request Jun 11, 2026
…-legacysyncing

Eleven upstream commits, several touching legacy code this branch also
changed. Resolutions:

- netsync/manager.go: upstream bsv-blockchain#1067 deleted the catchingBlocks early-return
  in handleBlockMsg (the swallowed orphan tip is the legacy batch-continuation
  signal; swallowing it stalled sync). Took upstream's always-request
  behaviour and updated the surrounding comments to post-LEGACYSYNCING wording.
- subtreevalidation/Server.go: kept upstream bsv-blockchain#1065's richer rationale for
  disabling block-assembly additions, on this branch's CATCHINGBLOCKS-only
  condition.
- Re-applied the LEGACYSYNCING removal to symbols upstream reintroduced:
  validator publishPolicyRejectedTx gate (bsv-blockchain#799) now checks CATCHINGBLOCKS
  only; the new legacy gate-pinning tests (bsv-blockchain#1065) drive CATCHINGBLOCKS
  instead of the removed state; comment references updated in adaptivefetch,
  Validator_test, and the Server.go incident note.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants