Skip to content

test(blockassembly/subtreeprocessor): reproduce #852 moveForwardBlock drain loss#856

Merged
liam merged 2 commits into
bsv-blockchain:mainfrom
liam:liam/blockassembly-issue-852-repro
May 14, 2026
Merged

test(blockassembly/subtreeprocessor): reproduce #852 moveForwardBlock drain loss#856
liam merged 2 commits into
bsv-blockchain:mainfrom
liam:liam/blockassembly-issue-852-repro

Conversation

@liam

@liam liam commented May 13, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds TestMoveForwardBlockDrainLoss_BatchesLostOnPostDrainError - a deterministic, tests-only reproduction of the rollback gap described in #852.

Refs #852.

What the test does

  • errOnCreateUtxoStore wraps a real sqlitememory utxo store via interface embedding and overrides Create to return a sentinel error. All other methods delegate. This injects a step-6 failure (processCoinbaseUtxos -> utxoStore.Create) after step-5 has already drained the queue.
  • Clocks pinned via fixedClock so DoubleSpendWindow=0 admits the enqueued batch at drain time (consistent with the fix(blockassembly/subtreeprocessor): zero-guard validFromMillis in dequeueDuringBlockMovement #846 fix).
  • SubtreeProcessor built without Start so the event-loop goroutine cannot drain the batch concurrently. moveForwardBlock is invoked directly and the production rollback at SubtreeProcessor.go:711-717 is replayed inline.

Output

[#852 repro] pre-call queue.length=1, post-call queue.length=0,
lost tx hash=ef2e562c... - not in queue, not in any subtree after rollback

The test pins this current (buggy) behaviour as a regression guard. When the fix lands (any of the three remediation options from #852), the queue-length assertion will flip to require.Equal(t, preLen, postLen) and an additional require.Contains will assert the batch is still in the queue.

Test plan

  • go vet ./services/blockassembly/subtreeprocessor/ - clean
  • go test -race -count=1 -run TestMoveForwardBlockDrainLoss ./services/blockassembly/subtreeprocessor/ - pass (~7s)
  • go test -race -count=1 ./services/blockassembly/subtreeprocessor/ - pass (~155s)

…veForwardBlock drain loss

Adds TestMoveForwardBlockDrainLoss_BatchesLostOnPostDrainError - a
deterministic, tests-only reproduction of the rollback gap described in
issue bsv-blockchain#852.

Setup:

* errOnCreateUtxoStore wraps a real sqlitememory utxo store via
  interface embedding and overrides Create to return a sentinel error.
  All other methods delegate to the embedded store. This injects a
  step-6 failure (processCoinbaseUtxos -> utxoStore.Create) AFTER
  step-5 has already drained the queue.
* Clocks pinned via fixedClock so DoubleSpendWindow=0 admits the
  enqueued batch at drain time (consistent with the bsv-blockchain#846 fix).
* SubtreeProcessor built without Start so the event-loop goroutine
  cannot drain the batch concurrently. moveForwardBlock is invoked
  directly (lowercase) and the production rollback (lines 711-717) is
  replayed inline by the test.

What the test demonstrates:

1. Pre-call: queue contains one enqueued batch.
2. moveForwardBlock errors with the sentinel from the Create wrapper.
3. After the inline rollback, the snapshotted in-memory fields
   (chainedSubtrees, currentSubtree, currentTxMap, currentBlockHeader)
   are restored.
4. The queue is empty: processRemainderTransactionsAndDequeue drained
   the batch into the now-discarded new subtree state.
5. The drained tx hash is in neither the queue nor any subtree -> the
   batch is silently lost.

This test PINS the current buggy behaviour. When the underlying fix
lands (option 1, 2, or 3 from the issue), the assertion at queue
length 0 will need to flip to require.Equal(t, preLen, postLen) and
require.Contains for the tx hash in the queue.

Test plan:
- go vet ./services/blockassembly/subtreeprocessor/ - clean
- go test -race -count=1 -run TestMoveForwardBlockDrainLoss ./services/blockassembly/subtreeprocessor/ - pass (~7s)
- go test -race -count=1 ./services/blockassembly/subtreeprocessor/ - pass (~155s)

Refs bsv-blockchain#852.
@github-actions

github-actions Bot commented May 13, 2026

Copy link
Copy Markdown
Contributor

🤖 Claude Code Review

Status: Complete

Current Review:
No issues found. The test accurately reproduces the #852 rollback gap with correct implementation and documentation.

Analysis:

  • Test correctly injects step-6 failure via errOnCreateUtxoStore wrapper
  • Reproduces the exact production rollback path from SubtreeProcessor.go:711-717
  • Verifies all aspects of the bug: queue drain, in-memory rollback, batch loss
  • Test is properly isolated (no concurrent Start loop)
  • All dependencies exist: fixedClock, collectSubtreeHashes, testdata fixture
  • Documentation clearly states this pins buggy behavior for regression detection
  • Inline comments explain when assertions will flip after fix lands

@github-actions

github-actions Bot commented May 13, 2026

Copy link
Copy Markdown
Contributor

Benchmark Comparison Report

Baseline: main (unknown)

Current: PR-856 (c4de7e3)

Summary

  • Regressions: 0
  • Improvements: 0
  • Unchanged: 142
  • Significance level: p < 0.05
All benchmark results (sec/op)
Benchmark Baseline Current Change p-value
_NewBlockFromBytes-4 1.671µ 1.686µ ~ 1.000
SplitSyncedParentMap_SetIfNotExists/256_buckets-4 61.47n 61.57n ~ 0.700
SplitSyncedParentMap_SetIfNotExists/16_buckets-4 61.56n 61.73n ~ 0.400
SplitSyncedParentMap_SetIfNotExists/1_bucket-4 61.76n 61.38n ~ 0.100
SplitSyncedParentMap_ConcurrentSetIfNotExists/256_buckets... 30.31n 30.85n ~ 0.200
SplitSyncedParentMap_ConcurrentSetIfNotExists/16_buckets_... 51.55n 51.42n ~ 1.000
SplitSyncedParentMap_ConcurrentSetIfNotExists/1_bucket_pa... 106.6n 108.3n ~ 0.100
MiningCandidate_Stringify_Short-4 261.0n 262.1n ~ 0.400
MiningCandidate_Stringify_Long-4 1.901µ 1.893µ ~ 1.000
MiningSolution_Stringify-4 984.7n 1000.0n ~ 0.700
BlockInfo_MarshalJSON-4 1.775µ 1.786µ ~ 0.400
NewFromBytes-4 125.0n 125.5n ~ 0.300
Mine_EasyDifficulty-4 61.07µ 61.08µ ~ 0.400
Mine_WithAddress-4 7.616µ 6.852µ ~ 0.100
DirectSubtreeAdd/4_per_subtree-4 57.55n 60.72n ~ 0.100
DirectSubtreeAdd/64_per_subtree-4 28.28n 28.36n ~ 1.000
DirectSubtreeAdd/256_per_subtree-4 27.31n 27.41n ~ 1.000
DirectSubtreeAdd/1024_per_subtree-4 26.21n 26.29n ~ 0.200
DirectSubtreeAdd/2048_per_subtree-4 25.89n 25.84n ~ 0.400
SubtreeProcessorAdd/4_per_subtree-4 284.2n 289.0n ~ 0.400
SubtreeProcessorAdd/64_per_subtree-4 274.4n 282.1n ~ 0.100
SubtreeProcessorAdd/256_per_subtree-4 278.9n 279.5n ~ 1.000
SubtreeProcessorAdd/1024_per_subtree-4 270.5n 272.0n ~ 0.800
SubtreeProcessorAdd/2048_per_subtree-4 271.5n 272.0n ~ 0.500
SubtreeProcessorRotate/4_per_subtree-4 274.4n 274.7n ~ 0.400
SubtreeProcessorRotate/64_per_subtree-4 275.5n 273.5n ~ 0.700
SubtreeProcessorRotate/256_per_subtree-4 271.8n 277.6n ~ 0.100
SubtreeProcessorRotate/1024_per_subtree-4 270.0n 274.8n ~ 0.200
SubtreeNodeAddOnly/4_per_subtree-4 54.47n 54.44n ~ 1.000
SubtreeNodeAddOnly/64_per_subtree-4 34.17n 34.45n ~ 0.100
SubtreeNodeAddOnly/256_per_subtree-4 33.52n 33.66n ~ 0.700
SubtreeNodeAddOnly/1024_per_subtree-4 32.73n 32.76n ~ 0.500
SubtreeCreationOnly/4_per_subtree-4 113.1n 115.9n ~ 0.100
SubtreeCreationOnly/64_per_subtree-4 398.6n 406.0n ~ 0.100
SubtreeCreationOnly/256_per_subtree-4 1.335µ 1.487µ ~ 0.100
SubtreeCreationOnly/1024_per_subtree-4 4.354µ 4.623µ ~ 0.100
SubtreeCreationOnly/2048_per_subtree-4 8.009µ 8.447µ ~ 0.100
SubtreeProcessorOverheadBreakdown/64_per_subtree-4 269.6n 275.9n ~ 0.100
SubtreeProcessorOverheadBreakdown/1024_per_subtree-4 270.3n 272.3n ~ 0.100
ParallelGetAndSetIfNotExists/1k_nodes-4 604.2µ 826.4µ ~ 0.100
ParallelGetAndSetIfNotExists/10k_nodes-4 1.306m 1.625m ~ 0.100
ParallelGetAndSetIfNotExists/50k_nodes-4 6.699m 6.688m ~ 1.000
ParallelGetAndSetIfNotExists/100k_nodes-4 13.53m 13.44m ~ 1.000
SequentialGetAndSetIfNotExists/1k_nodes-4 660.3µ 655.9µ ~ 0.200
SequentialGetAndSetIfNotExists/10k_nodes-4 3.177m 2.775m ~ 0.100
SequentialGetAndSetIfNotExists/50k_nodes-4 11.07m 10.40m ~ 0.100
SequentialGetAndSetIfNotExists/100k_nodes-4 21.22m 20.12m ~ 0.100
ProcessOwnBlockSubtreeNodesParallel/1k_nodes-4 644.1µ 628.4µ ~ 0.100
ProcessOwnBlockSubtreeNodesParallel/10k_nodes-4 4.227m 4.176m ~ 0.400
ProcessOwnBlockSubtreeNodesParallel/100k_nodes-4 16.84m 16.90m ~ 1.000
ProcessOwnBlockSubtreeNodesSequential/1k_nodes-4 716.0µ 688.8µ ~ 0.100
ProcessOwnBlockSubtreeNodesSequential/10k_nodes-4 5.899m 5.692m ~ 0.100
ProcessOwnBlockSubtreeNodesSequential/100k_nodes-4 39.14m 38.06m ~ 0.100
BlockAssembler_AddTx-4 0.03360n 0.02812n ~ 0.200
AddNode-4 12.44 12.59 ~ 0.400
AddNodeWithMap-4 12.83 13.07 ~ 0.200
DiskTxMap_SetIfNotExists-4 4.106µ 4.067µ ~ 0.400
DiskTxMap_SetIfNotExists_Parallel-4 3.972µ 3.931µ ~ 1.000
DiskTxMap_ExistenceOnly-4 332.2n 324.0n ~ 0.400
Queue-4 201.3n 195.7n ~ 0.200
AtomicPointer-4 8.158n 8.128n ~ 0.300
ReorgOptimizations/DedupFilterPipeline/Old/10K-4 785.2µ 792.8µ ~ 1.000
ReorgOptimizations/DedupFilterPipeline/New/10K-4 732.8µ 706.4µ ~ 0.100
ReorgOptimizations/AllMarkFalse/Old/10K-4 120.3µ 112.5µ ~ 0.400
ReorgOptimizations/AllMarkFalse/New/10K-4 58.54µ 58.44µ ~ 0.100
ReorgOptimizations/HashSlicePool/Old/10K-4 65.68µ 57.95µ ~ 0.100
ReorgOptimizations/HashSlicePool/New/10K-4 11.81µ 11.78µ ~ 0.100
ReorgOptimizations/NodeFlags/Old/10K-4 5.388µ 4.702µ ~ 0.200
ReorgOptimizations/NodeFlags/New/10K-4 1.702µ 1.593µ ~ 0.100
ReorgOptimizations/DedupFilterPipeline/Old/100K-4 10.849m 9.139m ~ 0.100
ReorgOptimizations/DedupFilterPipeline/New/100K-4 12.078m 9.582m ~ 0.100
ReorgOptimizations/AllMarkFalse/Old/100K-4 1.215m 1.102m ~ 0.100
ReorgOptimizations/AllMarkFalse/New/100K-4 726.8µ 733.7µ ~ 1.000
ReorgOptimizations/HashSlicePool/Old/100K-4 573.1µ 551.7µ ~ 0.400
ReorgOptimizations/HashSlicePool/New/100K-4 299.7µ 300.1µ ~ 1.000
ReorgOptimizations/NodeFlags/Old/100K-4 49.86µ 46.20µ ~ 0.100
ReorgOptimizations/NodeFlags/New/100K-4 16.97µ 16.86µ ~ 0.700
TxMapSetIfNotExists-4 51.43n 51.45n ~ 1.000
TxMapSetIfNotExistsDuplicate-4 43.35n 43.82n ~ 0.100
ChannelSendReceive-4 670.9n 664.5n ~ 0.100
CalcBlockWork-4 494.9n 495.0n ~ 0.700
CalculateWork-4 677.3n 692.5n ~ 1.000
BuildBlockLocatorString_Helpers/Size_10-4 1.329µ 1.324µ ~ 0.800
BuildBlockLocatorString_Helpers/Size_100-4 15.23µ 14.92µ ~ 1.000
BuildBlockLocatorString_Helpers/Size_1000-4 125.0µ 125.8µ ~ 0.400
CatchupWithHeaderCache-4 104.4m 104.4m ~ 1.000
_BufferPoolAllocation/16KB-4 4.851µ 3.553µ ~ 0.100
_BufferPoolAllocation/32KB-4 7.815µ 7.470µ ~ 0.400
_BufferPoolAllocation/64KB-4 16.17µ 18.81µ ~ 0.700
_BufferPoolAllocation/128KB-4 32.35µ 31.02µ ~ 0.700
_BufferPoolAllocation/512KB-4 107.1µ 114.5µ ~ 0.100
_BufferPoolConcurrent/32KB-4 20.07µ 20.42µ ~ 0.700
_BufferPoolConcurrent/64KB-4 32.22µ 31.98µ ~ 1.000
_BufferPoolConcurrent/512KB-4 151.1µ 151.4µ ~ 1.000
_SubtreeDeserializationWithBufferSizes/16KB-4 650.6µ 619.4µ ~ 0.400
_SubtreeDeserializationWithBufferSizes/32KB-4 675.3µ 624.4µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/64KB-4 666.0µ 619.1µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/128KB-4 655.2µ 620.0µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/512KB-4 669.3µ 640.2µ ~ 0.100
_SubtreeDataDeserializationWithBufferSizes/16KB-4 35.71m 35.62m ~ 0.400
_SubtreeDataDeserializationWithBufferSizes/32KB-4 35.84m 35.72m ~ 1.000
_SubtreeDataDeserializationWithBufferSizes/64KB-4 35.82m 35.55m ~ 1.000
_SubtreeDataDeserializationWithBufferSizes/128KB-4 35.98m 35.71m ~ 0.100
_SubtreeDataDeserializationWithBufferSizes/512KB-4 35.58m 36.12m ~ 0.400
_PooledVsNonPooled/Pooled-4 739.0n 741.3n ~ 0.400
_PooledVsNonPooled/NonPooled-4 7.383µ 7.497µ ~ 0.200
_MemoryFootprint/Current_512KB_32concurrent-4 6.651µ 6.894µ ~ 0.100
_MemoryFootprint/Proposed_32KB_32concurrent-4 9.575µ 11.043µ ~ 0.100
_MemoryFootprint/Alternative_64KB_32concurrent-4 9.383µ 9.767µ ~ 0.100
_prepareTxsPerLevel-4 415.7m 412.8m ~ 0.700
_prepareTxsPerLevelOrdered-4 3.563m 3.602m ~ 1.000
_prepareTxsPerLevel_Comparison/Original-4 418.6m 422.6m ~ 0.700
_prepareTxsPerLevel_Comparison/Optimized-4 3.585m 3.746m ~ 0.400
SubtreeSizes/10k_tx_4_per_subtree-4 1.374m 1.402m ~ 0.100
SubtreeSizes/10k_tx_16_per_subtree-4 328.3µ 326.5µ ~ 1.000
SubtreeSizes/10k_tx_64_per_subtree-4 78.07µ 77.87µ ~ 1.000
SubtreeSizes/10k_tx_256_per_subtree-4 19.61µ 19.40µ ~ 0.700
SubtreeSizes/10k_tx_512_per_subtree-4 9.642µ 9.649µ ~ 0.400
SubtreeSizes/10k_tx_1024_per_subtree-4 4.788µ 4.812µ ~ 1.000
SubtreeSizes/10k_tx_2k_per_subtree-4 2.398µ 2.398µ ~ 1.000
BlockSizeScaling/10k_tx_64_per_subtree-4 77.58µ 76.21µ ~ 0.400
BlockSizeScaling/10k_tx_256_per_subtree-4 19.24µ 19.41µ ~ 0.700
BlockSizeScaling/10k_tx_1024_per_subtree-4 4.845µ 4.813µ ~ 1.000
BlockSizeScaling/50k_tx_64_per_subtree-4 404.4µ 408.7µ ~ 0.400
BlockSizeScaling/50k_tx_256_per_subtree-4 96.89µ 96.39µ ~ 0.400
BlockSizeScaling/50k_tx_1024_per_subtree-4 23.88µ 23.65µ ~ 0.400
SubtreeAllocations/small_subtrees_exists_check-4 164.4µ 164.5µ ~ 1.000
SubtreeAllocations/small_subtrees_data_fetch-4 168.3µ 168.1µ ~ 1.000
SubtreeAllocations/small_subtrees_full_validation-4 336.6µ 337.6µ ~ 1.000
SubtreeAllocations/medium_subtrees_exists_check-4 9.726µ 9.653µ ~ 0.100
SubtreeAllocations/medium_subtrees_data_fetch-4 9.983µ 9.908µ ~ 0.400
SubtreeAllocations/medium_subtrees_full_validation-4 19.44µ 19.62µ ~ 0.400
SubtreeAllocations/large_subtrees_exists_check-4 2.334µ 2.316µ ~ 0.400
SubtreeAllocations/large_subtrees_data_fetch-4 2.430µ 2.401µ ~ 0.700
SubtreeAllocations/large_subtrees_full_validation-4 4.927µ 4.880µ ~ 0.500
StoreBlock_Sequential/BelowCSVHeight-4 253.1µ 251.8µ ~ 0.700
StoreBlock_Sequential/AboveCSVHeight-4 243.2µ 250.8µ ~ 0.400
GetUtxoHashes-4 258.2n 262.4n ~ 0.700
GetUtxoHashes_ManyOutputs-4 44.53µ 44.90µ ~ 0.400
_NewMetaDataFromBytes-4 241.9n 239.6n ~ 0.700
_Bytes-4 628.7n 639.1n ~ 0.700
_MetaBytes-4 580.4n 577.2n ~ 1.000

Threshold: >10% with p < 0.05 | Generated: 2026-05-14 08:35 UTC

…entation

gofmt/gci flagged the numbered list in the repro test's doc comment for
non-canonical indentation (3-space vs 2-space). Reformat to gofmt's
canonical list shape so CI passes.
@sonarqubecloud

Copy link
Copy Markdown

@liam liam requested review from freemans13 and sugh01 May 14, 2026 08:51
@liam liam merged commit 6a369d9 into bsv-blockchain:main May 14, 2026
25 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants