Skip to content

chore(utxopersister): clarify footer-read match per #985 review#994

Merged
liam merged 1 commit into
bsv-blockchain:mainfrom
liam:liam/utxopersister-footer-review-followup
Jun 1, 2026
Merged

chore(utxopersister): clarify footer-read match per #985 review#994
liam merged 1 commit into
bsv-blockchain:mainfrom
liam:liam/utxopersister-footer-review-followup

Conversation

@liam

@liam liam commented May 31, 2026

Copy link
Copy Markdown
Collaborator

Follow-up to #985, addressing review comments that merged before they could be applied. No behavior change — purely clarity plus one stale-comment fix.

The reviewer correctly pointed out that the match in #985 is not the clean structural check it appeared to be:

  • "carries no record count" was wrong. The 16-byte footer is txCount + utxoCount. The loop just doesn't consult it. Reworded.
  • errors.Is was masquerading as structural. teranode's errors.New flattens a non-*Error cause to its message string (errors/errors.go:334-336), discarding the io.ErrUnexpectedEOF sentinel — so errors.Is(err, io.ErrUnexpectedEOF) reduces to strings.Contains(err.Error(), "unexpected EOF"). Switched to the explicit strings.Contains so the substring nature is visible in the code, with a comment explaining it. Also documented the footgun the reviewer flagged: errors.Is(err, io.EOF) would also match this footer error ("EOF""unexpected EOF"), so the io.EOF clause stays a pointer comparison.
  • Documented the limitation and the structural follow-up. A genuinely truncated tail is indistinguishable from the footer and silently accepted (pre-existing; same as cmd/utxovalidator). The real structural fix — FromReader returning a typed sentinel, plus validating records-read == txCount across the consolidation loop, the seeder, and utxovalidator — is noted as a follow-up.
  • Fixed the stale // Read the next 36 bytes... comment (a txid is 32 bytes; pre-existing, sat right above the change).

Verification

  • go test ./services/utxopersister/ — pass
  • go vet, gofmt, gci — clean

…review

Addresses review comments on bsv-blockchain#985 that merged before they were applied.
No behavior change.

- The previous comment claimed the wrapper stream 'carries no record
  count'; it does - the 16-byte footer is txCount + utxoCount. Reworded
  to say the loop simply does not consult it.
- The match read as a clean structural errors.Is, but teranode's
  errors.New flattens a non-*Error cause to its message string
  (errors/errors.go:334-336), so errors.Is(err, io.ErrUnexpectedEOF)
  degrades to strings.Contains(err.Error(), "unexpected EOF"). Switched
  to the explicit strings.Contains so the substring nature is visible,
  and documented why - including that errors.Is(err, io.EOF) would also
  match this footer error ("EOF" is a substring of "unexpected EOF"),
  so the io.EOF clause stays a pointer compare.
- Noted the silent-truncation limitation and the structural follow-up
  (typed sentinel + records-read == txCount validation).
- Fixed the stale 'Read the next 36 bytes...' comment (a txid is 32).
@github-actions

github-actions Bot commented May 31, 2026

Copy link
Copy Markdown
Contributor

🤖 Claude Code Review

Status: Complete

Current Review:

  • Minor documentation inaccuracy at services/utxopersister/UTXOSet.go:659 — comment claims behavior is "same as utxovalidator" but this code is actually more restrictive (intentionally). See inline comment for details.

This is a documentation and clarity improvement PR with no behavior change. The code changes are sound:

  • Fixed stale comment (txid is 32 bytes, not 36)
  • Switched from errors.Is to explicit strings.Contains making the substring matching visible
  • Accurately documented the error-flattening behavior in the custom errors package

The one issue found is a minor documentation claim that contradicts the actual utxovalidator implementation.

//
// Consequence: a genuinely truncated tail is
// indistinguishable from the footer and is silently
// accepted (pre-existing; same as utxovalidator). Matching

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor] Documentation inaccuracy — The comment states this code behaves "same as utxovalidator" (line 659) and claims it matches "only 'unexpected EOF' - not the broader 'failed to read txid' utxovalidator also matches" (lines 660-662).

However, cmd/utxovalidator/utxo_validator.go:127 matches BOTH patterns:

if strings.Contains(errStr, "failed to read txid") || strings.Contains(errStr, "unexpected EOF") {

This code is actually more restrictive than utxovalidator (intentionally, for better error visibility per the comment). The documentation should reflect this difference rather than claiming equivalence.

Suggested fix: Replace "same as utxovalidator" with "like utxovalidator, though this matches only the footer-specific error for better diagnostics".

@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

Benchmark Comparison Report

Baseline: main (unknown)

Current: PR-994 (ff76ee1)

Summary

  • Regressions: 0
  • Improvements: 0
  • Unchanged: 144
  • Significance level: p < 0.05
All benchmark results (sec/op)
Benchmark Baseline Current Change p-value
_NewBlockFromBytes-4 1.635µ 1.627µ ~ 1.000
SplitSyncedParentMap_SetIfNotExists/256_buckets-4 71.19n 71.24n ~ 0.700
SplitSyncedParentMap_SetIfNotExists/16_buckets-4 71.16n 71.11n ~ 0.700
SplitSyncedParentMap_SetIfNotExists/1_bucket-4 71.15n 71.06n ~ 0.700
SplitSyncedParentMap_ConcurrentSetIfNotExists/256_buckets... 32.78n 33.05n ~ 0.700
SplitSyncedParentMap_ConcurrentSetIfNotExists/16_buckets_... 55.64n 55.19n ~ 0.200
SplitSyncedParentMap_ConcurrentSetIfNotExists/1_bucket_pa... 127.6n 127.8n ~ 0.700
MiningCandidate_Stringify_Short-4 219.0n 219.9n ~ 0.400
MiningCandidate_Stringify_Long-4 1.634µ 1.640µ ~ 0.100
MiningSolution_Stringify-4 845.6n 848.8n ~ 0.500
BlockInfo_MarshalJSON-4 1.722µ 1.718µ ~ 0.700
NewFromBytes-4 125.0n 124.4n ~ 0.700
AddTxBatchColumnar_Validation-4 2.642µ 2.522µ ~ 1.000
OffsetValidationLoop-4 545.7n 544.7n ~ 0.700
Mine_EasyDifficulty-4 65.03µ 64.77µ ~ 0.200
Mine_WithAddress-4 6.858µ 6.848µ ~ 1.000
DirectSubtreeAdd/4_per_subtree-4 62.72n 58.22n ~ 0.700
DirectSubtreeAdd/64_per_subtree-4 29.73n 29.78n ~ 0.300
DirectSubtreeAdd/256_per_subtree-4 28.78n 28.76n ~ 1.000
DirectSubtreeAdd/1024_per_subtree-4 27.76n 27.78n ~ 0.700
DirectSubtreeAdd/2048_per_subtree-4 27.37n 27.33n ~ 0.700
SubtreeProcessorAdd/4_per_subtree-4 283.4n 284.7n ~ 1.000
SubtreeProcessorAdd/64_per_subtree-4 277.2n 279.0n ~ 1.000
SubtreeProcessorAdd/256_per_subtree-4 278.5n 279.0n ~ 0.700
SubtreeProcessorAdd/1024_per_subtree-4 270.1n 267.5n ~ 0.100
SubtreeProcessorAdd/2048_per_subtree-4 270.6n 268.3n ~ 0.700
SubtreeProcessorRotate/4_per_subtree-4 277.1n 278.4n ~ 0.100
SubtreeProcessorRotate/64_per_subtree-4 274.1n 273.5n ~ 0.800
SubtreeProcessorRotate/256_per_subtree-4 275.1n 272.6n ~ 0.200
SubtreeProcessorRotate/1024_per_subtree-4 272.0n 273.5n ~ 0.400
SubtreeNodeAddOnly/4_per_subtree-4 54.40n 54.09n ~ 0.700
SubtreeNodeAddOnly/64_per_subtree-4 34.37n 34.14n ~ 0.100
SubtreeNodeAddOnly/256_per_subtree-4 33.19n 33.17n ~ 1.000
SubtreeNodeAddOnly/1024_per_subtree-4 32.55n 32.50n ~ 0.700
SubtreeCreationOnly/4_per_subtree-4 113.0n 114.2n ~ 0.100
SubtreeCreationOnly/64_per_subtree-4 398.2n 399.8n ~ 0.700
SubtreeCreationOnly/256_per_subtree-4 1.339µ 1.360µ ~ 0.100
SubtreeCreationOnly/1024_per_subtree-4 4.436µ 4.483µ ~ 0.400
SubtreeCreationOnly/2048_per_subtree-4 7.996µ 8.116µ ~ 0.100
SubtreeProcessorOverheadBreakdown/64_per_subtree-4 271.3n 270.7n ~ 1.000
SubtreeProcessorOverheadBreakdown/1024_per_subtree-4 271.4n 270.5n ~ 0.100
ParallelGetAndSetIfNotExists/1k_nodes-4 2.177m 2.193m ~ 0.400
ParallelGetAndSetIfNotExists/10k_nodes-4 5.345m 5.371m ~ 0.100
ParallelGetAndSetIfNotExists/50k_nodes-4 7.374m 7.283m ~ 0.200
ParallelGetAndSetIfNotExists/100k_nodes-4 10.38m 10.01m ~ 0.100
SequentialGetAndSetIfNotExists/1k_nodes-4 1.938m 1.925m ~ 0.400
SequentialGetAndSetIfNotExists/10k_nodes-4 4.434m 4.363m ~ 0.400
SequentialGetAndSetIfNotExists/50k_nodes-4 13.18m 12.65m ~ 0.200
SequentialGetAndSetIfNotExists/100k_nodes-4 23.43m 22.98m ~ 0.200
ProcessOwnBlockSubtreeNodesParallel/1k_nodes-4 2.219m 2.239m ~ 0.700
ProcessOwnBlockSubtreeNodesParallel/10k_nodes-4 8.244m 8.276m ~ 0.700
ProcessOwnBlockSubtreeNodesParallel/100k_nodes-4 13.06m 13.20m ~ 0.700
ProcessOwnBlockSubtreeNodesSequential/1k_nodes-4 1.972m 1.970m ~ 0.700
ProcessOwnBlockSubtreeNodesSequential/10k_nodes-4 7.449m 7.514m ~ 0.100
ProcessOwnBlockSubtreeNodesSequential/100k_nodes-4 39.35m 39.85m ~ 0.100
DiskTxMap_SetIfNotExists-4 3.506µ 3.685µ ~ 0.700
DiskTxMap_SetIfNotExists_Parallel-4 3.419µ 3.398µ ~ 1.000
DiskTxMap_ExistenceOnly-4 316.7n 314.5n ~ 0.400
Queue-4 193.7n 193.5n ~ 0.400
AtomicPointer-4 4.447n 4.911n ~ 0.200
ReorgOptimizations/DedupFilterPipeline/Old/10K-4 904.5µ 883.4µ ~ 0.100
ReorgOptimizations/DedupFilterPipeline/New/10K-4 808.5µ 836.5µ ~ 0.100
ReorgOptimizations/AllMarkFalse/Old/10K-4 113.1µ 108.4µ ~ 0.100
ReorgOptimizations/AllMarkFalse/New/10K-4 62.35µ 61.89µ ~ 0.400
ReorgOptimizations/HashSlicePool/Old/10K-4 60.90µ 58.91µ ~ 0.400
ReorgOptimizations/HashSlicePool/New/10K-4 12.03µ 12.34µ ~ 0.700
ReorgOptimizations/NodeFlags/Old/10K-4 4.864µ 4.799µ ~ 0.700
ReorgOptimizations/NodeFlags/New/10K-4 1.665µ 1.638µ ~ 0.200
ReorgOptimizations/DedupFilterPipeline/Old/100K-4 9.732m 9.530m ~ 0.700
ReorgOptimizations/DedupFilterPipeline/New/100K-4 10.496m 9.820m ~ 0.200
ReorgOptimizations/AllMarkFalse/Old/100K-4 1.148m 1.079m ~ 0.100
ReorgOptimizations/AllMarkFalse/New/100K-4 687.8µ 683.7µ ~ 0.100
ReorgOptimizations/HashSlicePool/Old/100K-4 610.4µ 681.9µ ~ 0.100
ReorgOptimizations/HashSlicePool/New/100K-4 309.7µ 285.7µ ~ 0.400
ReorgOptimizations/NodeFlags/Old/100K-4 49.55µ 49.96µ ~ 1.000
ReorgOptimizations/NodeFlags/New/100K-4 17.20µ 17.20µ ~ 1.000
TxMapSetIfNotExists-4 52.40n 52.19n ~ 0.400
TxMapSetIfNotExistsDuplicate-4 40.55n 40.46n ~ 0.400
ChannelSendReceive-4 601.8n 597.7n ~ 0.100
BlockAssembler_AddTx-4 0.03000n 0.02914n ~ 1.000
AddNode-4 11.94 11.80 ~ 1.000
AddNodeWithMap-4 11.40 11.85 ~ 0.700
CalcBlockWork-4 527.9n 522.7n ~ 0.700
CalculateWork-4 711.7n 722.1n ~ 0.700
BuildBlockLocatorString_Helpers/Size_10-4 1.340µ 1.342µ ~ 0.600
BuildBlockLocatorString_Helpers/Size_100-4 12.89µ 12.87µ ~ 0.700
BuildBlockLocatorString_Helpers/Size_1000-4 146.0µ 148.1µ ~ 1.000
CatchupWithHeaderCache-4 104.5m 104.4m ~ 1.000
_prepareTxsPerLevel-4 408.1m 411.8m ~ 0.100
_prepareTxsPerLevelOrdered-4 3.446m 3.535m ~ 0.400
_prepareTxsPerLevel_Comparison/Original-4 412.3m 407.2m ~ 0.200
_prepareTxsPerLevel_Comparison/Optimized-4 3.611m 3.521m ~ 0.400
_BufferPoolAllocation/16KB-4 4.012µ 3.982µ ~ 1.000
_BufferPoolAllocation/32KB-4 10.060µ 9.879µ ~ 1.000
_BufferPoolAllocation/64KB-4 16.45µ 16.58µ ~ 0.400
_BufferPoolAllocation/128KB-4 31.52µ 33.04µ ~ 0.100
_BufferPoolAllocation/512KB-4 120.2µ 104.4µ ~ 0.100
_BufferPoolConcurrent/32KB-4 19.85µ 19.62µ ~ 1.000
_BufferPoolConcurrent/64KB-4 30.17µ 31.90µ ~ 0.700
_BufferPoolConcurrent/512KB-4 159.5µ 156.0µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/16KB-4 706.7µ 680.7µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/32KB-4 734.0µ 673.7µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/64KB-4 730.3µ 678.6µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/128KB-4 735.1µ 666.3µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/512KB-4 644.0µ 674.1µ ~ 0.100
_SubtreeDataDeserializationWithBufferSizes/16KB-4 37.04m 37.29m ~ 0.400
_SubtreeDataDeserializationWithBufferSizes/32KB-4 37.23m 37.36m ~ 0.700
_SubtreeDataDeserializationWithBufferSizes/64KB-4 37.18m 37.11m ~ 1.000
_SubtreeDataDeserializationWithBufferSizes/128KB-4 37.26m 36.88m ~ 0.400
_SubtreeDataDeserializationWithBufferSizes/512KB-4 36.83m 36.99m ~ 0.700
_PooledVsNonPooled/Pooled-4 681.4n 842.3n ~ 0.100
_PooledVsNonPooled/NonPooled-4 8.024µ 8.358µ ~ 0.100
_MemoryFootprint/Current_512KB_32concurrent-4 7.403µ 7.906µ ~ 0.100
_MemoryFootprint/Proposed_32KB_32concurrent-4 12.35µ 12.86µ ~ 0.100
_MemoryFootprint/Alternative_64KB_32concurrent-4 10.75µ 10.86µ ~ 0.700
SubtreeSizes/10k_tx_4_per_subtree-4 1.474m 1.569m ~ 0.100
SubtreeSizes/10k_tx_16_per_subtree-4 342.4µ 354.9µ ~ 0.100
SubtreeSizes/10k_tx_64_per_subtree-4 82.47µ 84.30µ ~ 0.400
SubtreeSizes/10k_tx_256_per_subtree-4 20.80µ 20.61µ ~ 0.700
SubtreeSizes/10k_tx_512_per_subtree-4 10.25µ 10.29µ ~ 0.500
SubtreeSizes/10k_tx_1024_per_subtree-4 5.176µ 5.230µ ~ 0.700
SubtreeSizes/10k_tx_2k_per_subtree-4 2.611µ 2.624µ ~ 0.700
BlockSizeScaling/10k_tx_64_per_subtree-4 80.75µ 80.96µ ~ 0.400
BlockSizeScaling/10k_tx_256_per_subtree-4 20.85µ 21.01µ ~ 1.000
BlockSizeScaling/10k_tx_1024_per_subtree-4 5.163µ 5.144µ ~ 0.700
BlockSizeScaling/50k_tx_64_per_subtree-4 409.3µ 429.6µ ~ 1.000
BlockSizeScaling/50k_tx_256_per_subtree-4 102.5µ 104.8µ ~ 0.700
BlockSizeScaling/50k_tx_1024_per_subtree-4 25.74µ 25.60µ ~ 1.000
SubtreeAllocations/small_subtrees_exists_check-4 171.0µ 173.2µ ~ 0.700
SubtreeAllocations/small_subtrees_data_fetch-4 178.0µ 175.8µ ~ 1.000
SubtreeAllocations/small_subtrees_full_validation-4 340.9µ 340.6µ ~ 1.000
SubtreeAllocations/medium_subtrees_exists_check-4 10.34µ 10.37µ ~ 0.700
SubtreeAllocations/medium_subtrees_data_fetch-4 11.21µ 11.05µ ~ 0.200
SubtreeAllocations/medium_subtrees_full_validation-4 21.19µ 21.12µ ~ 0.400
SubtreeAllocations/large_subtrees_exists_check-4 2.568µ 2.555µ ~ 0.400
SubtreeAllocations/large_subtrees_data_fetch-4 2.771µ 2.745µ ~ 0.700
SubtreeAllocations/large_subtrees_full_validation-4 5.382µ 5.339µ ~ 0.700
StoreBlock_Sequential/BelowCSVHeight-4 333.2µ 332.7µ ~ 1.000
StoreBlock_Sequential/AboveCSVHeight-4 331.9µ 339.2µ ~ 0.100
GetUtxoHashes-4 263.2n 257.9n ~ 0.700
GetUtxoHashes_ManyOutputs-4 42.70µ 42.63µ ~ 1.000
_NewMetaDataFromBytes-4 229.4n 228.8n ~ 0.700
_Bytes-4 401.8n 400.1n ~ 0.700
_MetaBytes-4 138.3n 137.6n ~ 0.400

Threshold: >10% with p < 0.05 | Generated: 2026-05-31 19:17 UTC

@liam liam requested review from ordishs and oskarszoon June 1, 2026 09:13

@ordishs ordishs left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified no behavior change: traced both the old errors.Is(err, io.ErrUnexpectedEOF) and the new strings.Contains(err.Error(), "unexpected EOF") to the identical string comparison via the custom Error.Is substring fallback (errors/errors.go:154-164) and errors.New cause-flattening (errors/errors.go:330-333). Confirmed the bare io.EOF return path (UTXO.go:191-193) and the footer short-read message (UTXO.go:212-218). gofmt and go vet clean.

The swap makes the substring dependency explicit at the call site rather than hidden behind errors.Is, and the expanded comment correctly documents the brittleness and the structural follow-up. Approve.

@oskarszoon oskarszoon left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No P0/P1. Traced the equivalence end to end and it's exact: after errors.New flattens the io.ErrUnexpectedEOF cause to its message string (errors/errors.go:334-336), the old errors.Is(err, io.ErrUnexpectedEOF) resolves through (*Error).Is to strings.Contains(e.Error(), "unexpected EOF") — byte-identical to the new explicit call. The "no behavior change" claim holds, and the footer branch is pinned by TestCreateUTXOSet_PreviousSetWithFooterTerminatesCleanly. Verified the cmd/utxovalidator cross-reference too: matching only "unexpected EOF" here (vs utxovalidator's broader "failed to read txid") is the correct call — it keeps a real non-EOF read error loud instead of swallowing it. gofmt/vet/build clean. LGTM.

Nit, non-blocking: the errors/errors.go:334-336 line ref in the comment will rot on the next edit there — the symbol name carries it, so leave it or drop the line numbers.

@liam liam merged commit 44f163a into bsv-blockchain:main Jun 1, 2026
27 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants