Skip to content

fix(blockassembly): fall back to row-oriented AddTxBatch on Unimplemented#897

Merged
icellan merged 2 commits into
bsv-blockchain:mainfrom
icellan:feat/blockassembly-columnar-fallback
May 19, 2026
Merged

fix(blockassembly): fall back to row-oriented AddTxBatch on Unimplemented#897
icellan merged 2 commits into
bsv-blockchain:mainfrom
icellan:feat/blockassembly-columnar-fallback

Conversation

@icellan

@icellan icellan commented May 19, 2026

Copy link
Copy Markdown
Contributor

Split out of #828 so it can be reviewed independently.

Summary

During a rolling deploy where one block-assembly replica has been updated to expose AddTxBatchColumnar (introduced in #889) and another has not, gRPC returns codes.Unimplemented from the older server. Before this change that error propagated as a hard failure and the batch was dropped. With this change the client transparently retries the same batch via the existing row-oriented sendBatchRowOriented path. Other errors continue to surface to the caller untouched.

Cost

  • One wasted RPC round-trip per batch against a persistently-old server. Harmless during the deploy window; immediately moot once all replicas are upgraded.

Follow-up (not in this PR)

Test plan

  • go build ./... clean
  • go vet ./services/blockassembly/... clean
  • make lint-new clean
  • Manual: verify a server that returns codes.Unimplemented for AddTxBatchColumnar causes the client to drop down to the row path with no error surfaced; verify a server that returns any OTHER gRPC code still surfaces the error.

…nted

During a rolling deploy where one block-assembly replica has been
updated to expose AddTxBatchColumnar (bsv-blockchain#889) and another has not,
gRPC returns codes.Unimplemented from the older server. Before this
fix that propagated as a hard error and dropped the batch. Now the
client transparently retries the same batch via the row-oriented
sendBatchRowOriented path. Other errors continue to surface to the
caller.

Cost: one wasted RPC round-trip per batch against a persistently-
old server; harmless during the deploy window and immediately moot
once all replicas are upgraded. No metric or sticky disable is
added in this commit — flagged in bsv-blockchain#828 review for a follow-up.

Surfaced as a follow-up split out of PR bsv-blockchain#828; reviewable on its own.
@github-actions

github-actions Bot commented May 19, 2026

Copy link
Copy Markdown
Contributor

🤖 Claude Code Review

Status: Complete


Current Review:

  • Test coverage gap remains for the Unimplemented fallback path (see inline comment)

History:

  • ✅ Fixed: Debug logging added for fallback scenario (line 469)

The implementation is correct and follows the project's error handling patterns. The fallback logic properly distinguishes between Unimplemented (graceful fallback) and other errors (propagate to caller). Debug logging provides visibility during rolling deploys. Consider adding test coverage for the fallback path as noted in the inline comment.

Comment thread services/blockassembly/Client.go
Comment thread services/blockassembly/Client.go
@icellan icellan requested review from freemans13 and oskarszoon May 19, 2026 13:13
@github-actions

github-actions Bot commented May 19, 2026

Copy link
Copy Markdown
Contributor

Benchmark Comparison Report

Baseline: main (unknown)

Current: PR-897 (fa4011c)

Summary

  • Regressions: 0
  • Improvements: 0
  • Unchanged: 144
  • Significance level: p < 0.05
All benchmark results (sec/op)
Benchmark Baseline Current Change p-value
_NewBlockFromBytes-4 1.664µ 1.690µ ~ 0.100
SplitSyncedParentMap_SetIfNotExists/256_buckets-4 61.81n 61.76n ~ 0.700
SplitSyncedParentMap_SetIfNotExists/16_buckets-4 61.90n 61.82n ~ 1.000
SplitSyncedParentMap_SetIfNotExists/1_bucket-4 61.83n 61.61n ~ 0.200
SplitSyncedParentMap_ConcurrentSetIfNotExists/256_buckets... 29.11n 29.38n ~ 0.700
SplitSyncedParentMap_ConcurrentSetIfNotExists/16_buckets_... 50.50n 49.60n ~ 0.100
SplitSyncedParentMap_ConcurrentSetIfNotExists/1_bucket_pa... 106.5n 107.6n ~ 0.800
MiningCandidate_Stringify_Short-4 262.3n 265.1n ~ 0.600
MiningCandidate_Stringify_Long-4 1.883µ 1.890µ ~ 0.400
MiningSolution_Stringify-4 982.1n 987.3n ~ 0.100
BlockInfo_MarshalJSON-4 1.759µ 1.751µ ~ 0.200
NewFromBytes-4 121.3n 122.4n ~ 0.500
AddTxBatchColumnar_Validation-4 2.537µ 2.680µ ~ 0.700
OffsetValidationLoop-4 545.2n 550.3n ~ 0.700
Mine_EasyDifficulty-4 67.54µ 67.20µ ~ 0.700
Mine_WithAddress-4 6.986µ 6.962µ ~ 0.400
BlockAssembler_AddTx-4 0.02847n 0.02641n ~ 0.700
AddNode-4 12.23 11.69 ~ 0.400
AddNodeWithMap-4 12.92 12.17 ~ 0.700
DirectSubtreeAdd/4_per_subtree-4 58.77n 58.92n ~ 1.000
DirectSubtreeAdd/64_per_subtree-4 31.76n 31.66n ~ 0.100
DirectSubtreeAdd/256_per_subtree-4 30.42n 30.72n ~ 0.100
DirectSubtreeAdd/1024_per_subtree-4 29.21n 29.14n ~ 0.400
DirectSubtreeAdd/2048_per_subtree-4 28.67n 28.71n ~ 1.000
SubtreeProcessorAdd/4_per_subtree-4 285.6n 281.2n ~ 0.400
SubtreeProcessorAdd/64_per_subtree-4 280.3n 279.5n ~ 0.700
SubtreeProcessorAdd/256_per_subtree-4 282.3n 280.0n ~ 1.000
SubtreeProcessorAdd/1024_per_subtree-4 271.7n 272.6n ~ 0.200
SubtreeProcessorAdd/2048_per_subtree-4 270.9n 270.3n ~ 0.700
SubtreeProcessorRotate/4_per_subtree-4 277.0n 276.0n ~ 0.100
SubtreeProcessorRotate/64_per_subtree-4 274.2n 274.8n ~ 0.400
SubtreeProcessorRotate/256_per_subtree-4 273.2n 275.3n ~ 0.400
SubtreeProcessorRotate/1024_per_subtree-4 276.0n 276.0n ~ 0.800
SubtreeNodeAddOnly/4_per_subtree-4 54.69n 54.40n ~ 1.000
SubtreeNodeAddOnly/64_per_subtree-4 34.43n 34.36n ~ 0.700
SubtreeNodeAddOnly/256_per_subtree-4 33.37n 33.35n ~ 1.000
SubtreeNodeAddOnly/1024_per_subtree-4 32.81n 32.79n ~ 0.700
SubtreeCreationOnly/4_per_subtree-4 115.0n 115.9n ~ 0.400
SubtreeCreationOnly/64_per_subtree-4 407.8n 412.1n ~ 0.200
SubtreeCreationOnly/256_per_subtree-4 1.384µ 1.387µ ~ 0.400
SubtreeCreationOnly/1024_per_subtree-4 4.510µ 4.389µ ~ 0.200
SubtreeCreationOnly/2048_per_subtree-4 8.361µ 8.327µ ~ 0.200
SubtreeProcessorOverheadBreakdown/64_per_subtree-4 274.9n 275.6n ~ 0.400
SubtreeProcessorOverheadBreakdown/1024_per_subtree-4 273.7n 274.2n ~ 1.000
ParallelGetAndSetIfNotExists/1k_nodes-4 2.206m 2.230m ~ 0.100
ParallelGetAndSetIfNotExists/10k_nodes-4 5.383m 5.628m ~ 0.100
ParallelGetAndSetIfNotExists/50k_nodes-4 7.426m 7.737m ~ 0.100
ParallelGetAndSetIfNotExists/100k_nodes-4 10.19m 10.23m ~ 0.200
SequentialGetAndSetIfNotExists/1k_nodes-4 1.940m 1.986m ~ 0.100
SequentialGetAndSetIfNotExists/10k_nodes-4 4.373m 4.364m ~ 0.400
SequentialGetAndSetIfNotExists/50k_nodes-4 12.36m 12.29m ~ 0.700
SequentialGetAndSetIfNotExists/100k_nodes-4 21.86m 22.01m ~ 0.200
ProcessOwnBlockSubtreeNodesParallel/1k_nodes-4 2.267m 2.272m ~ 1.000
ProcessOwnBlockSubtreeNodesParallel/10k_nodes-4 8.312m 8.403m ~ 0.400
ProcessOwnBlockSubtreeNodesParallel/100k_nodes-4 13.35m 13.41m ~ 0.700
ProcessOwnBlockSubtreeNodesSequential/1k_nodes-4 1.971m 1.990m ~ 0.400
ProcessOwnBlockSubtreeNodesSequential/10k_nodes-4 7.721m 7.600m ~ 1.000
ProcessOwnBlockSubtreeNodesSequential/100k_nodes-4 39.86m 39.90m ~ 1.000
DiskTxMap_SetIfNotExists-4 3.946µ 3.901µ ~ 0.700
DiskTxMap_SetIfNotExists_Parallel-4 3.664µ 3.487µ ~ 0.100
DiskTxMap_ExistenceOnly-4 422.8n 495.5n ~ 0.700
Queue-4 188.2n 187.7n ~ 0.400
AtomicPointer-4 3.239n 3.252n ~ 0.200
ReorgOptimizations/DedupFilterPipeline/Old/10K-4 799.5µ 815.5µ ~ 0.200
ReorgOptimizations/DedupFilterPipeline/New/10K-4 766.4µ 773.9µ ~ 0.200
ReorgOptimizations/AllMarkFalse/Old/10K-4 105.3µ 103.5µ ~ 0.100
ReorgOptimizations/AllMarkFalse/New/10K-4 65.37µ 64.21µ ~ 0.100
ReorgOptimizations/HashSlicePool/Old/10K-4 63.84µ 53.08µ ~ 0.100
ReorgOptimizations/HashSlicePool/New/10K-4 11.28µ 11.25µ ~ 1.000
ReorgOptimizations/NodeFlags/Old/10K-4 5.278µ 4.453µ ~ 0.100
ReorgOptimizations/NodeFlags/New/10K-4 2.018µ 1.471µ ~ 0.100
ReorgOptimizations/DedupFilterPipeline/Old/100K-4 9.643m 9.269m ~ 0.100
ReorgOptimizations/DedupFilterPipeline/New/100K-4 10.528m 9.942m ~ 0.100
ReorgOptimizations/AllMarkFalse/Old/100K-4 1.102m 1.077m ~ 0.100
ReorgOptimizations/AllMarkFalse/New/100K-4 707.1µ 707.6µ ~ 0.700
ReorgOptimizations/HashSlicePool/Old/100K-4 491.0µ 475.8µ ~ 0.100
ReorgOptimizations/HashSlicePool/New/100K-4 214.5µ 216.3µ ~ 0.400
ReorgOptimizations/NodeFlags/Old/100K-4 48.84µ 47.12µ ~ 0.100
ReorgOptimizations/NodeFlags/New/100K-4 17.07µ 16.04µ ~ 0.100
TxMapSetIfNotExists-4 50.02n 49.99n ~ 0.700
TxMapSetIfNotExistsDuplicate-4 41.23n 41.60n ~ 0.100
ChannelSendReceive-4 597.3n 602.1n ~ 1.000
CalcBlockWork-4 529.7n 493.0n ~ 0.100
CalculateWork-4 741.1n 688.8n ~ 0.700
BuildBlockLocatorString_Helpers/Size_10-4 1.343µ 1.359µ ~ 0.100
BuildBlockLocatorString_Helpers/Size_100-4 12.93µ 12.95µ ~ 1.000
BuildBlockLocatorString_Helpers/Size_1000-4 138.4µ 158.9µ ~ 0.400
CatchupWithHeaderCache-4 104.4m 104.4m ~ 1.000
_BufferPoolAllocation/16KB-4 4.587µ 3.816µ ~ 0.700
_BufferPoolAllocation/32KB-4 7.225µ 7.426µ ~ 0.100
_BufferPoolAllocation/64KB-4 17.21µ 21.44µ ~ 0.700
_BufferPoolAllocation/128KB-4 29.53µ 26.57µ ~ 1.000
_BufferPoolAllocation/512KB-4 107.73µ 94.92µ ~ 0.100
_BufferPoolConcurrent/32KB-4 17.80µ 18.10µ ~ 0.700
_BufferPoolConcurrent/64KB-4 27.93µ 27.85µ ~ 1.000
_BufferPoolConcurrent/512KB-4 138.8µ 144.3µ ~ 0.200
_SubtreeDeserializationWithBufferSizes/16KB-4 608.8µ 630.8µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/32KB-4 604.1µ 634.5µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/64KB-4 600.8µ 624.0µ ~ 0.200
_SubtreeDeserializationWithBufferSizes/128KB-4 583.4µ 616.5µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/512KB-4 600.9µ 622.7µ ~ 0.700
_SubtreeDataDeserializationWithBufferSizes/16KB-4 36.70m 36.85m ~ 0.700
_SubtreeDataDeserializationWithBufferSizes/32KB-4 36.84m 36.98m ~ 1.000
_SubtreeDataDeserializationWithBufferSizes/64KB-4 36.79m 36.80m ~ 1.000
_SubtreeDataDeserializationWithBufferSizes/128KB-4 36.75m 36.40m ~ 0.200
_SubtreeDataDeserializationWithBufferSizes/512KB-4 36.62m 36.37m ~ 1.000
_PooledVsNonPooled/Pooled-4 831.0n 834.0n ~ 0.100
_PooledVsNonPooled/NonPooled-4 7.063µ 7.419µ ~ 0.100
_MemoryFootprint/Current_512KB_32concurrent-4 6.613µ 6.656µ ~ 0.400
_MemoryFootprint/Proposed_32KB_32concurrent-4 9.232µ 9.334µ ~ 0.400
_MemoryFootprint/Alternative_64KB_32concurrent-4 8.933µ 8.930µ ~ 0.700
_prepareTxsPerLevel-4 409.9m 410.7m ~ 1.000
_prepareTxsPerLevelOrdered-4 3.662m 3.849m ~ 0.100
_prepareTxsPerLevel_Comparison/Original-4 406.2m 398.7m ~ 0.700
_prepareTxsPerLevel_Comparison/Optimized-4 3.808m 3.785m ~ 1.000
SubtreeSizes/10k_tx_4_per_subtree-4 1.253m 1.291m ~ 0.700
SubtreeSizes/10k_tx_16_per_subtree-4 293.4µ 302.7µ ~ 0.200
SubtreeSizes/10k_tx_64_per_subtree-4 70.95µ 70.90µ ~ 1.000
SubtreeSizes/10k_tx_256_per_subtree-4 17.62µ 17.74µ ~ 0.200
SubtreeSizes/10k_tx_512_per_subtree-4 8.727µ 8.801µ ~ 0.400
SubtreeSizes/10k_tx_1024_per_subtree-4 4.303µ 4.340µ ~ 0.400
SubtreeSizes/10k_tx_2k_per_subtree-4 2.149µ 2.165µ ~ 1.000
BlockSizeScaling/10k_tx_64_per_subtree-4 68.89µ 69.60µ ~ 0.200
BlockSizeScaling/10k_tx_256_per_subtree-4 17.33µ 17.51µ ~ 0.400
BlockSizeScaling/10k_tx_1024_per_subtree-4 4.342µ 4.305µ ~ 0.100
BlockSizeScaling/50k_tx_64_per_subtree-4 367.6µ 370.1µ ~ 0.400
BlockSizeScaling/50k_tx_256_per_subtree-4 87.26µ 87.15µ ~ 1.000
BlockSizeScaling/50k_tx_1024_per_subtree-4 21.90µ 21.47µ ~ 0.400
SubtreeAllocations/small_subtrees_exists_check-4 151.4µ 149.9µ ~ 0.100
SubtreeAllocations/small_subtrees_data_fetch-4 160.0µ 158.9µ ~ 0.100
SubtreeAllocations/small_subtrees_full_validation-4 308.8µ 310.1µ ~ 1.000
SubtreeAllocations/medium_subtrees_exists_check-4 8.892µ 8.842µ ~ 1.000
SubtreeAllocations/medium_subtrees_data_fetch-4 9.426µ 9.372µ ~ 0.700
SubtreeAllocations/medium_subtrees_full_validation-4 17.45µ 17.49µ ~ 1.000
SubtreeAllocations/large_subtrees_exists_check-4 2.103µ 2.102µ ~ 0.800
SubtreeAllocations/large_subtrees_data_fetch-4 2.253µ 2.211µ ~ 0.400
SubtreeAllocations/large_subtrees_full_validation-4 4.414µ 4.330µ ~ 0.100
StoreBlock_Sequential/BelowCSVHeight-4 333.6µ 325.7µ ~ 0.700
StoreBlock_Sequential/AboveCSVHeight-4 327.0µ 327.9µ ~ 0.400
GetUtxoHashes-4 259.1n 257.7n ~ 0.400
GetUtxoHashes_ManyOutputs-4 42.18µ 41.98µ ~ 0.700
_NewMetaDataFromBytes-4 231.7n 228.4n ~ 0.100
_Bytes-4 398.9n 398.2n ~ 1.000
_MetaBytes-4 342.8n 338.2n ~ 0.200

Threshold: >10% with p < 0.05 | Generated: 2026-05-19 13:44 UTC

A rolling-deploy mismatch where one block-assembly replica predates bsv-blockchain#889
silently drops to the row path; without a log line operators
investigating "why is this batch slow" can't tell the fallback fired.
Debug level keeps it out of normal log volume but available when
needed.
@icellan

icellan commented May 19, 2026

Copy link
Copy Markdown
Contributor Author

Addressed the observability suggestion in 8f991f6: added s.logger.Debugf on the Unimplemented fallback path so operators can see when the row fallback fires during rolling deploys.

Skipping the test suggestion for this PR — Client_test.go has no existing mock-gRPC scaffolding for sendBatchColumnar, and the unit needed to test this path is the gRPC client mock, which is invasive to add for a 14-line change. Happy to add it as a separate follow-up if a reviewer wants it.

@sonarqubecloud

Copy link
Copy Markdown

@icellan icellan merged commit fff6d3e into bsv-blockchain:main May 19, 2026
25 checks passed
@icellan icellan self-assigned this May 19, 2026
icellan added a commit that referenced this pull request May 19, 2026
Brings in fff6d3e..origin/main (4 commits since previous merge):
  - #897 blockassembly columnar->row fallback on Unimplemented
  - #896 asset on-demand subtree-data admission control + 503/Retry-After
  - #766 utxo: verify spend ownership in Unspend
  - #700 health: disk space check

Conflict resolutions:
  - services/blockassembly/Client.go, util/http.go, util/http_test.go:
    take theirs. Main has the post-Copilot-fix versions of these files
    from #896 and #897 (parseRetryAfter strict-Atoi, debug log on
    columnar fallback, extended TestParseRetryAfter cases) — they
    supersede the pre-fix versions still living on pr-828.
  - stores/utxo/aerospike/un_spend.go: take ours (native-op call,
    no SpendingData arg). The ownership check #766 added to the UDF
    path requires a coordinated update to the BSV-forked aerospike-
    server's subOpUnspend dispatcher. Until that lands, the
    native-op path on this branch skips ownership verification.
    Tracking issue filed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants