Skip to content

toggle to allow raw display of miner tag as per WOC#441

Merged
freemans13 merged 13 commits into
bsv-blockchain:mainfrom
freemans13:stu/allow-raw-miner-tag
Jun 5, 2026
Merged

toggle to allow raw display of miner tag as per WOC#441
freemans13 merged 13 commits into
bsv-blockchain:mainfrom
freemans13:stu/allow-raw-miner-tag

Conversation

@freemans13

@freemans13 freemans13 commented Jan 28, 2026

Copy link
Copy Markdown
Collaborator

What this does

Adds a setting, blockchain_raw_miner_tag (default false), that turns off the cleanup Teranode normally applies to the miner tag read from a block's coinbase transaction.

This is for TAAL, who want to compare Teranode's miner tags against WhatsOnChain. WhatsOnChain shows the raw coinbase text; Teranode cleans it up, so the two don't line up.

Fixes https://github.com/bitcoin-sv/teranode/issues/1298

The problem

For mainnet block 514587:

  • Teranode showed: A% Zj?d?;?
  • WhatsOnChain showed: �^G^GA% Zj�d�;��

The difference is that Teranode strips out non-printable characters and trims the tag, while WhatsOnChain leaves the raw bytes in.

How it works

When blockchain_raw_miner_tag=false (the default), nothing changes. Miner tags are cleaned up exactly as before:

  • non-printable characters are removed
  • surrounding whitespace and quotes are trimmed
  • anything before the first / is dropped
  • anything after the second / is dropped

When blockchain_raw_miner_tag=true, the coinbase text is returned as-is, with no cleanup.

One thing to be aware of: JSON encoding

The miner tag is sent to clients as JSON (over the HTTP API and WebSocket notifications). Raw coinbase bytes are often not valid UTF-8, and Go's JSON encoder replaces any invalid bytes with the Unicode replacement character (). So clients do not get the exact original bytes back — they get a string where the invalid bytes have become .

That happens to be the same thing WhatsOnChain shows (its characters are the same replacement character). So this gets us close to WhatsOnChain's display, but it is not a byte-for-byte copy of the coinbase. If byte-exact data were ever needed we'd have to expose it as hex or base64 — but that wouldn't match WhatsOnChain, so it is deliberately not done here.

TestExtractCoinbaseMinerRawJSONRoundTrip pins this behaviour down so it can't change unnoticed.

Where the change lives

The setting only affects how the SQL blockchain store extracts the miner tag. Everything that displays a miner name (the HTTP block-header endpoints, and any WebSocket notification that carries it) gets its value from the store's block header metadata, so they all pick up the setting automatically. No other code needed changing.

Changing the setting

The store reads the setting once at startup, so changing blockchain_raw_miner_tag requires a restart to take effect.

Code changes

  • New setting blockchain_raw_miner_tag in BlockChainSettings (default false).
  • New function ExtractCoinbaseMinerRaw(coinbaseTx, raw bool) in util/coinbase.go. The existing ExtractCoinbaseMiner() is kept and now just calls the new function with raw=false, so existing callers are unchanged.
  • The SQL store reads the setting into s.rawMinerTag and passes it at every miner-extraction point: block_helpers.go, GetBlockHeader.go, GetBlockHeaders.go, GetBestBlockHeader.go, GetLatestHeaderFromBlockLocator.go.

Tests

  • TestExtractCoinbaseMinerRaw — checks raw vs sanitized output.
  • TestExtractCoinbaseMinerRawPreservesAllBytes — checks raw mode keeps null bytes, high bytes, and control characters, while sanitized mode strips them.
  • TestExtractCoinbaseMinerRawJSONRoundTrip — documents and pins the JSON replacement-character behaviour described above.
  • TestRawMinerTagSettingWiring (SQL store) — checks the setting is honoured end-to-end through the store.

How to turn it on

blockchain_raw_miner_tag=true

@freemans13 freemans13 self-assigned this Jan 28, 2026
@github-actions

github-actions Bot commented Jan 28, 2026

Copy link
Copy Markdown
Contributor

🤖 Claude Code Review

Status: Complete

No issues found. This PR implements a clean, well-tested feature with strong documentation.

Review Summary:

  • Implementation is minimal and focused (single boolean flag controlling sanitization behavior)
  • Backward compatibility preserved (default behavior unchanged)
  • Tests are comprehensive and verify end-to-end behavior
  • Documentation accurately describes behavior including JSON encoding limitations
  • Code follows project conventions from CLAUDE.md

@sonarqubecloud

Copy link
Copy Markdown

@github-actions

github-actions Bot commented Apr 9, 2026

Copy link
Copy Markdown
Contributor

Benchmark Comparison Report

Baseline: main (unknown)

Current: PR-441 (529c3d2)

Summary

  • Regressions: 0
  • Improvements: 0
  • Unchanged: 148
  • Significance level: p < 0.05
All benchmark results (sec/op)
Benchmark Baseline Current Change p-value
_NewBlockFromBytes-4 1.786µ 1.751µ ~ 0.200
SplitSyncedParentMap_SetIfNotExists/256_buckets-4 61.73n 61.56n ~ 0.700
SplitSyncedParentMap_SetIfNotExists/16_buckets-4 62.03n 61.99n ~ 1.000
SplitSyncedParentMap_SetIfNotExists/1_bucket-4 61.77n 61.50n ~ 0.100
SplitSyncedParentMap_ConcurrentSetIfNotExists/256_buckets... 32.64n 30.27n ~ 0.100
SplitSyncedParentMap_ConcurrentSetIfNotExists/16_buckets_... 52.92n 52.62n ~ 0.400
SplitSyncedParentMap_ConcurrentSetIfNotExists/1_bucket_pa... 112.5n 108.5n ~ 0.100
MiningCandidate_Stringify_Short-4 261.4n 261.3n ~ 0.700
MiningCandidate_Stringify_Long-4 1.918µ 1.911µ ~ 0.600
MiningSolution_Stringify-4 971.4n 966.3n ~ 0.700
BlockInfo_MarshalJSON-4 1.801µ 1.801µ ~ 1.000
NewFromBytes-4 142.1n 124.8n ~ 0.100
AddTxBatchColumnar_Validation-4 2.505µ 2.492µ ~ 0.700
OffsetValidationLoop-4 719.5n 725.7n ~ 0.700
Mine_EasyDifficulty-4 67.16µ 67.11µ ~ 0.700
Mine_WithAddress-4 7.040µ 7.076µ ~ 1.000
DiskTxMap_SetIfNotExists-4 3.606µ 3.669µ ~ 1.000
DiskTxMap_SetIfNotExists_Parallel-4 3.378µ 3.355µ ~ 1.000
DiskTxMap_ExistenceOnly-4 323.4n 319.4n ~ 0.600
Queue-4 188.6n 191.0n ~ 0.400
AtomicPointer-4 4.804n 4.805n ~ 0.700
ReorgOptimizations/DedupFilterPipeline/Old/10K-4 910.7µ 872.4µ ~ 0.100
ReorgOptimizations/DedupFilterPipeline/New/10K-4 849.5µ 787.1µ ~ 0.100
ReorgOptimizations/AllMarkFalse/Old/10K-4 105.9µ 105.5µ ~ 1.000
ReorgOptimizations/AllMarkFalse/New/10K-4 62.01µ 62.58µ ~ 0.200
ReorgOptimizations/HashSlicePool/Old/10K-4 55.10µ 58.38µ ~ 0.200
ReorgOptimizations/HashSlicePool/New/10K-4 11.91µ 11.56µ ~ 0.700
ReorgOptimizations/NodeFlags/Old/10K-4 4.732µ 4.813µ ~ 0.100
ReorgOptimizations/NodeFlags/New/10K-4 1.639µ 1.677µ ~ 0.100
ReorgOptimizations/DedupFilterPipeline/Old/100K-4 9.758m 10.273m ~ 0.100
ReorgOptimizations/DedupFilterPipeline/New/100K-4 9.751m 10.747m ~ 0.100
ReorgOptimizations/AllMarkFalse/Old/100K-4 1.090m 1.198m ~ 0.100
ReorgOptimizations/AllMarkFalse/New/100K-4 681.5µ 689.2µ ~ 0.200
ReorgOptimizations/HashSlicePool/Old/100K-4 697.1µ 592.7µ ~ 0.100
ReorgOptimizations/HashSlicePool/New/100K-4 331.6µ 318.6µ ~ 0.400
ReorgOptimizations/NodeFlags/Old/100K-4 51.74µ 54.72µ ~ 0.100
ReorgOptimizations/NodeFlags/New/100K-4 18.43µ 18.39µ ~ 1.000
TxMapSetIfNotExists-4 51.98n 52.64n ~ 0.200
TxMapSetIfNotExistsDuplicate-4 39.90n 40.41n ~ 0.100
ChannelSendReceive-4 606.5n 614.6n ~ 0.400
BlockAssembler_AddTx-4 0.02651n 0.02930n ~ 0.400
AddNode-4 11.38 10.47 ~ 0.100
AddNodeWithMap-4 10.72 11.81 ~ 0.100
DirectSubtreeAdd/4_per_subtree-4 48.06n 47.71n ~ 0.400
DirectSubtreeAdd/64_per_subtree-4 23.24n 23.00n ~ 0.100
DirectSubtreeAdd/256_per_subtree-4 22.01n 22.14n ~ 1.000
DirectSubtreeAdd/1024_per_subtree-4 21.02n 20.81n ~ 0.100
DirectSubtreeAdd/2048_per_subtree-4 20.56n 20.51n ~ 0.300
SubtreeProcessorAdd/4_per_subtree-4 228.1n 230.9n ~ 0.200
SubtreeProcessorAdd/64_per_subtree-4 228.9n 226.1n ~ 0.400
SubtreeProcessorAdd/256_per_subtree-4 224.2n 222.6n ~ 0.700
SubtreeProcessorAdd/1024_per_subtree-4 225.9n 224.3n ~ 1.000
SubtreeProcessorAdd/2048_per_subtree-4 217.9n 215.4n ~ 0.100
SubtreeProcessorRotate/4_per_subtree-4 222.6n 219.5n ~ 0.400
SubtreeProcessorRotate/64_per_subtree-4 217.8n 222.1n ~ 0.100
SubtreeProcessorRotate/256_per_subtree-4 221.4n 219.5n ~ 0.400
SubtreeProcessorRotate/1024_per_subtree-4 224.9n 218.9n ~ 0.100
SubtreeNodeAddOnly/4_per_subtree-4 43.91n 43.78n ~ 0.100
SubtreeNodeAddOnly/64_per_subtree-4 28.46n 28.42n ~ 0.400
SubtreeNodeAddOnly/256_per_subtree-4 27.67n 27.54n ~ 0.100
SubtreeNodeAddOnly/1024_per_subtree-4 27.04n 26.97n ~ 0.300
SubtreeCreationOnly/4_per_subtree-4 90.23n 90.51n ~ 1.000
SubtreeCreationOnly/64_per_subtree-4 321.7n 317.7n ~ 0.100
SubtreeCreationOnly/256_per_subtree-4 1.092µ 1.070µ ~ 0.700
SubtreeCreationOnly/1024_per_subtree-4 3.427µ 3.378µ ~ 0.400
SubtreeCreationOnly/2048_per_subtree-4 6.141µ 6.156µ ~ 0.700
SubtreeProcessorOverheadBreakdown/64_per_subtree-4 221.6n 219.0n ~ 0.300
SubtreeProcessorOverheadBreakdown/1024_per_subtree-4 223.1n 221.5n ~ 0.400
ParallelGetAndSetIfNotExists/1k_nodes-4 1.606m 1.600m ~ 0.100
ParallelGetAndSetIfNotExists/10k_nodes-4 4.295m 4.167m ~ 0.200
ParallelGetAndSetIfNotExists/50k_nodes-4 5.740m 5.879m ~ 0.100
ParallelGetAndSetIfNotExists/100k_nodes-4 7.934m 8.055m ~ 0.200
SequentialGetAndSetIfNotExists/1k_nodes-4 1.416m 1.395m ~ 0.100
SequentialGetAndSetIfNotExists/10k_nodes-4 3.485m 3.475m ~ 0.400
SequentialGetAndSetIfNotExists/50k_nodes-4 10.78m 10.47m ~ 0.100
SequentialGetAndSetIfNotExists/100k_nodes-4 19.86m 19.31m ~ 0.100
ProcessOwnBlockSubtreeNodesParallel/1k_nodes-4 1.635m 1.614m ~ 0.400
ProcessOwnBlockSubtreeNodesParallel/10k_nodes-4 6.670m 6.679m ~ 0.700
ProcessOwnBlockSubtreeNodesParallel/100k_nodes-4 10.71m 10.68m ~ 1.000
ProcessOwnBlockSubtreeNodesSequential/1k_nodes-4 1.441m 1.429m ~ 0.200
ProcessOwnBlockSubtreeNodesSequential/10k_nodes-4 7.660m 6.437m ~ 0.100
ProcessOwnBlockSubtreeNodesSequential/100k_nodes-4 36.71m 35.04m ~ 0.100
CalcBlockWork-4 537.8n 528.5n ~ 0.100
CalculateWork-4 726.6n 714.1n ~ 0.700
CheckOldBlockIDs/on-chain-prefetch/1000-4 64.27µ 63.37µ ~ 0.700
CheckOldBlockIDs/off-chain-prefetch/1000-4 53.06µ 57.67µ ~ 1.000
CheckOldBlockIDs/on-chain-prefetch/10000-4 463.2µ 465.2µ ~ 0.700
CheckOldBlockIDs/off-chain-prefetch/10000-4 352.8µ 355.2µ ~ 0.700
BuildBlockLocatorString_Helpers/Size_10-4 1.414µ 1.400µ ~ 0.100
BuildBlockLocatorString_Helpers/Size_100-4 13.51µ 13.36µ ~ 0.100
BuildBlockLocatorString_Helpers/Size_1000-4 133.6µ 129.9µ ~ 0.100
CatchupWithHeaderCache-4 104.7m 104.7m ~ 1.000
_BufferPoolAllocation/16KB-4 4.063µ 4.123µ ~ 0.100
_BufferPoolAllocation/32KB-4 10.02µ 10.23µ ~ 0.700
_BufferPoolAllocation/64KB-4 21.94µ 20.42µ ~ 1.000
_BufferPoolAllocation/128KB-4 34.95µ 32.66µ ~ 0.100
_BufferPoolAllocation/512KB-4 138.7µ 114.8µ ~ 0.100
_BufferPoolConcurrent/32KB-4 21.47µ 19.90µ ~ 0.200
_BufferPoolConcurrent/64KB-4 32.99µ 31.14µ ~ 0.100
_BufferPoolConcurrent/512KB-4 157.9µ 156.1µ ~ 0.700
_SubtreeDeserializationWithBufferSizes/16KB-4 645.0µ 681.0µ ~ 0.400
_SubtreeDeserializationWithBufferSizes/32KB-4 646.2µ 696.3µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/64KB-4 653.2µ 705.4µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/128KB-4 669.1µ 692.7µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/512KB-4 657.1µ 616.2µ ~ 0.100
_SubtreeDataDeserializationWithBufferSizes/16KB-4 37.53m 37.19m ~ 0.200
_SubtreeDataDeserializationWithBufferSizes/32KB-4 37.39m 37.75m ~ 0.400
_SubtreeDataDeserializationWithBufferSizes/64KB-4 37.33m 37.46m ~ 0.200
_SubtreeDataDeserializationWithBufferSizes/128KB-4 37.03m 37.38m ~ 0.200
_SubtreeDataDeserializationWithBufferSizes/512KB-4 36.83m 36.97m ~ 0.100
_PooledVsNonPooled/Pooled-4 745.5n 741.2n ~ 0.100
_PooledVsNonPooled/NonPooled-4 8.319µ 8.229µ ~ 0.700
_MemoryFootprint/Current_512KB_32concurrent-4 7.103µ 7.024µ ~ 0.700
_MemoryFootprint/Proposed_32KB_32concurrent-4 10.116µ 9.970µ ~ 0.100
_MemoryFootprint/Alternative_64KB_32concurrent-4 9.815µ 9.624µ ~ 0.400
_prepareTxsPerLevel-4 408.1m 415.8m ~ 1.000
_prepareTxsPerLevelOrdered-4 3.793m 4.019m ~ 0.700
_prepareTxsPerLevel_Comparison/Original-4 413.8m 416.2m ~ 0.100
_prepareTxsPerLevel_Comparison/Optimized-4 4.072m 3.589m ~ 0.100
SubtreeSizes/10k_tx_4_per_subtree-4 1.366m 1.402m ~ 0.400
SubtreeSizes/10k_tx_16_per_subtree-4 325.7µ 328.1µ ~ 0.700
SubtreeSizes/10k_tx_64_per_subtree-4 77.12µ 77.89µ ~ 0.400
SubtreeSizes/10k_tx_256_per_subtree-4 19.23µ 19.42µ ~ 0.100
SubtreeSizes/10k_tx_512_per_subtree-4 9.506µ 9.677µ ~ 0.100
SubtreeSizes/10k_tx_1024_per_subtree-4 4.724µ 4.792µ ~ 0.100
SubtreeSizes/10k_tx_2k_per_subtree-4 2.367µ 2.397µ ~ 0.200
BlockSizeScaling/10k_tx_64_per_subtree-4 75.98µ 76.33µ ~ 1.000
BlockSizeScaling/10k_tx_256_per_subtree-4 19.03µ 19.15µ ~ 0.100
BlockSizeScaling/10k_tx_1024_per_subtree-4 4.698µ 4.803µ ~ 0.200
BlockSizeScaling/50k_tx_64_per_subtree-4 403.7µ 399.6µ ~ 0.700
BlockSizeScaling/50k_tx_256_per_subtree-4 94.88µ 96.10µ ~ 0.400
BlockSizeScaling/50k_tx_1024_per_subtree-4 23.41µ 23.39µ ~ 1.000
SubtreeAllocations/small_subtrees_exists_check-4 159.3µ 163.0µ ~ 0.100
SubtreeAllocations/small_subtrees_data_fetch-4 167.2µ 168.1µ ~ 0.400
SubtreeAllocations/small_subtrees_full_validation-4 326.9µ 334.7µ ~ 0.100
SubtreeAllocations/medium_subtrees_exists_check-4 9.328µ 9.595µ ~ 0.100
SubtreeAllocations/medium_subtrees_data_fetch-4 9.877µ 9.931µ ~ 0.400
SubtreeAllocations/medium_subtrees_full_validation-4 18.98µ 19.34µ ~ 0.200
SubtreeAllocations/large_subtrees_exists_check-4 2.259µ 2.298µ ~ 0.100
SubtreeAllocations/large_subtrees_data_fetch-4 2.401µ 2.407µ ~ 0.100
SubtreeAllocations/large_subtrees_full_validation-4 4.749µ 4.810µ ~ 0.200
StoreBlock_Sequential/BelowCSVHeight-4 333.1µ 336.9µ ~ 0.400
StoreBlock_Sequential/AboveCSVHeight-4 343.0µ 337.4µ ~ 0.200
GetUtxoHashes-4 267.7n 269.8n ~ 1.000
GetUtxoHashes_ManyOutputs-4 46.37µ 48.22µ ~ 0.700
_NewMetaDataFromBytes-4 229.1n 229.6n ~ 0.500
_Bytes-4 399.8n 407.6n ~ 0.700
_MetaBytes-4 137.6n 139.2n ~ 0.300

Threshold: >10% with p < 0.05 | Generated: 2026-06-04 17:34 UTC

@sonarqubecloud

Copy link
Copy Markdown

Comment thread cmd/diagnose/config_checks.go Outdated

func runConfigChecks(s *settings.Settings) []ConfigResult {
var results []ConfigResult
results := make([]ConfigResult, 0, 16) //nolint:prealloc

@github-actions github-actions Bot May 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The //nolint:prealloc comment is confusing. This code IS preallocating with make([]ConfigResult, 0, 16). The prealloc linter suggests preallocating slices - which you are doing.

The linter might be suggesting make([]ConfigResult, 16) instead of make([]ConfigResult, 0, 16) since items will be appended. The nolint suppresses that, but the naming is misleading.

Suggestion: Either remove //nolint:prealloc or add a clarifying comment like // Using append pattern for clarity.

Update: This issue is still present in the current code.

Comment thread cmd/diagnose/health_checks.go Outdated

func runHealthChecks(ctx context.Context, logger ulogger.Logger, s *settings.Settings) []HealthResult {
var results []HealthResult
results := make([]HealthResult, 0, 8) //nolint:prealloc

@github-actions github-actions Bot May 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue here - the //nolint:prealloc comment is misleading since the code IS preallocating the slice.

Suggestion: Either remove the nolint directive or add a clarifying comment.

Update: This issue is still present in the current code.

Comment thread test/utils/svnode/blockcreator.go Outdated
// serializeBlock serializes a complete block (header + transactions) to hex string
func (bc *BlockCreator) serializeBlock(header *BlockHeader, txs []*bt.Tx) string {
var buf []byte
buf := make([]byte, 0, 256) //nolint:prealloc

@github-actions github-actions Bot May 14, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same issue - //nolint:prealloc is misleading when the code IS preallocating.

Suggestion: Either remove the nolint directive or add a clarifying comment.

Update: This issue is still present in the current code.

@oskarszoon oskarszoon left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve. Clean toggle gated by blockchain_raw_miner_tag defaulting false, real coinbase fixtures in the new tests, no behaviour change for existing deployments.

Three things worth checking before merge:

  • PR description says services/asset/centrifuge_impl/centrifuge.go is updated but that file isn't in the diff. Centrifuge websocket notifications still go through ExtractCoinbaseMiner (no raw parameter), so they'll stay on the sanitized variant regardless of the setting. Intentional, or a missed call site?
  • test CI job is failing on the latest run while everything else is green (same pattern as a couple of unrelated PRs this week). Worth a re-run to confirm flake vs real.
  • cmd/diagnose/{config,health}_checks.go + test/utils/svnode/blockcreator.go prealloc fixes look unrelated — bundled in via rebase? Fine to leave but normally separate-PR territory.

@freemans13 freemans13 requested a review from liam June 4, 2026 16:11
- Revert unrelated //nolint:prealloc changes in cmd/diagnose and
  test/utils/svnode (scope creep; main already uses `var results`, and CI
  disables the prealloc linter so the preallocation was never required).
- Fix TestExtractCoinbaseMinerRawPreservesAllBytes: it ignored its own
  table and ran identical hardcoded logic 3x. Now drives each case from
  the table and asserts exact bytes for raw vs sanitized modes.
- Add TestExtractCoinbaseMinerRawJSONRoundTrip documenting that raw bytes
  become U+FFFD over JSON (matching how WhatsOnChain renders them), so the
  encoding behaviour is pinned.
- Add TestRawMinerTagSettingWiring verifying the SQL store honours
  blockchain_raw_miner_tag end-to-end.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@sonarqubecloud

sonarqubecloud Bot commented Jun 4, 2026

Copy link
Copy Markdown

@freemans13 freemans13 merged commit a688243 into bsv-blockchain:main Jun 5, 2026
34 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants