Skip to content

feat(health): add disk space check to file store health endpoint#700

Merged
ordishs merged 3 commits into
bsv-blockchain:mainfrom
ordishs:feat/health-disk-space-check
May 19, 2026
Merged

feat(health): add disk space check to file store health endpoint#700
ordishs merged 3 commits into
bsv-blockchain:mainfrom
ordishs:feat/health-disk-space-check

Conversation

@ordishs

@ordishs ordishs commented Apr 14, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Adds disk space checking to the file store Health() method via syscall.Statfs
  • Returns 503 when available space drops below 1 GiB
  • Includes human-readable available/total disk space in the health response for observability
  • Affects all services using file-backed blob storage (external UTXO files, subtree store, etc.)

Healthy response: File Store: Healthy (245.3 GiB available of 500.0 GiB)
Unhealthy response: File Store: Low disk space (512.0 MiB available of 500.0 GiB)

Test plan

  • Verify /health/readiness includes disk space info in the file store dependency
  • Simulate low disk (e.g. tmpfs with small size limit) and verify 503 is returned
  • Verify the threshold of 1 GiB is appropriate for production workloads

Check available disk space via Statfs during file store health checks.
Returns 503 when available space drops below 1 GiB. Includes available
and total disk space in the health response message for observability.
@github-actions

github-actions Bot commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

🤖 Claude Code Review

Status: Complete

Current Issues:

Two minor issues remain from previous review:

New Observations:

  • Platform compatibility: syscall.Statfs is Linux/Unix-specific. Consider adding //go:build !windows if cross-platform builds are planned, or documenting Linux-only requirement.
  • The 1 GiB threshold may be tight for high-write workloads. Consider if this should be configurable or increased based on production metrics.

Fixed:

  • ✅ Test format updated to match new health message format (file_test.go:975)

@ordishs ordishs requested a review from teracoder April 14, 2026 15:49
@github-actions

github-actions Bot commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

Benchmark Comparison Report

Baseline: main (unknown)

Current: PR-700 (009835d)

Summary

  • Regressions: 0
  • Improvements: 0
  • Unchanged: 144
  • Significance level: p < 0.05
All benchmark results (sec/op)
Benchmark Baseline Current Change p-value
_NewBlockFromBytes-4 1.683µ 1.684µ ~ 1.000
SplitSyncedParentMap_SetIfNotExists/256_buckets-4 61.62n 61.90n ~ 0.700
SplitSyncedParentMap_SetIfNotExists/16_buckets-4 61.70n 61.71n ~ 1.000
SplitSyncedParentMap_SetIfNotExists/1_bucket-4 61.67n 61.65n ~ 1.000
SplitSyncedParentMap_ConcurrentSetIfNotExists/256_buckets... 29.75n 29.14n ~ 0.700
SplitSyncedParentMap_ConcurrentSetIfNotExists/16_buckets_... 50.06n 51.03n ~ 0.700
SplitSyncedParentMap_ConcurrentSetIfNotExists/1_bucket_pa... 106.6n 107.5n ~ 0.700
MiningCandidate_Stringify_Short-4 267.2n 263.3n ~ 0.100
MiningCandidate_Stringify_Long-4 1.889µ 1.873µ ~ 0.400
MiningSolution_Stringify-4 981.0n 976.4n ~ 0.700
BlockInfo_MarshalJSON-4 1.748µ 1.755µ ~ 1.000
NewFromBytes-4 129.0n 129.0n ~ 1.000
AddTxBatchColumnar_Validation-4 2.524µ 2.546µ ~ 1.000
OffsetValidationLoop-4 637.5n 637.6n ~ 1.000
Mine_EasyDifficulty-4 60.91µ 61.16µ ~ 0.700
Mine_WithAddress-4 6.826µ 6.932µ ~ 0.700
DiskTxMap_SetIfNotExists-4 3.414µ 3.629µ ~ 0.200
DiskTxMap_SetIfNotExists_Parallel-4 3.323µ 3.281µ ~ 0.400
DiskTxMap_ExistenceOnly-4 305.8n 307.4n ~ 0.200
Queue-4 186.9n 187.8n ~ 0.100
AtomicPointer-4 4.358n 4.381n ~ 1.000
ReorgOptimizations/DedupFilterPipeline/Old/10K-4 833.3µ 843.3µ ~ 0.400
ReorgOptimizations/DedupFilterPipeline/New/10K-4 787.8µ 789.2µ ~ 0.100
ReorgOptimizations/AllMarkFalse/Old/10K-4 102.6µ 101.7µ ~ 0.100
ReorgOptimizations/AllMarkFalse/New/10K-4 62.13µ 62.28µ ~ 0.400
ReorgOptimizations/HashSlicePool/Old/10K-4 55.67µ 60.89µ ~ 0.100
ReorgOptimizations/HashSlicePool/New/10K-4 11.95µ 11.45µ ~ 0.100
ReorgOptimizations/NodeFlags/Old/10K-4 4.627µ 4.598µ ~ 0.100
ReorgOptimizations/NodeFlags/New/10K-4 1.572µ 1.587µ ~ 0.700
ReorgOptimizations/DedupFilterPipeline/Old/100K-4 10.264m 9.153m ~ 0.100
ReorgOptimizations/DedupFilterPipeline/New/100K-4 9.903m 9.683m ~ 0.400
ReorgOptimizations/AllMarkFalse/Old/100K-4 1.051m 1.051m ~ 0.700
ReorgOptimizations/AllMarkFalse/New/100K-4 686.3µ 681.1µ ~ 0.400
ReorgOptimizations/HashSlicePool/Old/100K-4 516.1µ 556.4µ ~ 0.100
ReorgOptimizations/HashSlicePool/New/100K-4 310.4µ 311.3µ ~ 1.000
ReorgOptimizations/NodeFlags/Old/100K-4 47.17µ 51.15µ ~ 0.100
ReorgOptimizations/NodeFlags/New/100K-4 16.89µ 17.44µ ~ 0.100
TxMapSetIfNotExists-4 52.28n 52.85n ~ 0.100
TxMapSetIfNotExistsDuplicate-4 39.93n 39.82n ~ 0.700
ChannelSendReceive-4 613.1n 612.6n ~ 1.000
BlockAssembler_AddTx-4 0.02512n 0.02725n ~ 0.200
AddNode-4 10.56 10.73 ~ 0.700
AddNodeWithMap-4 10.98 11.66 ~ 0.700
DirectSubtreeAdd/4_per_subtree-4 59.53n 59.13n ~ 1.000
DirectSubtreeAdd/64_per_subtree-4 29.39n 29.36n ~ 1.000
DirectSubtreeAdd/256_per_subtree-4 28.02n 28.30n ~ 0.200
DirectSubtreeAdd/1024_per_subtree-4 26.62n 26.66n ~ 1.000
DirectSubtreeAdd/2048_per_subtree-4 26.25n 26.29n ~ 1.000
SubtreeProcessorAdd/4_per_subtree-4 322.1n 381.7n ~ 0.200
SubtreeProcessorAdd/64_per_subtree-4 323.0n 367.7n ~ 0.100
SubtreeProcessorAdd/256_per_subtree-4 345.7n 400.4n ~ 0.500
SubtreeProcessorAdd/1024_per_subtree-4 347.2n 358.8n ~ 1.000
SubtreeProcessorAdd/2048_per_subtree-4 380.1n 359.8n ~ 0.700
SubtreeProcessorRotate/4_per_subtree-4 351.2n 370.8n ~ 0.100
SubtreeProcessorRotate/64_per_subtree-4 349.9n 358.5n ~ 0.200
SubtreeProcessorRotate/256_per_subtree-4 335.4n 358.3n ~ 0.400
SubtreeProcessorRotate/1024_per_subtree-4 339.4n 373.1n ~ 0.400
SubtreeNodeAddOnly/4_per_subtree-4 57.78n 55.54n ~ 0.100
SubtreeNodeAddOnly/64_per_subtree-4 36.20n 36.28n ~ 0.700
SubtreeNodeAddOnly/256_per_subtree-4 35.13n 35.04n ~ 0.400
SubtreeNodeAddOnly/1024_per_subtree-4 34.44n 34.43n ~ 1.000
SubtreeCreationOnly/4_per_subtree-4 110.7n 112.2n ~ 0.200
SubtreeCreationOnly/64_per_subtree-4 361.2n 358.0n ~ 1.000
SubtreeCreationOnly/256_per_subtree-4 1.250µ 1.252µ ~ 0.500
SubtreeCreationOnly/1024_per_subtree-4 3.816µ 3.849µ ~ 0.700
SubtreeCreationOnly/2048_per_subtree-4 6.933µ 7.043µ ~ 0.200
SubtreeProcessorOverheadBreakdown/64_per_subtree-4 313.1n 339.3n ~ 0.100
SubtreeProcessorOverheadBreakdown/1024_per_subtree-4 310.3n 353.0n ~ 0.100
ParallelGetAndSetIfNotExists/1k_nodes-4 2.074m 2.130m ~ 1.000
ParallelGetAndSetIfNotExists/10k_nodes-4 5.550m 5.383m ~ 0.100
ParallelGetAndSetIfNotExists/50k_nodes-4 8.387m 8.237m ~ 0.100
ParallelGetAndSetIfNotExists/100k_nodes-4 11.54m 11.24m ~ 0.200
SequentialGetAndSetIfNotExists/1k_nodes-4 1.845m 1.808m ~ 0.700
SequentialGetAndSetIfNotExists/10k_nodes-4 4.400m 5.836m ~ 0.100
SequentialGetAndSetIfNotExists/50k_nodes-4 14.05m 17.60m ~ 0.100
SequentialGetAndSetIfNotExists/100k_nodes-4 25.58m 30.66m ~ 0.100
ProcessOwnBlockSubtreeNodesParallel/1k_nodes-4 2.056m 2.107m ~ 0.100
ProcessOwnBlockSubtreeNodesParallel/10k_nodes-4 8.508m 8.590m ~ 0.700
ProcessOwnBlockSubtreeNodesParallel/100k_nodes-4 13.84m 14.59m ~ 0.100
ProcessOwnBlockSubtreeNodesSequential/1k_nodes-4 1.814m 1.881m ~ 0.200
ProcessOwnBlockSubtreeNodesSequential/10k_nodes-4 8.256m 10.033m ~ 0.100
ProcessOwnBlockSubtreeNodesSequential/100k_nodes-4 44.38m 56.83m ~ 0.100
CalcBlockWork-4 506.4n 499.9n ~ 0.700
CalculateWork-4 710.0n 702.5n ~ 0.200
BuildBlockLocatorString_Helpers/Size_10-4 1.349µ 1.372µ ~ 0.700
BuildBlockLocatorString_Helpers/Size_100-4 14.53µ 14.73µ ~ 0.700
BuildBlockLocatorString_Helpers/Size_1000-4 126.7µ 128.1µ ~ 0.100
CatchupWithHeaderCache-4 104.2m 104.2m ~ 0.400
_BufferPoolAllocation/16KB-4 4.910µ 5.046µ ~ 1.000
_BufferPoolAllocation/32KB-4 9.396µ 9.099µ ~ 0.100
_BufferPoolAllocation/64KB-4 19.45µ 18.11µ ~ 0.400
_BufferPoolAllocation/128KB-4 36.49µ 27.47µ ~ 0.100
_BufferPoolAllocation/512KB-4 118.6µ 105.9µ ~ 0.100
_BufferPoolConcurrent/32KB-4 17.96µ 19.57µ ~ 0.100
_BufferPoolConcurrent/64KB-4 28.87µ 29.61µ ~ 0.400
_BufferPoolConcurrent/512KB-4 148.4µ 155.3µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/16KB-4 635.7µ 650.9µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/32KB-4 627.8µ 648.4µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/64KB-4 610.1µ 628.2µ ~ 0.100
_SubtreeDeserializationWithBufferSizes/128KB-4 613.8µ 616.1µ ~ 1.000
_SubtreeDeserializationWithBufferSizes/512KB-4 622.0µ 630.5µ ~ 0.100
_SubtreeDataDeserializationWithBufferSizes/16KB-4 36.05m 36.56m ~ 0.100
_SubtreeDataDeserializationWithBufferSizes/32KB-4 36.09m 36.21m ~ 0.400
_SubtreeDataDeserializationWithBufferSizes/64KB-4 36.06m 36.15m ~ 0.700
_SubtreeDataDeserializationWithBufferSizes/128KB-4 36.02m 36.04m ~ 1.000
_SubtreeDataDeserializationWithBufferSizes/512KB-4 35.52m 36.05m ~ 0.100
_PooledVsNonPooled/Pooled-4 740.7n 742.9n ~ 0.700
_PooledVsNonPooled/NonPooled-4 7.751µ 7.992µ ~ 0.700
_MemoryFootprint/Current_512KB_32concurrent-4 6.807µ 6.619µ ~ 0.700
_MemoryFootprint/Proposed_32KB_32concurrent-4 9.547µ 10.461µ ~ 0.100
_MemoryFootprint/Alternative_64KB_32concurrent-4 9.286µ 9.545µ ~ 0.200
_prepareTxsPerLevel-4 319.3m 312.0m ~ 0.400
_prepareTxsPerLevelOrdered-4 3.018m 2.833m ~ 0.100
_prepareTxsPerLevel_Comparison/Original-4 318.5m 314.2m ~ 1.000
_prepareTxsPerLevel_Comparison/Optimized-4 2.924m 2.793m ~ 0.100
SubtreeSizes/10k_tx_4_per_subtree-4 1.282m 1.309m ~ 1.000
SubtreeSizes/10k_tx_16_per_subtree-4 298.3µ 308.1µ ~ 0.700
SubtreeSizes/10k_tx_64_per_subtree-4 71.10µ 72.48µ ~ 0.100
SubtreeSizes/10k_tx_256_per_subtree-4 17.88µ 18.34µ ~ 0.400
SubtreeSizes/10k_tx_512_per_subtree-4 8.872µ 9.068µ ~ 0.100
SubtreeSizes/10k_tx_1024_per_subtree-4 4.361µ 4.456µ ~ 0.100
SubtreeSizes/10k_tx_2k_per_subtree-4 2.179µ 2.205µ ~ 0.100
BlockSizeScaling/10k_tx_64_per_subtree-4 70.18µ 70.65µ ~ 0.700
BlockSizeScaling/10k_tx_256_per_subtree-4 17.68µ 17.77µ ~ 0.400
BlockSizeScaling/10k_tx_1024_per_subtree-4 4.418µ 4.382µ ~ 1.000
BlockSizeScaling/50k_tx_64_per_subtree-4 375.8µ 370.8µ ~ 0.200
BlockSizeScaling/50k_tx_256_per_subtree-4 89.13µ 88.85µ ~ 0.100
BlockSizeScaling/50k_tx_1024_per_subtree-4 21.90µ 21.82µ ~ 0.700
SubtreeAllocations/small_subtrees_exists_check-4 154.9µ 151.1µ ~ 0.200
SubtreeAllocations/small_subtrees_data_fetch-4 159.6µ 161.0µ ~ 0.700
SubtreeAllocations/small_subtrees_full_validation-4 306.5µ 311.7µ ~ 0.400
SubtreeAllocations/medium_subtrees_exists_check-4 8.984µ 8.914µ ~ 0.100
SubtreeAllocations/medium_subtrees_data_fetch-4 9.376µ 9.439µ ~ 0.400
SubtreeAllocations/medium_subtrees_full_validation-4 17.59µ 17.77µ ~ 0.200
SubtreeAllocations/large_subtrees_exists_check-4 2.140µ 2.110µ ~ 0.200
SubtreeAllocations/large_subtrees_data_fetch-4 2.228µ 2.266µ ~ 0.300
SubtreeAllocations/large_subtrees_full_validation-4 4.388µ 4.356µ ~ 0.700
StoreBlock_Sequential/BelowCSVHeight-4 331.3µ 329.7µ ~ 1.000
StoreBlock_Sequential/AboveCSVHeight-4 335.2µ 336.0µ ~ 1.000
GetUtxoHashes-4 207.5n 206.8n ~ 0.400
GetUtxoHashes_ManyOutputs-4 34.62µ 34.55µ ~ 1.000
_NewMetaDataFromBytes-4 165.1n 166.6n ~ 0.100
_Bytes-4 301.9n 300.5n ~ 0.400
_MetaBytes-4 262.5n 256.3n ~ 0.100

Threshold: >10% with p < 0.05 | Generated: 2026-05-19 12:39 UTC

Comment thread stores/blob/file/file.go
return http.StatusInternalServerError, "File Store: Unable to check disk space", err
}

availableBytes := stat.Bavail * uint64(stat.Bsize)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Major] Missing overflow protection: stat.Bavail * uint64(stat.Bsize) could overflow on filesystems with large block sizes. Consider checking for overflow or using math/big for very large values.

Comment thread stores/blob/file/file.go

s.debugf("[File] Health check succeeded path=%s", s.path)
return http.StatusOK, "File Store: Healthy", nil
return http.StatusOK, fmt.Sprintf("File Store: Healthy (%s available of %s)", humanBytes(availableBytes), humanBytes(totalBytes)), nil

@github-actions github-actions Bot May 19, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor] Test update needed: The existing TestFileHealth test at file_test.go:975 expects the old message format "File Store: Healthy" but the implementation now returns "File Store: Healthy ()". The test should be updated to match the new format or use string contains/prefix matching.

FIXED - Test now uses regexp matching at file_test.go:975

Comment thread stores/blob/file/file.go
exp++
}

return fmt.Sprintf("%.1f %ciB", float64(b)/float64(div), "KMGTPE"[exp])

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Minor] Potential index out of bounds: humanBytes() accesses "KMGTPE"[exp] without validating exp < 6. While unlikely with uint64, consider adding a bounds check or using a slice lookup with bounds checking for defensive programming.

@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
77.8% Coverage on New Code (required ≥ 80%)

See analysis details on SonarQube Cloud

@ordishs ordishs merged commit aa0ed02 into bsv-blockchain:main May 19, 2026
25 checks passed
icellan added a commit that referenced this pull request May 19, 2026
Brings in fff6d3e..origin/main (4 commits since previous merge):
  - #897 blockassembly columnar->row fallback on Unimplemented
  - #896 asset on-demand subtree-data admission control + 503/Retry-After
  - #766 utxo: verify spend ownership in Unspend
  - #700 health: disk space check

Conflict resolutions:
  - services/blockassembly/Client.go, util/http.go, util/http_test.go:
    take theirs. Main has the post-Copilot-fix versions of these files
    from #896 and #897 (parseRetryAfter strict-Atoi, debug log on
    columnar fallback, extended TestParseRetryAfter cases) — they
    supersede the pre-fix versions still living on pr-828.
  - stores/utxo/aerospike/un_spend.go: take ours (native-op call,
    no SpendingData arg). The ownership check #766 added to the UDF
    path requires a coordinated update to the BSV-forked aerospike-
    server's subOpUnspend dispatcher. Until that lands, the
    native-op path on this branch skips ownership verification.
    Tracking issue filed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants