-
Notifications
You must be signed in to change notification settings - Fork 38.7k
Open
Labels
Tracking IssueTracks longer-running projects and releases. Gathers TODOs, issues, PRs, and status updates.Tracks longer-running projects and releases. Gathers TODOs, issues, PRs, and status updates.
Description
The coins cache (the CCoinsView* stack, CCoinsViewCache, and friends) is a correctness and performance critical part of validation, IBD, and assumeutxo.
Over time it has accumulated subtle invariants and legacy states (for example around cache entry flags, spent coin handling, and error catching), and we have been chipping away at it through focused refactors, bugfixes, tests, and performance work.
This issue tracks coins-cache-related work to simplify reviewers finding the other related PRs.
IRC context: https://bitcoin-irc.chaincode.com/bitcoin-core-dev/2026-01-08
PRs needing review
Coins cache cleanup and validation refactors
- coins: don't mutate main cache when connecting block #34165 - add
PeekCoin()and a non-mutating view so invalid block connects cannot pollute the main cache; prerequisite for multi-threaded input fetching. - validation: add reusable coins view for ConnectBlock #34164 - add
CCoinsViewCache::Reset()and a reusablem_connect_block_viewcache to reduce allocations and cheaply discard failed connects. - validation: reuse
should_emptycheck for chainstate flush #34125 - tiny refactor to keep theSync()vsFlush()decision consistent (no behavior change). - coins: replace remaining
HaveCoincalls withGetCoin#34320 - merge the two separate ways to check for coins intoGetCoinonly. - refactor: make
CCoinsViewa purely virtual abstract base class #34124 - replace implicit no-op behavior with explicitCCoinsViewEmpty, tightening the coins-view interface contract. - refactor: inline
CCoinsViewErrorCatcherintoCCoinsViewDB#34132 - simplify DB read error handling and remove wrapper layers on the hotGetCoin()path. - validation: fetch block inputs on parallel threads 3x faster IBD #31132 - introduce
CoinsViewCacheAsyncto prefetch input coins duringConnectBlock(includes reviewable subsets in validation: add reusable coins view for ConnectBlock #34164 and coins: don't mutate main cache when connecting block #34165). - coins/refactor: enforce
GetCoin()returns only unspent coins #34207 - assert production invariants and align unit/fuzz tests to avoid impossible spent-coin states. - coins: remove SetFresh method from CCoinsCacheEntry #33018 - remove the unreachable FRESH-but-not-DIRTY state to simplify cache entry semantics and tests.
- validation: do not wipe utxo cache for stats/scans/snapshots #33680 - split FORCE_SYNC vs FORCE_FLUSH and keep the cache warm for
scantxoutset,gettxoutsetinfo, and snapshot creation. - coins: use number of dirty cache entries in flush warnings/logs #33512 - track
dirty_countso flush progress/warnings reflect actual disk writes after non-wiping flush changes.
- fix assumevalid is ignored during reindex #33854 - fix assumevalid being ignored during reindex when best header chainwork is below minimumchainwork (see assumevalid is not always applied when reindexing #31494).
- kernel: Separate UTXO set access from validation functions #32317 - move UTXO access/spending out of
ConnectBlockintoSpendBlock(kernel API, Utreexo, SwiftSync).
Adjacent IBD and performance work (related)
- validation: cache tip recency for lock-free
IsInitialBlockDownload()#34253 - make IBD checks a lock-free atomic read to reducecs_maincontention (enables scheduler-thread IBD gating like net processing: Add ibd check before processing block for txdownloadman #34054). - net processing: Add ibd check before processing block for txdownloadman #34054 - skip orphanage filter maintenance during IBD to save scheduler CPU.
- merkle: pre‑reserve leaves to prevent reallocs with odd vtx count #32497 - pre-reserve Merkle leaves to prevent reallocs with odd vtx count.
- (RFC) kernel: Replace leveldb-based BlockTreeDB with flat-file based store #32427 - concept/approach discussion for simplifying node storage backends (kernelization adjacent).
- blocks: add
-reobfuscate-blocksargument to enable (de)obfuscating existing blocks #33324 - resumable maintenance tool to (de)obfuscate existing block/undo files without resync. - Add initial vectorized chacha20 implementation for 2-3x speedup #34083 - SIMD speedups (not coins-specific, but part of broader performance work).
- [IBD] specialize CheckBlock's input & coinbase checks #31682 - speed up consensus-critical duplicate-input and related checks (focused change, needs careful review).
- [IBD] specialize block serialization #31868 - specialize block serialization to reduce generic serialization overhead (draft).
- Implementation of SwiftSync #34004 - prototype to reduce disk interactions during IBD by aggregating outpoint adds/spends and using an external hints file.
Bench and tooling (supporting work)
- bench: replace embedded raw block with configurable block generator #32554 - remove hardcoded 1.1MB benchmark block and enable flexible benchmark scenarios.
- bench: add fluent API for untimed
setupsteps in nanobench #34208 - simplify benchmark setup/reset patterns.
Merged PRs
- refactor: Let CCoinsViewCache::BatchWrite return void #33866 - simplify cache flush plumbing by removing an always-true return value (also
Sync()andFlush()). - refactor: Improve assumeutxo state representation #30214 - make
Chainstateobjects more self-contained and clarify assumeutxo terminology (maintainability groundwork). - rest: allow reading partial block data from storage #33657 - allow reading partial block data from storage for REST, enabling fetching specific transactions via an external index (useful for tools like bindex).
- validation: periodically flush dbcache during reindex-chainstate #32414 - reduce reindex progress loss risk with large
-dbcache. - precalculate SipHash constant salt XORs #30442 - precalculate SipHash constant salt XORs (hot-path hashing micro-optimization).
- [IBD] coins: reduce lookups in dbcache layer propagation #33602 - reduce redundant map lookups/hashes in
BatchWrite, improving cache flush performance. - refactor: inline constant return values from
dbwrapperwrite methods #33042 - shrink wrapper code and clarify call sites (no behavior change). - [IBD] coins: increase default UTXO flush batch size to 32 MiB #31645 - tune flush batching for better performance on large flushes (still configurable via
-dbbatchsize). - coins: fix
cachedCoinsUsageaccounting inCCoinsViewCache#32313 - fix under/over-counting that triggered UBSan overflow and tighten cache accounting correctness. - coins: warn on oversized
-dbcache#33333 - add a startup warning when configured-dbcacheexceeds a RAM-based cap to reduce OOM/swapping surprises. - test: use local
CBlockIndexin block read hash mismatch check #33154 - avoid mutating shared state and fix a reported data race in tests. - test/refactor: revive test verifying that
GetCoinsCacheSizeStateswitches from OK→LARGE→CRITICAL #33021 - ensure cache size state transitions remain covered after refactors. - [IBD] prevector: store
P2WSH/P2TR/P2PKscripts inline #32279 - reduce dynamic allocations and improve locality for common script sizes. - mempool: Avoid needless vtx iteration during IBD #32827 - avoid needless vtx iteration during IBD.
- [IBD] multi-byte block obfuscation #31144 - speed up block/undo obfuscation by batching XOR operations.
- blocks: force hash validations on disk read #32638 - make on-disk block reads validate hashes now that header hashing is cheap (pairs with blocks: avoid recomputing block header hash in
ReadBlock#32487). - blocks: avoid recomputing block header hash in
ReadBlock#32487 - reuse header hash for PoW and optional integrity checks, saving CPU per read. - validation: write chainstate to disk every hour #30611 - increase periodic flush frequency now that non-wiping flush cost scales with dirty entries.
- [IBD] batch block reads/writes during
AutoFileserialization #31551 - reduce syscall overhead and enable faster obfuscation during block/undo IO. - coins: replace manual
CDBBatchsize estimation with LevelDB's nativeApproximateSize#32185 - use the nativeWriteBatchsize estimator for accuracy and maintainability. - refactor: inline
UndoWriteToDiskandWriteBlockToDiskto reduce serialization calls #31490 - reduce redundant serialization work and simplify disk-write code paths. - coins: warn on shutdown for big UTXO set flushes #31534 - make long shutdown flushes visible to users.
- refactor: prohibit direct flags access in CCoinsCacheEntry and remove invalid tests #30906 - de-risk coins cache refactors by tightening/cleaning the cache-entry interface.
- dbwrapper: Bump LevelDB max file size to 32 MiB to avoid system slowdown from high disk cache flush rate #30039 - reduce LevelDB writes/compactions/open files by bumping LevelDB max file size to 32 MiB.
- init: Correct coins db cache size setting #31064 - avoid unnecessary coins DB reopen and cache reallocation on startup.
- refactor: migrate
bool GetCointo returnoptional<Coin>#30849 - clarify API semantics and align tests with reachable cache states. - streams: cache file position within AutoFile #30884 - cache file position within
AutoFile(Windows regression fix). - Drop -dbcache limit #28358 - allow the full UTXO set to fit into memory by dropping the
-dbcachelimit. - optimization: Reduce cache lookups in CCoinsViewCache::FetchCoin #30326 - avoid double lookups by using
try_emplace(). - Don't empty dbcache on prune flushes: >30% faster IBD #28280 - keep the coins cache warm across prune flushes for faster block connection.
- validation: don't clear cache on periodic flush: >2x block connection speed #28233 - keep the coins cache warm across periodic flushes.
- net: don't lock cs_main while reading blocks in net processing #26326 - reduce validation lock contention by moving disk reads out of
cs_main. - rpc,rest,zmq: faster getblock, NotifyBlock and rest_block by reading raw block #26415 - avoid deserializing when serving raw block data (
getblockverbosity 0, REST bin/hex, ZMQ). - Add pool based memory resource #25325 - use a pool-based memory resource for the in-memory cache (large IBD speedup at the time).
Related context and superseded attempts (reference)
- [IBD] Tracking PR for speeding up Initial Block Download #32043 - [IBD] Tracking PR for speeding up Initial Block Download (draft umbrella PR).
- coins: remove logic for spent-and-FRESH cache entries and writing non-DIRTY entries #30673 - spent-and-FRESH and non-DIRTY entry cleanup attempt (closed, superseded by follow-ups like coins: remove SetFresh method from CCoinsCacheEntry #33018 and coins/refactor: enforce
GetCoin()returns only unspent coins #34207). - Cache m_cached_finished_ibd where SetTip is called. #32885 - cache finished-IBD state near
SetTip(closed; follow-up is validation: cache tip recency for lock-freeIsInitialBlockDownload()#34253). - Don't wipe coins cache when full and instead evict LRU clean entries #31102 - try to evict clean entries instead of wiping the whole coins cache when full (draft/closed).
- sync: improve CCoinsViewCache ReallocateCache #28945 - preallocate memory for recreated caches (closed).
- bench: replace benchmark block with more representative one (413567 → 784588) #32457 - replace benchmark block with a more representative one (draft/closed; superseded by bench: replace embedded raw block with configurable block generator #32554).
- Draft: CCoinMap Experiments #32128 - exploratory CCoinMap experiments (draft/closed).
Metadata
Metadata
Assignees
Labels
Tracking IssueTracks longer-running projects and releases. Gathers TODOs, issues, PRs, and status updates.Tracks longer-running projects and releases. Gathers TODOs, issues, PRs, and status updates.