Improve Block related caching#9571
Merged
Merged
Conversation
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR optimizes block-related caching mechanisms in Nethermind by introducing a hash-to-number cache, preventing unnecessary cache updates, and using more efficient cache keys.
Key changes:
- Introduces a dedicated
Hash256AsKeycache for block hash to block number mappings - Adds cache update avoidance logic in
ClockCachewhen values are identical - Tracks whether data came from cache via
fromCacheparameter to prevent redundant cache operations - Replaces
ValueHash256withHash256AsKeyfor more efficient cache key storage
Reviewed Changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| Nethermind.Taiko/L1OriginStore.cs | Updates database lookup to use Hash256 instead of ValueHash256 |
| Nethermind.Serialization.Rlp/KeyValueStoreRlpExtensions.cs | Adds fromCache output parameter and refactors cache lookup logic to use Hash256AsKey |
| Nethermind.Facade/Simulate/SimulateDictionaryHeaderStore.cs | Implements fromCache tracking for simulated header store |
| Nethermind.Facade/Simulate/SimulateDictionaryBlockStore.cs | Implements fromCache tracking for simulated block store |
| Nethermind.Core/KeyValueStoreExtensions.cs | Updates key generation methods to use in ValueHash256 for better performance |
| Nethermind.Core/Caching/ClockCache.cs | Adds value comparison logic to avoid updating cache when value is unchanged |
| Nethermind.Blockchain/Headers/IHeaderStore.cs | Adds fromCache output parameter to Get method signature |
| Nethermind.Blockchain/Headers/HeaderStore.cs | Implements hash-to-number cache and integrates fromCache tracking |
| Nethermind.Blockchain/Blocks/IBlockStore.cs | Adds fromCache output parameter to Get method signature |
| Nethermind.Blockchain/Blocks/BlockStore.cs | Implements fromCache tracking and uses Hash256AsKey for cache |
| Nethermind.Blockchain/BlockTree.cs | Uses fromCache to conditionally cache headers and blocks |
| Nethermind.Blockchain.Test/Blocks/HeaderStoreTests.cs | Changes test variable declarations to use interface type |
smartprogrammer93
approved these changes
Oct 27, 2025
LukaszRozmej
approved these changes
Oct 27, 2025
asdacap
reviewed
Oct 28, 2025
marcindsobczak
approved these changes
Oct 28, 2025
Member
|
Could we add:
|
Member
Author
|
Made new bugs :( |
Member
Author
Not sure how this would help? (adding 256 blockhash refreshes to every block) |
062c906 to
d332636
Compare
d332636 to
2651c16
Compare
benaadams
added a commit
that referenced
this pull request
Oct 29, 2025
* Don't recache if from cache * Improve ClockCache fast path * Use smaller keys for Block and Header caches * Pass via in * Add number cache * formatting * sp * Actually use cache * Remove duplicated method * Tweak * Change hash compare * Break test fix speed * Feedback * Test fix * null refs * Fall back to prior implementation * Restore test * Less test changes * Also do reorged block * null ref * Speed up forkchoice
benaadams
added a commit
that referenced
this pull request
Nov 11, 2025
This reverts commit fe8adcc.
LukaszRozmej
pushed a commit
that referenced
this pull request
Nov 12, 2025
kamilchodola
added a commit
that referenced
this pull request
Nov 13, 2025
* Change IPrecompile to use Result<byte[]> allowing to return errors from precompiles (#9561) * Change IPrecompile to use Result<byte[]> * whitespace * fix benchmark * fix tests * Move back DataGasCost to simple long * Some fixes * fixes * refactor * expose precompile error from EVM * ModExpPrecompile should fail on run not on DataGasCost * fixes * Fix tests and bubble up precompile errors * fix build * Another try * Error(2) * Choose substate error * fixes * Simplify handling errors * fix PairingCheckPrecompile * Add syntatic suggar, while preserving lazy evaluation * fix --------- Co-authored-by: lukasz.rozmej <lukasz.rozmej@gmail.com> * Update outdated link (#9654) * Feature/xdc prevent finalized reorg (#9655) * override in blocktree * fix * unittest * format * fix * Fix * remove test * cleanup * Do not insert block if invalid fork * format * review comments * format * Update src/Nethermind/Nethermind.Xdc/XdcBlockTree.cs Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> --------- Co-authored-by: Lukasz Rozmej <lukasz.rozmej@gmail.com> * fix: DynamicCallWithInput to skip value for DELEGATECALL/STATICCALL (#9672) * fix: DynamicCallWithInput to skip value for DELEGATECALL/STATICCALL * update test * Route ColumnDb.Remove through Set(null) to use wrapper semantics (#9636) * Changes for test blockchain (#9680) * test blockchain changes * format * Improve diag Docker container (#9635) * Fix bump-version action and adds reviewers as default (#9557) * feat: add reviewers to the bump-version action * feat: use checkout action * Fix free disk space check for non existent directories (HealthChecks) (#9666) * Create non existent db directory to get drive info * Add db folder to ignored to pass tests * Update OP Superchain chains (#9675) Co-authored-by: emlautarom1 <emlautarom1@users.noreply.github.com> Co-authored-by: Lautaro Emanuel <31224949+emlautarom1@users.noreply.github.com> * Remove Nethermind.Analytics (#9683) * Feature/block production blob limit (#9686) * Introduce BlockProductionBlobLimit to limit number of blobs picked when producing block * Add MaxBlobCount calculation tests * whitespace * Rename * Update src/Nethermind/Nethermind.Config/IBlocksConfig.cs Co-authored-by: Ahmad Bitar <33181301+smartprogrammer93@users.noreply.github.com> --------- Co-authored-by: Ahmad Bitar <33181301+smartprogrammer93@users.noreply.github.com> * Fix resource leak in HttpJsonRpcSubmitter (#9687) Update HttpJsonRpcSubmitter.cs * Remove unused BlockFilter.StartBlockNumber and simplify CreateBlockFilter (#9630) * Remove unused BlockFilter.StartBlockNumber and simplify CreateBlockFilter * fix ci * formatting * Revert "Improve Block related caching (#9571)" (#9688) * Revert "Improve Block related caching (#9571)" This reverts commit fe8adcc. * merge fix * Update Dockerfiles (#9689) Co-authored-by: rubo <rubo@users.noreply.github.com> * Fix Chainspec for Taiko-Hoodi (#9699) fix transition for taiko-hoodi * Xdc : Add XdcTestBlockchain for Module testing (#9596) * start * more dependencies * build * rename * voting * vote task * bit of refactor * invoke block produced dont seal * refactor context * format * timeout missing * block producer * format * basic incomplete setup * test * block production test * format * merge cleanup * merged * use the constant * use timestamp from attributes * Merged * Merged fix * cleanup * fixes * cleanup * commit cert before vote propose in background * merge and refactor * rename * format * comment * refactor to continuous loop * remove TCM * Format * fix * draft implmentation * cleanup * handle regenesis block in XdcTestBlockchain * refactor to inherit from TestBlockchain * register FromXdcContainer * handle _fromContainer from base * attempt rotating signers * test fix * test fix * test and container setup fix * decoder fix * bug fixes * init QC * test xdc sealer rotation * bug fixes and working now * Format * Update packages.lock.json * remove * total difficulty fix * format * override in blocktree * fix * unittest * format * fix build issues * fix * Fix * remove test * fixes * format * cleanup * undo merge and cleanup usings * add custom genesis builder for xdc * format whitespaces * Do not insert block if invalid fork * format * reorg test * revert block processor * revert cloning of header in PrepareBlockForProcessing * reorgs tests working * fix bad db * fix seal validator * fix * format * implement add block without commiting qc and generate extra keys for testing * fix and refactor * optional params * fix * one fix and a bit of refactor * reorg test * rename * return block instead * reorg test * fixes * dont set author * 10 seconds per add block * test blockchain changes * format * fixes for hotstuff module * fix test block producer * fixes for epoch master calculation * format * rename * rollback QC test blockchain order * snapshotmanager tests fixed * format * cleanup * remove TestBlockchainTester * cleanup * small fix in header validator test --------- Co-authored-by: ak88 <anders.holmbjerg@hotmail.com> Co-authored-by: cicr99 <kr1000a@gmail.com> * XDC - Snapshot Manager refactor (#9695) refactor of snapshot manager * Xdc : Port BlockInfoValidator Module Tests (#9650) * start * more dependencies * build * rename * voting * vote task * bit of refactor * invoke block produced dont seal * refactor context * format * timeout missing * block producer * format * basic incomplete setup * test * block production test * format * merge cleanup * merged * use the constant * use timestamp from attributes * Merged * Merged fix * cleanup * fixes * cleanup * commit cert before vote propose in background * merge and refactor * rename * format * comment * refactor to continuous loop * remove TCM * Format * fix * draft implmentation * cleanup * handle regenesis block in XdcTestBlockchain * refactor to inherit from TestBlockchain * register FromXdcContainer * handle _fromContainer from base * attempt rotating signers * test fix * test fix * test and container setup fix * decoder fix * bug fixes * init QC * test xdc sealer rotation * bug fixes and working now * Format * Update packages.lock.json * remove * draft impl of blockinfo validator tests * total difficulty fix * format * override in blocktree * fix * unittest * format * fix build issues * fix * Fix * remove test * fixes * format * cleanup * undo merge and cleanup usings * add custom genesis builder for xdc * format whitespaces * Do not insert block if invalid fork * format * reorg test * revert block processor * revert cloning of header in PrepareBlockForProcessing * reorgs tests working * fix bad db * fix seal validator * fix * format * implement add block without commiting qc and generate extra keys for testing * fix and refactor * optional params * fix * one fix and a bit of refactor * reorg test * rename * return block instead * reorg test * fixes * fix failing test * dont set author * 10 seconds per add block * test blockchain changes * format * ws fixes * fixes for hotstuff module * fix test block producer * fixes for epoch master calculation * format * rename * rollback QC test blockchain order * CLEANUP * remove unused --------- Co-authored-by: ak88 <anders.holmbjerg@hotmail.com> Co-authored-by: cicr99 <kr1000a@gmail.com> * Unit tests for commit in XDC QC manager (#9700) unittests * Xdc : Port HeaderValidator Module Tests (#9617) * start * more dependencies * build * rename * voting * vote task * bit of refactor * invoke block produced dont seal * refactor context * format * timeout missing * block producer * format * basic incomplete setup * test * block production test * format * merge cleanup * merged * use the constant * use timestamp from attributes * Merged * Merged fix * cleanup * fixes * cleanup * commit cert before vote propose in background * merge and refactor * rename * format * comment * refactor to continuous loop * remove TCM * Format * fix * draft implmentation * cleanup * handle regenesis block in XdcTestBlockchain * refactor to inherit from TestBlockchain * register FromXdcContainer * handle _fromContainer from base * attempt rotating signers * test fix * test fix * test and container setup fix * draft port of some HeaderValidatorTests * fix build issue * extra tests * decoder fix * handle merge issues * bug fixes * init QC * test xdc sealer rotation * bug fixes and working now * Format * Update packages.lock.json * remove * fix build issues * more build fixes * total difficulty fix * format * override in blocktree * fix * unittest * format * fix build issues * fix * Fix * remove test * fixes * format * cleanup * undo merge and cleanup usings * add custom genesis builder for xdc * format whitespaces * Do not insert block if invalid fork * format * reorg test * revert block processor * revert cloning of header in PrepareBlockForProcessing * reorgs tests working * fix bad db * fix seal validator * fix * format * implement add block without commiting qc and generate extra keys for testing * fix and refactor * optional params * fix * one fix and a bit of refactor * reorg test * rename * return block instead * minor changes and fixes * reorg test * fixes * bring back cache field in sealValidator * dont set author * 10 seconds per add block * test blockchain changes * format * ws fixes * fixes for hotstuff module * fix test block producer * fixes for epoch master calculation * format * rename * rollback QC test blockchain order * cleanup --------- Co-authored-by: ak88 <anders.holmbjerg@hotmail.com> Co-authored-by: cicr99 <kr1000a@gmail.com> * Update base-sepolia testnet matrix (#9705) chore: update base-sepolia testnet matrix * Comment out Linea in workflows (#9706) chore: comment linea on our workflows --------- Co-authored-by: Alexey Osipov <me@flcl.me> Co-authored-by: lukasz.rozmej <lukasz.rozmej@gmail.com> Co-authored-by: Jaye Turner <jayeturnerjatu@gmail.com> Co-authored-by: ak88 <anders@nethermind.io> Co-authored-by: Snezhkko <snezhkodaria38@gmail.com> Co-authored-by: radik878 <radikpadik76@gmail.com> Co-authored-by: Ruben Buniatyan <rubo@users.noreply.github.com> Co-authored-by: Marcos Antonio Maceo <35319980+stdevMac@users.noreply.github.com> Co-authored-by: Damian Orzechowski <114909782+damian-orzechowski@users.noreply.github.com> Co-authored-by: core-repository-dispatch-app[bot] <173070810+core-repository-dispatch-app[bot]@users.noreply.github.com> Co-authored-by: emlautarom1 <emlautarom1@users.noreply.github.com> Co-authored-by: Lautaro Emanuel <31224949+emlautarom1@users.noreply.github.com> Co-authored-by: Ahmad Bitar <33181301+smartprogrammer93@users.noreply.github.com> Co-authored-by: maradini77 <140460067+maradini77@users.noreply.github.com> Co-authored-by: GarmashAlex <garmasholeksii@gmail.com> Co-authored-by: Ben {chmark} Adams <thundercat@illyriad.co.uk> Co-authored-by: Diptanshu Kakwani <dipkakwani@gmail.com> Co-authored-by: Ayman Bouchareb <Ayman.bouchareb@outlook.fr> Co-authored-by: ak88 <anders.holmbjerg@hotmail.com> Co-authored-by: cicr99 <kr1000a@gmail.com> Co-authored-by: Carmen Irene Cabrera Rodríguez <49727740+cicr99@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
Types of changes
What types of changes does your code introduce?
Testing
Requires testing