Improve Block caching and reduce lookup allocations#8943
Open
benaadams wants to merge 9 commits into
Open
Conversation
4b1b634 to
8a7564d
Compare
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR enhances block and transaction caching, introduces a new OnlyHashes RLP decoding behavior to reduce allocations, replaces direct transaction object instantiation with a faster ObjectCreator<T>, and refactors GC and receipt pruning logic to use configuration constants and async methods.
- Propagate
RlpBehaviors.OnlyHashesthrough RLP decoders to skip full decoding when only hashes are needed. - Replace
new T()calls withObjectCreator<T>.Createto leverage expression-based instantiation. - Expand caching interfaces (
IClockCache), integrate clock-based caches across key-value extensions, header store, block store, and block tree. - Refactor GC delay values into
GCConfig, updatePersistentReceiptStorageto async pruning.
Reviewed Changes
Copilot reviewed 24 out of 24 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| src/Nethermind/Nethermind.Serialization.Rlp/TxDecoders/BlobTxDecoder.cs | Pass forceHashes flag to CalculateHash based on OnlyHashes. |
| src/Nethermind/Nethermind.Serialization.Rlp/TxDecoders/BaseTxDecoder.cs | Expose static CalculateHash with forceHashes parameter. |
| src/Nethermind/Nethermind.Serialization.Rlp/TxDecoder.cs | Handle OnlyHashes case in top‐level decoder. |
| src/Nethermind/Nethermind.Serialization.Rlp/RlpBehaviors.cs | Add new OnlyHashes enum flag. |
| src/Nethermind/Nethermind.Serialization.Rlp/Rlp.cs | Extend DecodeArray to accept RlpBehaviors. |
| src/Nethermind/Nethermind.Serialization.Rlp/KeyValueStoreRlpExtensions.cs | Switch to IClockCache<> and add new Get overloads. |
| src/Nethermind/Nethermind.Serialization.Rlp/BlockDecoder.cs | Propagate rlpBehaviors to BlockBodyDecoder.DecodeUnwrapped. |
| src/Nethermind/Nethermind.Serialization.Rlp/BlockBodyDecoder.cs | Use rlpBehaviors to skip header and withdrawal decoding. |
| src/Nethermind/Nethermind.Merge.Plugin/Handlers/ForkchoiceUpdatedHandler.cs | Rename boolean parameters in UpdateMainChain call. |
| src/Nethermind/Nethermind.Merge.Plugin/GC/GCKeeper.cs | Replace hardcoded delay with GCConfig.GCBlockProcessingDelayMs. |
| src/Nethermind/Nethermind.Facade/Simulate/SimulateDictionaryHeaderStore.cs | Remove unused shouldCache, update Cache signature. |
| src/Nethermind/Nethermind.Core/Utils/ObjectCreator.cs | Add ObjectCreator<T> using expression trees for instantiation. |
| src/Nethermind/Nethermind.Core/Caching/IClockCache.cs | Introduce new IClockCache interface. |
| src/Nethermind/Nethermind.Core/Caching/ClockCache.cs | Implement IClockCache on ClockCache. |
| src/Nethermind/Nethermind.Config/GCConfig.cs | Add static GC delay constant. |
| src/Nethermind/Nethermind.Blockchain/Receipts/PersistentReceiptStorage.cs | Refactor receipt pruning to async using GCConfig. |
| src/Nethermind/Nethermind.Blockchain/Headers/IHeaderStore.cs | Extend header store interface for new caching semantics. |
| src/Nethermind/Nethermind.Blockchain/Headers/HeaderStore.cs | Integrate new header‐number cache and multi‐tier header caches. |
| src/Nethermind/Nethermind.Blockchain/Blocks/BlockStore.cs | Inline cache declaration. |
| src/Nethermind/Nethermind.Blockchain/BlockTreeMethodOptions.cs | Add OnlyTxHashes lookup option. |
| src/Nethermind/Nethermind.Blockchain/BlockTree.cs | Integrate OnlyTxHashes into block tree cache and lookup logic. |
| src/Nethermind/Nethermind.Blockchain.Test/Receipts/PersistentReceiptStorageTests.cs | Convert tests to async/Task and increase delays. |
| src/Nethermind/Nethermind.Blockchain.Test/Blocks/HeaderStoreTests.cs | Update Cache call to new signature. |
Comments suppressed due to low confidence (1)
src/Nethermind/Nethermind.Serialization.Rlp/KeyValueStoreRlpExtensions.cs:26
- [nitpick] This overload uses stackalloc for the 40-byte key buffer but lacks [SkipLocalsInit], which could zero-init locals unnecessarily. Consider adding [SkipLocalsInit] for performance.
public static TItem? Get<TItem>(this IReadOnlyKeyValueStore db, long blockNumber, Hash256 hash, IRlpStreamDecoder<TItem> decoder,
0211e45 to
8a7564d
Compare
alexb5dh
reviewed
Jul 28, 2025
| AllowInvalid = 8, | ||
| ExcludeTxHashes = 16, | ||
| All = 31 | ||
| All = 31, |
Contributor
There was a problem hiding this comment.
Shouldn't All option also be updated? Or maybe renamed somehow to avoid confusion?
LukaszRozmej
approved these changes
Sep 4, 2025
Comment on lines
+17
to
+19
| bool Cache(BlockHeader header, bool hasDifficulty, bool isCanonical = false); | ||
| bool TryGetCache(Hash256 blockHash, bool needsDifficulty, bool requiresCanonical, [NotNullWhen(true)] out BlockHeader? header); | ||
| void DeleteCanonicalCache(Hash256 blockHash); |
Member
There was a problem hiding this comment.
a bit weird that caching is exposed
|
|
||
| private async Task RemoveOldReceipts(long blockNumber) | ||
| { | ||
| await Task.Delay(GCConfig.GCBlockProcessingDelayMs / 2); |
Comment on lines
+13
to
+29
| public static Func<T> Create { get; } = BuildCreator(); | ||
|
|
||
| private static Func<T> BuildCreator() | ||
| { | ||
| // find the public parameterless ctor (throws if none) | ||
| ConstructorInfo? ci = typeof(T).GetConstructor(BindingFlags.Public | BindingFlags.Instance, []) ?? | ||
| throw new InvalidOperationException($"{typeof(T).Name} has no parameterless ctor."); | ||
|
|
||
| NewExpression newExpr = Expression.New(ci); | ||
| Expression<Func<T>> lambda = Expression.Lambda<Func<T>>( | ||
| newExpr, | ||
| name: $"Create{typeof(T).Name}", | ||
| tailCall: false, | ||
| parameters: Array.Empty<ParameterExpression>() | ||
| ); | ||
| return lambda.Compile(); | ||
| } |
Member
There was a problem hiding this comment.
In newest dotnet Activator isn't similar speed? There were improvements in dontet 8 and 9 to it
2 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Changes
RemoveOldReceiptsrather than allocating everything; also use foreth_eth_getBlockBy...when only hashesnew T()vis CreateInstanceTTypes of changes
What types of changes does your code introduce?
Testing
Requires testing