Skip to content

Reset prefetcher correctly#1853

Merged
cffls merged 1 commit intodevelopfrom
fix_prefetch
Oct 30, 2025
Merged

Reset prefetcher correctly#1853
cffls merged 1 commit intodevelopfrom
fix_prefetch

Conversation

@cffls
Copy link
Copy Markdown
Contributor

@cffls cffls commented Oct 30, 2025

Description

Remove live state objects from statedb when resetting prefetchers. This prevents error logs when adding state sync transactions like the following:

Oct 21 13:03:19 nodes-sp bor[2932588]: ERROR[10-21|13:03:19.519] Prefetcher missed to load trie           owner=391e2d..3bc266 root=73c9d1..41dd08
Oct 21 13:03:19 nodes-sp bor[2932588]: ERROR[10-21|13:03:19.519] Prefetcher missed to load trie           owner=36ecd3..069c93 root=e24c9c..5963c7
Oct 21 13:03:19 nodes-sp bor[2932588]: ERROR[10-21|13:03:19.519] Prefetcher missed to load trie           owner=411e7d..4f7b8d root=311530..2566e9
Oct 21 13:03:19 nodes-sp bor[2932588]: ERROR[10-21|13:03:19.519] Prefetcher missed to load trie           owner=1f5802..a52cff root=d7d24c..972121
Oct 21 13:03:19 nodes-sp bor[2932588]: ERROR[10-21|13:03:19.519] Prefetcher missed to load trie           owner=696ab8..9def84 root=9a91a5..410438
Oct 21 13:03:19 nodes-sp bor[2932588]: INFO [10-21|13:03:19.519] Fetching state updates from Heimdall     fromID=3,108,563 to=2025-10-21T13:01:11Z
Oct 21 13:03:19 nodes-sp bor[2932588]: INFO [10-21|13:03:19.519] Fetching state sync events               queryParams="from_id=3108563&to_time=2025-10-21T13:01:11Z&pagination.limit=50"
Oct 21 13:03:19 nodes-sp bor[2932588]: INFO [10-21|13:03:19.522] StateSyncData                            gas=0     number=77,973,520 lastStateID=3,108,562 "total records"=0 "fetch time"=24 "process time"=0
Oct 21 13:03:19 nodes-sp bor[2932588]: INFO [10-21|13:03:19.556] Imported new chain segment               number=77,973,520 hash=2afe3b..c92646 blocks=1  txs=94   mgas=13.625  elapsed=122.833ms mgasps=110.924 snapdiffs=8.16MiB triediffs=220.11MiB triedirty=202.09MiB
Oct 21 13:03:21 nodes-sp bor[2932588]: INFO [10-21|13:03:21.507] Imported new chain segment               number=77,973,521 hash=014401..af2709 blocks=1  txs=55   mgas=14.392  elapsed=89.331ms  mgasps=161.103 snapdiffs=8.18MiB triediffs=218.94MiB triedirty=202.49MiB

Changes

  • Bugfix (non-breaking change that solves an issue)
  • Hotfix (change that solves an urgent issue, and requires immediate attention)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (change that is not backwards-compatible and/or changes current functionality)
  • Changes only for a subset of nodes

Checklist

  • I have added at least 2 reviewer or the whole pos-v1 team
  • I have added sufficient documentation in code
  • I will be resolving comments - if any - by pushing each fix in a separate commit and linking the commit hash in the comment reply
  • Created a task in Jira and informed the team for implementation in Erigon client (if applicable)
  • Includes RPC methods changes, and the Notion documentation has been updated

Cross repository changes

  • This PR requires changes to heimdall
    • In case link the PR here:
  • This PR requires changes to matic-cli
    • In case link the PR here:

Testing

  • I have added unit tests
  • I have added tests to CI
  • I have tested this code manually on local environment
  • I have tested this code manually on remote devnet using express-cli
  • I have tested this code manually on amoy
  • I have created new e2e tests into express-cli

Manual tests

Please complete this section with the steps you performed if you ran manual tests for this functionality, otherwise delete it

Additional comments

Please post additional comments in this section if you have them, otherwise delete it

@sonarqubecloud
Copy link
Copy Markdown

@cffls cffls requested a review from lucca30 October 30, 2025 17:07
@cffls cffls merged commit 44acd51 into develop Oct 30, 2025
10 of 12 checks passed
marcello33 added a commit that referenced this pull request Nov 17, 2025
* Parallel block import (#1662)

* core: naive first implmenetation of parallel block import in InsertChainStateless

* core: add logic to handle dependency while parallel import

* core: handle unknown ancestor + init some UTs

* core: implement alternative way to check for unknown ancestor

* core: defer ValidateWitnessPreState

* zero commit

* core: do sequential insert if the importing chain is small

* core: fix  error handling

* core: update UTs

* core,eth,internal/cli: add config to enable/disable parallel import + refactor

* core: add metrics for seq and parallel import

* internal/cli: moved parallelstatelessimport flag to the witness config

* core: changed debug import log to info

* core: added 2 new metrics to track the number of blocks processed in sequential and parallel stateless imports

* core: init benchmarks

* core: fix benchmarks

* core: use worker pool for parallel block import

* core: try parallel import for smaller batch

* core: dedup header verification logic

* zero commit

* core,eth,internal/cli: improvements based on comments

* core: add adversarial tests for stateless insert

* core: re-add empty chain insertion case

* zero commit to trigger CI

* core,eth/catalyst: report correct execution stats for stateless insert

* CI: temp update for running stateless tests

* zero commit to trigger CI

* zero commit to trigger CI

* CI: try running on stateless branch

* Revert "CI: try running on stateless branch"

This reverts commit 98b5704.

* zero commit to trigger CI

* core: add log for deferred exec of blocks due to invalid root err in parallel import

* core: changed log to debug

* core,eth,internal/ethapi: suppress error log for state sync when parallel import is enabled

* eth/filters: fix gomock

* eth: add missing configs

* core: retry execution in parallel import for block validator errors

* core: commit block before retrying failed execution

* core: rm redundant witness in execResult

* core: try flushing state db asap post execution

* core: add temp log when committing code

* core: enable retry for first block in batch

* Revert "core: add temp log when committing code"

This reverts commit cf63cc3.

* core: cap workers spun

* Revert "core: cap workers spun"

This reverts commit 96367ee.

* core: ensure statedb cleanup

* core: add temp debug logs

* core: add temp debug logs

* core: add temp log to track sidechain

* core: add temp log for contract code in commitAndFlush

* core: add retry for all processing and internal state db errors

* core: add state db error validation during deferred exec

* core: add TestParallelStateless_ContractDeployedThenCalled

* Revert "core: add temp log for contract code in commitAndFlush"

This reverts commit 174765f.

* Revert "core: add temp log to track sidechain"

This reverts commit 29b3366.

* Revert "core: add temp debug logs"

This reverts commit 2c4f840.

* internal/cli: turn off parallel import by default

* internal/cli: update flag identation

* CI: update comment

* zero commit to trigger CI

* internal/cli: rm witness prune flags

---------

Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>

* emit event when write block in stateless node (#1840)

* Address Biased Trie Cache (#1837)

* Address-biased trie cache

* Stop preload early

* Print more stats

* Increase cache size

* Store by depth

* More customizable

* Add metrics

* Fix key format

* Async preload

* Simplify

* Fix linter

* Allow interrupt

* Remove free disk step in CI (#1843)

Remove this step since the runner has 600G of disk space.

* chore: bump kurtosis-pos (#1845)

* chore: bump kurtosis-pos

* chore: bump to v1.2.1

* PIP-74: state-sync txs inclusion (#1726)

* first working version of new state sync tx

* remove logs

* fix lint

* fix integration tests

* test fixes

* lint fix

* fix parallel processor

* type improvement and unit tests

* remove duplicates

* ignore data test output

* fixing method calls and fields

* build fix

* remove bor filter for after hf blocks

* sort logs just when necessary

* ssTxs: fix receiptsHash mismatch (#1829)

* ssTxs: fix receiptsHash mismatch

* chore: remove block hash and number

* revert: add block number

* core(tx): fix Hash method for StateSyncTxType (#1830)

* eth: fixes in receipt handling via p2p post HF (#1825)

* eth: include bor receipts in ReceiptHash post HF

* eth: rename to receiptListHash

* eth: extrat typecasting logic for better testing, fix type matching issue

* eth: add e2e tests for receipt delivery

* eth/protocols/eth: apply HF logic while handling receipt query over eth69

* core: skip split receipts post HF

* tests/bor: extend e2e test to check presence of state-sync in block

* core/types: return nil for chainID() call over state-sync tx

* internal/ethapi: handle bor txs and receipts post HF (#1834)

* fix: cumulativeGasUsed in insertStateSyncTransactionAndCalculateReceipt (#1835)

* fix: sort logs and remove duplicate append in Finalize (#1836)

* fix: append tx only in FinalizeAndAssemble and use state instead of wrappedState

* consensus/bor: sort logs before extracting state-sync logs

* chore: revert statedb changes

---------

Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* chore: nit (#1838)

* chore: typos

* fix: lint

* Renaming to Madhugiri HF

* chore: nits

* (fix): handle pointers to receipt list received via p2p

* (chore): rename tests with HF name

* chore: more nits

* core: add blocktime in bor receipt logs (#1848)

* core/types: derive bloom for bor receipts

---------

Co-authored-by: kamuikatsurgi <shahkrishang11@gmail.com>
Co-authored-by: Krishang Shah <109511742+kamuikatsurgi@users.noreply.github.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* core, miner, params, cmd: implement EIP-7823, EIP-7825 and EIP-7883 (#1842)

* first working version of new state sync tx

* remove logs

* fix lint

* fix integration tests

* test fixes

* lint fix

* fix parallel processor

* type improvement and unit tests

* remove duplicates

* ignore data test output

* fixing method calls and fields

* build fix

* remove bor filter for after hf blocks

* sort logs just when necessary

* ssTxs: fix receiptsHash mismatch (#1829)

* ssTxs: fix receiptsHash mismatch

* chore: remove block hash and number

* revert: add block number

* core(tx): fix Hash method for StateSyncTxType (#1830)

* eth: fixes in receipt handling via p2p post HF (#1825)

* eth: include bor receipts in ReceiptHash post HF

* eth: rename to receiptListHash

* eth: extrat typecasting logic for better testing, fix type matching issue

* eth: add e2e tests for receipt delivery

* eth/protocols/eth: apply HF logic while handling receipt query over eth69

* core: skip split receipts post HF

* tests/bor: extend e2e test to check presence of state-sync in block

* core/types: return nil for chainID() call over state-sync tx

* internal/ethapi: handle bor txs and receipts post HF (#1834)

* fix: cumulativeGasUsed in insertStateSyncTransactionAndCalculateReceipt (#1835)

* fix: sort logs and remove duplicate append in Finalize (#1836)

* fix: append tx only in FinalizeAndAssemble and use state instead of wrappedState

* consensus/bor: sort logs before extracting state-sync logs

* chore: revert statedb changes

---------

Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* chore: nit (#1838)

* chore: typos

* fix: lint

* Renaming to Madhugiri HF

* chore: nits

* (fix): handle pointers to receipt list received via p2p

* (chore): rename tests with HF name

* chore: more nits

* core: implement eip7883

* core: implement eip7825

* fix lint

* fix isOsaka condition

* fix lint and add comment for int test failing

* fix failing test / address comments

* core: add blocktime in bor receipt logs (#1848)

* core/types: derive bloom for bor receipts

* prioritize ss hf / remove todo

* fix lint

* split precompiled contracts and addresses for madhugiri HF from osaka

* revert err msg to align with geth

* eth/gasestimator: check ErrGasLimitTooHigh conditions (#32348)

This PR makes 2 changes to how
[EIP-7825](ethereum/go-ethereum#31824) behaves.

When `eth_estimateGas` or `eth_createAccessList` is called without any
gas limit in the payload, geth will choose the block's gas limit or the
`RPCGasCap`, which can be larger than the `maxTxGas`.

When this happens for `estimateGas`, the gas estimation just errors out
and ends, when it should continue doing binary search to find the lowest
possible gas limit.

This PR will: 
- Add a check to see if `hi` is larger than `maxTxGas` and cap it to
`maxTxGas` if it's larger. And add a special case handling for gas
estimation execute when it errs with `ErrGasLimitTooHigh`

---------

Co-authored-by: Gary Rong <garyrong0905@gmail.com>

* internal/ethapi: skip tx gas limit check for calls (#32641)

This disables the tx gaslimit cap for eth_call and related RPC operations.

I don't like how this fix works. Ideally we'd be checking the tx
gaslimit somewhere else, like in the block validator, or any other place
that considers block transactions. Doing the check in StateTransition
means it affects all possible ways of executing a message.

The challenge is finding a place for this check that also triggers
correctly in tests where it is wanted. So for now, we are just combining
this with the EOA sender check for transactions. Both are disabled for
call-type messages.

* internal/ethapi: use gas from gaspool for call defaults

* eth/gasestimator: check for madhugiri HF

* consensus/bor: honour MaxTxGas for system calls

---------

Co-authored-by: Lucca Martins <lucca_martins30@yahoo.com.br>
Co-authored-by: kamuikatsurgi <shahkrishang11@gmail.com>
Co-authored-by: Krishang Shah <109511742+kamuikatsurgi@users.noreply.github.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Minhyuk Kim <kimminhyuk1004@gmail.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>

* (chore): update madhugiri block number for amoy and update consensus block time (#1851)

* first working version of new state sync tx

* remove logs

* fix lint

* fix integration tests

* test fixes

* lint fix

* fix parallel processor

* type improvement and unit tests

* remove duplicates

* ignore data test output

* fixing method calls and fields

* build fix

* remove bor filter for after hf blocks

* sort logs just when necessary

* ssTxs: fix receiptsHash mismatch (#1829)

* ssTxs: fix receiptsHash mismatch

* chore: remove block hash and number

* revert: add block number

* core(tx): fix Hash method for StateSyncTxType (#1830)

* eth: fixes in receipt handling via p2p post HF (#1825)

* eth: include bor receipts in ReceiptHash post HF

* eth: rename to receiptListHash

* eth: extrat typecasting logic for better testing, fix type matching issue

* eth: add e2e tests for receipt delivery

* eth/protocols/eth: apply HF logic while handling receipt query over eth69

* core: skip split receipts post HF

* tests/bor: extend e2e test to check presence of state-sync in block

* core/types: return nil for chainID() call over state-sync tx

* internal/ethapi: handle bor txs and receipts post HF (#1834)

* fix: cumulativeGasUsed in insertStateSyncTransactionAndCalculateReceipt (#1835)

* fix: sort logs and remove duplicate append in Finalize (#1836)

* fix: append tx only in FinalizeAndAssemble and use state instead of wrappedState

* consensus/bor: sort logs before extracting state-sync logs

* chore: revert statedb changes

---------

Co-authored-by: Manav Darji <manavdarji.india@gmail.com>

* chore: nit (#1838)

* chore: typos

* fix: lint

* Renaming to Madhugiri HF

* chore: nits

* (fix): handle pointers to receipt list received via p2p

* (chore): rename tests with HF name

* chore: more nits

* core: add blocktime in bor receipt logs (#1848)

* core/types: derive bloom for bor receipts

* (chore): update madhugiri block for amoy

* Change consensus block time to 1s at Madhugiri HF (#1852)

---------

Co-authored-by: Lucca Martins <lucca_martins30@yahoo.com.br>
Co-authored-by: kamuikatsurgi <shahkrishang11@gmail.com>
Co-authored-by: Krishang Shah <109511742+kamuikatsurgi@users.noreply.github.com>
Co-authored-by: Jerry <jerrycgh@gmail.com>

* Completed reset prefetcher (#1853)

* internal/ethapi: restore original RPC gas cap (#1856)

* Restore original RPC gas cap

* Fix test

* chore: fix wrong function name in comment (#1841)

Signed-off-by: reddaisyy <reddaisy@outlook.jp>

* Revert "(chore): update madhugiri block number for amoy and update consensus block time (#1851)" (#1857)

This reverts commit 103f1f4.

* params: bump version to v2.4.0-beta

* Dont OpenTrie because HistoricDatabase doesnt implement trie (#1858)

* chore: use 2^25 as the MaxTxGas (#1859)

* params: bump version to v2.4.0-beta2

* chore: set madhugiri block for amoy and consensus block time (#1867)

* chore: set madhugiri block for amoy and consensus block time

* params: bump version

* Disable txn indexer in stateless mode

Since stateless nodes are using a different pruner from the regular pruner in geth, the transaction indexer will have problem with pruned blocks, resulting hanging goroutines in memory. This change prevents stateless node from running indexer on pruned blocks.

* core: init PrecompiledAddressesMadhugiri

* params: bump version

* params: bump version for stable release

* chore: set madhugiri block for mainnet and consensus block time

* chore: fix lint

* Include missing check of P256 on Amoy (#1877)

* checks p256 instruction

* lint fix

* v2.5.0 candidate (#1878)

* chore: MadhugiriPro hf

* params: use stable tag

* chore: set new HF heights

* Reinforce Precompile Check on any new HF (#1881)

* Include missing check of P256 on Amoy (#1877)

* checks p256 instruction

* lint fix

* reinforce precompilers check

---------

Co-authored-by: Lucca Martins <lucca_martins30@yahoo.com.br>

---------

Signed-off-by: reddaisyy <reddaisy@outlook.jp>
Co-authored-by: Raneet Debnath <35629432+Raneet10@users.noreply.github.com>
Co-authored-by: Pratik Patil <pratikspatil024@gmail.com>
Co-authored-by: Lucca Martins <lucca_martins30@yahoo.com.br>
Co-authored-by: Jerry <jerrycgh@gmail.com>
Co-authored-by: Krishang Shah <109511742+kamuikatsurgi@users.noreply.github.com>
Co-authored-by: kamuikatsurgi <shahkrishang11@gmail.com>
Co-authored-by: Manav Darji <manavdarji.india@gmail.com>
Co-authored-by: Minhyuk Kim <kimminhyuk1004@gmail.com>
Co-authored-by: Gary Rong <garyrong0905@gmail.com>
Co-authored-by: Felix Lange <fjl@twurst.com>
Co-authored-by: reddaisyy <reddaisy@outlook.jp>
Co-authored-by: Angel Valkov <avalkov@polygon.technology>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants