Merged
Conversation
branch: merge v1.3.0 fix ups from master to develop
This changes implements faster post-selfdestruct iteration of storage slots for deletion, by using snapshot-storage+stacktrie to recover the trienodes to be deleted. This mechanism is only implemented for path-based schema. For hash-based schema, the entire post-selfdestruct storage iteration is skipped, with this change, since hash-based does not actually perform deletion anyway. --------- Co-authored-by: Martin Holst Swende <martin@swende.se>
core/state: check err for iter.Error
Avoid truncating files, if ancients are opened in readonly mode. With this change, we return error instead of trying (and failing) to repair
…(#28163) * trie: remove internal nodes between shortNode and child in path mode * trie: address comments * core/rawdb, trie: address comments * core/rawdb: delete unused func * trie: change comments * trie: add missing tests * trie: fix lint
* go.mod: upgrade prysm and the indrect dependency prysm from v4.0.2 to v4.0.8, and run go mod tidy * ci: upgrade go version from 1.19 to 1.20 * go-version: upgrade from v1.19 to v1.20 there is some dependency on go v1.20, such as go-libp2p v0.27.8 and also run go mod tidy * dependency: upgrade docker version for security it is not a big issue, since docker is only used for test purpose. * rand: update the usage of math/rand after golang v1.20 2 APIs of math/rand module were deprecated since golang v1.20. that is: rand.Seed() and rand.Read(), refer: ettps://pkg.go.dev/math/rand "rand.Seed(seed int64)" has been replaced by: "r := rand.New(rand.NewSource(seed int64))", need to initialize it with an instance before use "rand.Read()" has been replaced by "crypto/rand.Read()" * readme: need golang v1.20+ to build bsc
Contributor
Author
|
replaced by: #1941 |
accountTrieCache and storageTrieCache were introduced in this PR: #257, which is to improve performance. Actually the performance gain is quite limited, as there is already dirty and clean cache for trie node. And after big merge, these 2 cache can not be used when PBSS is enabled. So remove these code to simplify the logic.
* fix: GetTransactionReceiptsByBlockNumber & GetTransactionDataAndReceipt
eth, trie/triedb/pathdb: address comments
trie/triedb/pathdb: add tests trie/triedb/pathdb: address comment
cherry pick pbss patches from go-ethereum
* fix: use the top root hash for rewinding under path schema * feat: add async flush nodebuffer in path schema * chore: add prun-block param suffix check * fix: code review comments
Contributor
|
It is necessary to add a description of PBSS optimization. In the high-pressure situation of the BSC validator scenario, the |
galaio
approved these changes
Nov 9, 2023
fynnss
approved these changes
Nov 9, 2023
joey0612
approved these changes
Nov 9, 2023
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
11 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Release v1.3.1 is another maintenance release, it mainly has these changes:
About PBSS
PBSS stands for: Path-Based-Storage-Scheme, which is used to optimize the MPT trie tree access, to improve its efficiency and also brings the inline state prune. You could refer this post on how it works: Geth Path-Based Storage Model and Newly Inline State Prune
Currently, PBSS is disabled by default, use this new flag to enable it:
--state.scheme pathImportant
Before use PBSS, you need to make sure your MPT storage in levelDB are already in PBSS format. There are 2 options to get the PBSS storage:
--state.scheme path. // Not recommend, could take 3 months to catch up the latest block.Another thing worth to note is that in the high-pressure situation of the BSC validator scenario, the
nodebufferdisk writing of PBSS will become a performance bottleneck. We have optimized this and made the disk writing operation become asynchronous in the background by usingasynnodebuffer.Change Log
FEATURE
BUGFIX
IMPROVEMENT
Example
NA
Compatibility
PBSS will have a new MPT storage scheme, although it is still based the Key/Value database, like LevelDB. HashBased storage could not use PBSS and vice versa.