Skip to content

Reduce memory usage of CBlockIndex#49

Merged
DeckerSU merged 7 commits intostatic-experimentalfrom
reduce-equihash-solution-memory-new
Apr 20, 2023
Merged

Reduce memory usage of CBlockIndex#49
DeckerSU merged 7 commits intostatic-experimentalfrom
reduce-equihash-solution-memory-new

Conversation

@DeckerSU
Copy link
Copy Markdown
Owner

@DeckerSU DeckerSU commented Mar 31, 2023

This PR is a backport of zcash/zcash#6192 in our codebase:

The nSolution field of CBlockIndex holds a std::vector<unsigned char> containing a 1344-byte Equihash solution. In normal operation the node holds a linked tree of CBlockIndex nodes for the whole chain. Therefore, the total memory required for these nSolution fields for the current mainnet height 3358920, is at least 3358920 * 1344 = 4514388480 bytes, ~4.51 GiB.

It has to be possible to retrieve the nSolution field for any stored block in order to answer peer getheaders queries, for example. But once a block index entry has been flushed to leveldb (which happens asynchronously), it is no longer necessary to keep this field in memory, because it can be read back if needed. The cost of this should be small because of the caching done by leveldb.

Links:

With this PR old branch named reduce-equihash-solution-memory could be deleted.

Todo:

daira and others added 7 commits March 31, 2023 15:54
Signed-off-by: Daira Hopwood <daira@jacaranda.org>

# Conflicts:
#	qa/rpc-tests/test_framework/util.py
#	src/txdb.cpp
…as been written

to the leveldb database.

Co-authored-by: Jack Grigg <thestr4d@gmail.com>
Signed-off-by: Daira Hopwood <daira@jacaranda.org>

# Conflicts:
#	doc/release-notes.md
#	src/chain.h
#	src/main.cpp
#	src/rest.cpp
#	src/rpc/blockchain.cpp
#	src/txdb.h
Signed-off-by: Daira Hopwood <daira@jacaranda.org>
…lly so.

Signed-off-by: Daira Hopwood <daira@jacaranda.org>

# Conflicts:
#	src/txdb.cpp
#	src/txdb.h
The const annotation was removed from the blockinfo type in
zcash/zcash#6192, but not from the type of its iterator. Recent Clang
versions are able to handle this, but GCC 11 (and it appears older Clang
versions) raise an error.

Closes zcash/zcash#6306.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants