node: scale default -dbcache with system RAM#34641
node: scale default -dbcache with system RAM#34641l0rinc wants to merge 4 commits intobitcoin:masterfrom
-dbcache with system RAM#34641Conversation
|
The following sections might be updated with supplementary metadata relevant to reviewers and maintainers. ReviewsSee the guideline for information on the review process.
If your review is incorrectly listed, please copy-paste ConflictsReviewers, this pull request conflicts with the following ones:
If you consider this pull request important, please also help to review the conflicting pull requests. Ideally, start with the one that should be merged first. |
30bfcdf to
0f93881
Compare
|
It took a few rounds to tame (I had a Kernel Panic myself when I saw missing |
doc/reduce-memory.md
Outdated
| - `-dbcache=<n>` - the UTXO database cache size, this defaults to `450`. The unit is MiB (1024). | ||
| - The minimum value for `-dbcache` is 4. | ||
| - A lower `-dbcache` makes initial sync time much longer. After the initial sync, the effect is less pronounced for most use-cases, unless fast validation of blocks is important, such as for mining. | ||
| - `-dbcache=<n>` - UTXO database cache size in MiB (minimum `4`). |
There was a problem hiding this comment.
Why are we changing this so much?
I think we should just remove this defaults to 450`` from this line, and add the below point:
- Automatic default scales with system RAM and can be as high as 1000` MiB.`
Then keep the rest the same.
There was a problem hiding this comment.
Updated, simplified, thanks.
doc/release-notes-34641.md
Outdated
| ---------------------------------- | ||
|
|
||
| - When `-dbcache` is not set explicitly, Bitcoin Core now chooses a RAM-aware default that can be as high as `3000` MiB. | ||
| Explicit `-dbcache` values continue to override the automatic default. (#34641) |
There was a problem hiding this comment.
I think we should also say something like:
"To maintain the same behavior of previous releases, set dbcache to 450."
src/node/caches.h
Outdated
| //! Automatic -dbcache floor (bytes) | ||
| static constexpr size_t MIN_DEFAULT_DBCACHE{100_MiB}; | ||
| //! Automatic -dbcache cap (bytes) | ||
| static constexpr size_t MAX_DEFAULT_DBCACHE{3000_MiB}; |
There was a problem hiding this comment.
I think we should be more conservative and set this to 1GB as discussed in the IRC meeting. Such a large bump in memory might be unexpected, especially when running bitcoind alongside other applications and not a dedicated node machine.
| static constexpr size_t MAX_DEFAULT_DBCACHE{3000_MiB}; | |
| static constexpr size_t MAX_DEFAULT_DBCACHE{1000_MiB}; |
There was a problem hiding this comment.
The 1 GB limit was a suggestion for a constant dbcache, which I opposed because it was too high for cheaper nodes (and too low for performant ones).
But since we can scale with total memory now, it makes sense to go higher, especially since 3-4 GB already achieves most of the speedup we saw.
This way, we have a simple formula instead of custom values for each memory.
| System RAM | Automatic default |
|---|---|
| 1 GiB | 100 MiB |
| 2 GiB | 100 MiB |
| 4 GiB | 512 MiB |
| 8 GiB | 1536 MiB |
| 16 GiB | 3000 MiB |
There was a problem hiding this comment.
I'm not too worried about this, because I suspect that most low-memory systems have explicit configurations anyway (either because they need it already, or because they're managed by node-in-a-box / distributions). And we should have loud and clear release notes for this.
There was a problem hiding this comment.
Not really about low memory systems. We don't clear the cache anymore, so if we sync and then go to steady state the node will use lots of cache for pretty much its lifetime. For instance it finishes IBD at 2GB, it will take maybe a year to fill up to 3GB and finally clear. That's just being a bad OS citizen using that much memory when we don't need it.
There was a problem hiding this comment.
Ok that sort of calls for a dbcache value that is different between IBD and later, which I guess makes sense if you're planning to run other things post-IBD. But on the other hand, a 1 GiB difference seems tiny for sufficiently large systems (which I suspect most non-dedicated setups are).
Assuming we don't want to do that in this PR, reducing to 2 GiB or 1 GiB default makes sense perhaps, but that reduces the benefit too.
There was a problem hiding this comment.
What is the difference in benefit? I think more than doubling the default (450->1000) is a pretty big speedup. What's going from 1->2->3?
I would think the initial bump would have the lions share of speedup. Could be wrong though.
There was a problem hiding this comment.
You may be right. I haven't actually used a dbcache setting below 8 GiB or so in years, so 🤷♂️.
There was a problem hiding this comment.
I'm measuring with 1 and 2 cap so that we can compare.
Other that that, does the formula make sense?
There was a problem hiding this comment.
There still looks like a substantial benefit to bumping to 1GB.
Another user posted the same concern as I https://x.com/lukechilds/status/2026875795080359955.
Perhaps it's worth doing a conservative bump for now, and work on a separate steady state dbcache value for v32. We could even have a lower general steady state value than 450MB, so running normally uses less RAM. Then we can bump to 3GB or more for IBD in v32.
There was a problem hiding this comment.
Yes, now that a few more measurements have come in, it's clear that the 2 GiB to 3 GiB speedup is negligible.
The 1 GiB to 2 GiB is still very much relevant - and affect 16 GiB machines only, 8 GiB machines were already capped at 1536. Changed the upper cap to 2 GiB for now.
I will push the steady state dbcache change after this change, separately, it makes more sense to restrict that once your parallelization PR lands and we don't need so much memory anyway.
0f93881 to
4e68c80
Compare
4e68c80 to
b67bb06
Compare
|
Q: Now that the total memory is needed for more than just the warning, should we mandate that this method works for the whole CI infra (cc: @hebasto), see bitcoin/src/test/system_ram_tests.cpp Lines 18 to 21 in 56791b5 I have addresses a few nits, I'm still measuring 1 and 2 GB dbcache values for comparison, added a few ones already to the PR description. The difference between 1 and 3 still seems relevant. I don't think a 16 GB machine should be surprised by that amount and an 8 GB is already capped at 1536 MB. Let's see the results and we can decide. |
A related discussion happened here: #33435 (review). We could make the test mandatory. If it fails in an unsupported environment, users can still explicitly disable it. |
hebasto
left a comment
There was a problem hiding this comment.
Could the new source files common/system_ram.cpp and node/dbcache.cpp be added to FILES_WITH_ENFORCED_IWYU?
willcl-ark
left a comment
There was a problem hiding this comment.
Concept ACK
Seems like a reasonable first step towards something more like #8437, which I've wanted to see for a long time... In the case that never happens, this seems perfectly worthwhile on it's own, too.
Just one comment so far, re. handling the implementation-defined truncation, but mostly LGTM.
b67bb06 to
7385c5b
Compare
|
Thanks @hebasto & @willcl-ark, addressed your concerns, added both as coauthor. |
The IWYU warnings in the CI seem to be legit. |
7385c5b to
0348bda
Compare
0348bda to
c4077b3
Compare
src/node/caches.h
Outdated
| //! -dbcache default (bytes) | ||
| static constexpr size_t DEFAULT_DB_CACHE{DEFAULT_KERNEL_CACHE}; | ||
| //! Automatic -dbcache floor (bytes) | ||
| static constexpr size_t MIN_DEFAULT_DBCACHE{100_MiB}; |
There was a problem hiding this comment.
I'm still not sure about reducing the default dbcache. The project has defaulted to 450 for nearly a decade. Looking through the issue tracker, I have not found too many users complaining about bitcoind oom-ing on low-RAM devices. Even the most popular raspberry pi installation guide bumps cache to 2GB: https://raspibolt.org/guide/bitcoin/bitcoin-client.html#configuration (I realize though that they are targeting devices with 4+GB of RAM). Do we really need to change this on the minimum side?
There was a problem hiding this comment.
I have not found too many users complaining about bitcoind oom-ing on low-RAM devices
I have tried it and we can't do it with 2 GB total memory currently with default dbcache. The memory usage grows with threads, with mempool, with sigcache, connection count, memory buffers, etc.
With 4 GB ram we're already increasing the dbcache in this PR, so reduction only happens for extremely constrained machines. A 2 GB machine can barely finish an IBD and only if we explicitly reduce the dbcache to 100 (and even this OOM-ed on some flushes). That's what we're doing here automatically now.
A 2 GB machine can't even compile the source code without explicit swapping, so it makes sense to make these more realistic.
There was a problem hiding this comment.
How does the same reasoning apply to virtualizations where e.g. 2GB are allocated to running bitcoind in a jail? Reading through some of the issues in the tracker that seems a likely deployment.
There was a problem hiding this comment.
I am running nodes in 2GB AWS EC2 instances just fine. Although mempool is fairly empty. I don't think IBD in such an instance is feasible though, with or without dbcache. Reducing this for steady state seems fine. I doubt users will notice.
Not sure why anyone would want to build on such an instance either.
src/common/system_ram.cpp
Outdated
| #ifdef WIN32 | ||
| if (MEMORYSTATUSEX m{}; (m.dwLength = sizeof(m), GlobalMemoryStatusEx(&m))) return clamp(m.ullTotalPhys); | ||
| #elif defined(__APPLE__) || defined(__FreeBSD__) || defined(__NetBSD__) || defined(__OpenBSD__) || defined(__illumos__) || defined(__linux__) | ||
| if (long p{sysconf(_SC_PHYS_PAGES)}, s{sysconf(_SC_PAGESIZE)}; p > 0 && s > 0) return clamp(1ULL * p * s); |
There was a problem hiding this comment.
I'm not sure if this will work correctly in a containerized environment. E.g. what happens if you create a docker container with bounded memory? From what I am gathering it will attempt to measure the hosts memory, not whatever the system administrator allocated for the container. I'm not sure if we can account for all these scenarios, meaning this change is likely to break a few deployments. A log message on init indicating what was measured and how much db cache allocation was scaled up would already go a ways to at least make this a bit more transparent.
There was a problem hiding this comment.
Thanks, this was exactly what I was working on after talking to @openoms.
We could extend the total memory calculator to take cgroups and BSD jails into consideration, but for now (to avoid bikeshedding here), I only added a warning if /sys/fs/cgroup exists.
That extra warning also made me realize that we are not displaying the cache sizes in a uniform way, and there is no point in pretending we care about decimals for MiB, but we do for GiB. Since GiB is now the expected unit for -dbcache, I unified the display of the cache units.
Total RAM is also cached now so that we can call it multiple times, and it was renamed to TryGetTotalRam to differentiate it from the alternative that falls back to the default value.
Also made GetDefaultCache more testable by reverting to system ran if the optional parameter is not provided.
And lastly I regrouped the tests (data separated from logic) to simplify the RAM-aware commit diff.
There was a problem hiding this comment.
Reverted many of these based on followup reviews
|
Reindex chainstate benchmark: A/B test bash script#!/usr/bin/env bash
set -euo pipefail
SRC_DIR="${SRC_DIR:-$HOME/bitcoin-core/review}"
COMMIT_A="${COMMIT_A:-701b8d714861874449c3360f31bdb01512f10644}"
COMMIT_B="${COMMIT_B:-fbbc6162c7e9ad4c13c99442849fc279b98803d3}"
STOP="${STOP:-930000}"
# DBCACHE="${DBCACHE:-450}"
DATA_DIR="${DATA_DIR:-/data1}"
JOBS="${JOBS:-$(nproc)}"
git reset --hard $COMMIT_A
cmake -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_GUI=OFF -DWITH_ZMQ=OFF -DBUILD_TESTS=OFF -DBUILD_BENCH=OFF --log-level=ERROR
ninja -C build bitcoind -j $JOBS
time ./build/bin/bitcoind -datadir=$DATA_DIR -stopatheight=$STOP -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 -daemon=0
git reset --hard $COMMIT_B
cmake -B build -DCMAKE_BUILD_TYPE=Release -DBUILD_GUI=OFF -DWITH_ZMQ=OFF -DBUILD_TESTS=OFF -DBUILD_BENCH=OFF --log-level=ERROR
ninja -C build bitcoind -j $JOBS
time ./build/bin/bitcoind -datadir=$DATA_DIR -stopatheight=$STOP -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 -daemon=0HardwareBefore After |
|
This is a "low-hanging fruit" to improve IBD performance on default config nodes. The RAM-dependent default value for dbcache makes a lot of sense, and it's great to see the emprical data points collected! Some comments follow. |
|
A relevant data point is the current size of the UTXO set. This info could be added, mainly for reference for the future. The message of the commit where the default is computed would be a good place for that. |
|
To add a bit more structure, the 3 values of recommended default, recommended minimum and recommended maximum could be computed in the very same location (the latter two are used only for potential warnings in case a config override is supplied). |
|
Since 32/64-bit system distinction is made sometimes around memory size checks (e.g. in tests in this PR), it would make sense to formalize that as well, as a property of the system, similar to memory size. A common method in |
src/test/caches_tests.cpp
Outdated
| BOOST_CHECK(!ShouldWarnOversizedDbCache(/*dbcache=*/1500_MiB, /*total_ram=*/2048_MiB)); // Under cap | ||
| BOOST_CHECK( ShouldWarnOversizedDbCache(/*dbcache=*/1600_MiB, /*total_ram=*/2048_MiB)); // Over cap | ||
| constexpr size_t total_ram{3072_MiB}; | ||
| BOOST_CHECK_EQUAL(GetDefaultCache(total_ram), (total_ram - RESERVED_RAM) / 4); |
There was a problem hiding this comment.
Nit: A concrete expected value would make more sense here IMHO, instead of repeating the implementation formula.
4ae9a10 doc: add release notes for dbcache bump (Andrew Toth) c510d12 doc: update dbcache default in reduce-memory.md (Andrew Toth) 027cac8 qt: show GetDefaultDBCache() in settings (Andrew Toth) 5b34f25 dbcache: bump default from 450MB -> 1024MB if enough memory (Andrew Toth) Pull request description: Alternative to #34641 This increases the default `dbcache` value from `450MiB` to `1024MiB` if: - `dbcache` is unset - The system is 64 bit - At least 4GiB of RAM is detected Otherwise fallback to previous `450MiB` default. This should be simple enough to get into v31. The bump to 1GiB shows significant performance increases in #34641. It also alleviates concerns of too high default for steady state, and of lowering the current dbcache for systems with less RAM. This change only changes bitcoind behavior, while kernel still defaults to 450 MiB. ACKs for top commit: ajtowns: ACK 4ae9a10 kevkevinpal: reACK [4ae9a10](4ae9a10) svanstaa: ACK [4ae9a10](4ae9a10) achow101: ACK 4ae9a10 sipa: ACK 4ae9a10 Tree-SHA512: ee3acf1fb08523ac80e37ec8f0caca226ffde6667f3a75ae6f4f4f54bc905a883ebcf1bf0e8a8a15c7cfabff96c23225825b3fff4506b9ab9936bf2c0a2c2513
Move the OS-dependent total RAM query from `common/system.cpp` into a dedicated `common/system_ram.cpp` translation unit and declare it in `common/system_ram.h`. Update call sites to include `common/system_ram.h` and rename `GetTotalRAM()` to `TryGetTotalRam()`. Add `common/system_ram` to `FILES_WITH_ENFORCED_IWYU` so this new source is checked at error level. Since we use it for more than just warnings now, make `system_ram_tests` require RAM detection instead of skipping when unavailable. Co-authored-by: sipa <sipa@bitcoincore.org> Co-authored-by: Hennadii Stepanov <32963518+hebasto@users.noreply.github.com>
a6ec8c9 to
363c043
Compare
| argsman.AddArg("-conf=<file>", strprintf("Specify path to read-only configuration file. Relative paths will be prefixed by datadir location (only useable from command line, not configuration file) (default: %s)", BITCOIN_CONF_FILENAME), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS); | ||
| argsman.AddArg("-datadir=<dir>", "Specify data directory", ArgsManager::ALLOW_ANY | ArgsManager::DISALLOW_NEGATION, OptionsCategory::OPTIONS); | ||
| argsman.AddArg("-dbbatchsize", strprintf("Maximum database write batch size in bytes (default: %u)", DEFAULT_DB_CACHE_BATCH), ArgsManager::ALLOW_ANY | ArgsManager::DEBUG_ONLY, OptionsCategory::OPTIONS); | ||
| argsman.AddArg("-dbcache=<n>", strprintf("Maximum database cache size <n> MiB (minimum %d, default: %d). Make sure you have enough RAM. In addition, unused memory allocated to the mempool is shared with this cache (see -maxmempool).", MIN_DB_CACHE >> 20, node::GetDefaultDBCache() >> 20), ArgsManager::ALLOW_ANY, OptionsCategory::OPTIONS); |
There was a problem hiding this comment.
I don't see why users suddenly don't need to make sure they have enough RAM..
There was a problem hiding this comment.
Since we have exact warnings for when that's the case, instead of a general warning, see the commit message:
And now that we have warnings for excessive dbcache in the logs, we can remove the warning from
-dbcachearg doc.
The current `dbcache` policy is split across `node`, `qt`, and `kernel`. The node startup path already chooses between `450 MiB` and `1024 MiB` based on detected RAM, while `bitcoinkernel` and the Qt settings migration still use fixed `450 MiB` values. Move the default selection and oversized warning helpers into dedicated `node/dbcache` code and use them from all current callers. This keeps the existing default selection logic in one place and makes the other users follow that same policy. This also changes the low-memory oversized warning to use the shared default helper instead of a fixed `450 MiB` cap. It changes `bitcoinkernel` and the Qt migration on 64-bit systems with at least 4 GiB RAM, where they now use the same `1024 MiB` default as the node path instead of a fixed `450 MiB`. Co-authored-by: Luke Dashjr <luke-jr+git@utopios.org>
The fixed `-dbcache` default of 450 MiB is too high for small systems and too conservative for larger ones. Compute the automatic default as `std::clamp((max(total_ram - 2 GiB, 0) / 4), 100 MiB, 2 GiB)`. The 2 GB upper limit was chosen because it's still a lot faster than 1 GiB, but barely faster than 3 GiB. When RAM detection is unavailable, fall back to 4 GiB on 64-bit and 2 GiB on 32-bit. Use that fallback for default calculation and logging. Align oversized `-dbcache` warnings with this policy on low-memory systems. For total RAM below `FALLBACK_RAM_BYTES`, warn when the configured value exceeds the automatic default. Keep the 75% of total RAM warning cap for larger systems. Log the automatically selected default when `-dbcache` is not explicitly set. And now that we have warnings for excessive dbcache in the logs, we can remove the warning from `-dbcache` arg doc. Also assert the default `dbcache` never triggers the oversized warning. Co-authored-by: Pieter Wuille <pieter@wuille.net> Co-authored-by: willcl-ark <will8clark@gmail.com> Co-authored-by: ismaelsadeeq <abubakarsadiqismail@proton.me>
Also update related docs. Co-authored-by: Andrew Toth <andrewstoth@gmail.com>
363c043 to
8ff5e8a
Compare
Yes, it was because the other change ignored kernel and the warning test failed, so it was easier to just revert it at the beginning. |
Bortlesboat
left a comment
There was a problem hiding this comment.
Concept ACK 8ff5e8a. Clean separation into system_ram.h and dbcache.h, and the (total_ram - RESERVED) / 4 formula is intuitive. Left one question about the warning threshold.
|
|
||
| bool ShouldWarnOversizedDbCache(size_t dbcache, size_t total_ram) noexcept | ||
| { | ||
| return (total_ram < FALLBACK_RAM_BYTES) ? dbcache > GetDefaultDBCache(total_ram) |
There was a problem hiding this comment.
Is the warning threshold discontinuity at FALLBACK_RAM_BYTES intentional? At 4095 MiB the warning trips at >511 MiB (the auto default), but at 4096 MiB it jumps to >3072 MiB (75% of RAM) — a 6x jump from 1 MiB of extra RAM. Would it make sense to use the 75% path unconditionally, or smooth the transition?
|
One follow-up question: now that #34692 merged with a fixed 1 GiB default for the kernel, this PR replaces |
|
|
||
| std::optional<size_t> GetTotalRAM() | ||
| { | ||
| [[maybe_unused]] auto clamp{[](uint64_t v) { return size_t(std::min(v, uint64_t{std::numeric_limits<size_t>::max()})); }}; |
There was a problem hiding this comment.
Probably should be a global utility function (templated?) somewhere else
Consider renaming it to clamp_to_type or something that can't be confused with std::clamp
There was a problem hiding this comment.
This code was just moved in this PR, the naming was deliberate - but I'll consider it next time I push, thanks for the hint
| [[maybe_unused]] auto clamp{[](uint64_t v) { return size_t(std::min(v, uint64_t{std::numeric_limits<size_t>::max()})); }}; | ||
| #ifdef WIN32 | ||
| if (MEMORYSTATUSEX m{}; (m.dwLength = sizeof(m), GlobalMemoryStatusEx(&m))) return clamp(m.ullTotalPhys); | ||
| #elif defined(__APPLE__) || \ | ||
| defined(__FreeBSD__) || \ | ||
| defined(__NetBSD__) || \ | ||
| defined(__OpenBSD__) || \ | ||
| defined(__illumos__) || \ | ||
| defined(__linux__) | ||
| if (long p{sysconf(_SC_PHYS_PAGES)}, s{sysconf(_SC_PAGESIZE)}; p > 0 && s > 0) return clamp(1ULL * p * s); | ||
| #endif |
There was a problem hiding this comment.
This all-on-one-line style is annoying to read. Suggest formatting it normally.
Bortlesboat
left a comment
There was a problem hiding this comment.
Tested ACK 8ff5e8a. Built and ran caches_tests + system_ram_tests locally, all 5 cases pass. The test coverage is solid — default_dbcache_never_warns is a nice invariant check, and the CheckDbCacheWarnThreshold helper makes the boundary tests much cleaner than the old inline checks. The system_ram_tests simplification (Assert instead of optional+early-return) makes sense given this should never fail on CI hardware.
Please stop these useless bot responses, we need real reviews |
|
Apologies |
| static constexpr size_t FALLBACK_RAM_BYTES{SIZE_MAX == UINT64_MAX ? 4096_MiB : 2048_MiB}; | ||
| //! Reserved non-dbcache memory usage. | ||
| static constexpr size_t RESERVED_RAM{2048_MiB}; | ||
| //! Maximum dbcache size on current architecture. | ||
| static constexpr size_t MAX_DBCACHE_BYTES{SIZE_MAX == UINT64_MAX ? std::numeric_limits<size_t>::max() : 1024_MiB}; |
There was a problem hiding this comment.
== seems too strict for this. Probably should be SIZE_MAX > UINT32_MAX
There was a problem hiding this comment.
How so? My understanding is that we only support 32-bit and 64-bit architectures.
There was a problem hiding this comment.
How about sizeof(void*) == 8, I've got the impression that this is used more often in the codebase (and it's also my weak personal preference).
There was a problem hiding this comment.
We can use that as well. I deliberately chose the version where the constant we compare against contains 64 explicitly.
There was a problem hiding this comment.
I suggest using sizeof(void*), on grounds of consistency. A quick search on sizeof(void*) yielded 10 instances where it is used to differentiate 32 vs. 64-bit systems. For SIZE_MAX I found only 2 (both in tests).
Problem
-dbcachecontrols how much RAM Bitcoin Core uses for the in-memory view of the UTXO set, while the authoritative chainstate remains on disk.The default is still the fixed
450 MiBvalue, chosen almost a decade ago, even though chainstate size and typical hardware profiles have changed significantly since then.In practice, a fixed
450 MiBdefault is often too conservative on modern systems, where larger cache sizes can reduce flush churn and disk roundtrips during validation, especially in IBD. At the same time, memory pressure remains important on smaller systems.Historical changes
IRC
The discussion converged on avoiding a single new fixed
-dbcachevalue and moving to RAM-aware defaults instead, balancing performance gains on modern hardware against OOM risk on lower-memory systems.IRC log links
Node provider defaults
Containerization
Container memory constraints may or may not be reflected by RAM detection.
We have considered adding container-detection but decided against it. Startup logs show detected/assumed system memory so users can set
-dbcacheexplicitly when needed.Fix
Measuring
-reindex-chainstateon different machines indicates that 450-1024-2048 jumps are all very relevant after which the effects taper off.Replace the fixed default with a RAM-aware policy:
If system RAM cannot be determined, assume 4 GiB on 64-bit builds and 2 GiB on 32-bit builds.
Manual
-dbcachevalues are unchanged and still override the automatic default.Warning threshold update
Oversized
-dbcachewarnings are kept and aligned with the new policy - warn if:dbcache > default-dbcachewhenRAM < fallback RAM(4 GiB on 64-bit, 2 GiB on 32-bit)dbcache>75% of total RAM, otherwisePerformance measurements
reindex-chainstate
N150 (Umbrel) - 1.18x faster
Rpi5 (16) - 1.24x faster
Rpi5 (8) - 1.43x faster
Apple M4 Max - 2.46x faster
Benchmark 1: ./build/bin/bitcoind -datadir="/Users/lorinc/Library/Application Support/Bitcoin" -stopatheight=900000 -dbcache=450 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 097c18239b58a8ee03794b3ebc6e722a0da30d8d) Time (abs ≡): 20261.315 s [User: 18768.521 s, System: 4271.673 s]Benchmark 1: ./build/bin/bitcoind -datadir="/Users/lorinc/Library/Application Support/Bitcoin" -stopatheight=900000 -dbcache=1000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 097c18239b58a8ee03794b3ebc6e722a0da30d8d) Time (abs ≡): 11574.620 s [User: 12147.446 s, System: 2417.914 s]Benchmark 1: ./build/bin/bitcoind -datadir="/Users/lorinc/Library/Application Support/Bitcoin" -stopatheight=900000 -dbcache=2000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 097c18239b58a8ee03794b3ebc6e722a0da30d8d) Time (abs ≡): 8226.676 s [User: 9083.949 s, System: 1179.971 s]Benchmark 1: ./build/bin/bitcoind -datadir="/Users/lorinc/Library/Application Support/Bitcoin" -stopatheight=900000 -dbcache=3000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 097c18239b58a8ee03794b3ebc6e722a0da30d8d) Time (abs ≡): 7064.184 s [User: 7701.391 s, System: 668.060 s]i7 HDD - 1.51x faster
Windows Xeon - 1.24x faster
Benchmark 1: ./build/bin/bitcoind.exe -datadir="C:\\my_storage\\BitcoinData" -stopatheight=933339 -dbcache=450 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 5401e673d56198f2c0bad366581e70d5d9cd765c) Time (abs ≡): 37691.648 s [User: 0.000 s, System: 0.001 s]Benchmark 1: ./build/bin/bitcoind.exe -datadir="C:\\my_storage\\BitcoinData" -stopatheight=933339 -dbcache=1000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 5401e673d56198f2c0bad366581e70d5d9cd765c) Time (abs ≡): 34250.996 s [User: 0.002 s, System: 0.000 s]Benchmark 1: ./build/bin/bitcoind.exe -datadir="C:\\my_storage\\BitcoinData" -stopatheight=933339 -dbcache=2000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 5401e673d56198f2c0bad366581e70d5d9cd765c) Time (abs ≡): 31001.091 s [User: 0.000 s, System: 0.002 s]Benchmark 2: ./build/bin/bitcoind.exe -datadir="C:\\my_storage\\BitcoinData" -stopatheight=933339 -dbcache=3000 -reindex-chainstate -blocksonly -connect=0 -printtoconsole=0 (COMMIT = 5401e673d56198f2c0bad366581e70d5d9cd765c) Time (abs ≡): 30276.311 s [User: 0.001 s, System: 0.000 s]i9 - 1.20x faster
Quick reference (64 bit)