Up to 100x perfomance difference for sqlite vs BDB in CreateWallet test #21628
Description
Building master @ 41a8d2b (sqlite):
./autogen.sh
./configure --without-bdb --without-gui --with-sqlite
with wallet = yes
with sqlite = yes
with bdb = no
gmake -j8
src/test/test_bitcoin --run_test=wallet_tests --log_level=all
...
wallet/test/wallet_tests.cpp:696: Entering test case "CreateWallet"
wallet/test/wallet_tests.cpp:69: info: check SignTransaction(mtx, &keystore, coins, SIGHASH_ALL, input_errors) has passed
wallet/test/wallet_tests.cpp:69: info: check SignTransaction(mtx, &keystore, coins, SIGHASH_ALL, input_errors) has passed
wallet/test/wallet_tests.cpp:734: info: check m_node.chain->broadcastTransaction(MakeTransactionRef(mempool_tx), DEFAULT_TRANSACTION_MAXFEE, false, error) has passed
wallet/test/wallet_tests.cpp:740: info: check rescan_completed has passed
wallet/test/wallet_tests.cpp:741: info: check addtx_count == 2 has passed
wallet/test/wallet_tests.cpp:744: info: check wallet->mapWallet.count(block_tx.GetHash()) == 1U has passed
wallet/test/wallet_tests.cpp:745: info: check wallet->mapWallet.count(mempool_tx.GetHash()) == 1U has passed
wallet/test/wallet_tests.cpp:753: info: check addtx_count == 4 has passed
wallet/test/wallet_tests.cpp:767: info: check rescan_completed has passed
wallet/test/wallet_tests.cpp:69: info: check SignTransaction(mtx, &keystore, coins, SIGHASH_ALL, input_errors) has passed
wallet/test/wallet_tests.cpp:69: info: check SignTransaction(mtx, &keystore, coins, SIGHASH_ALL, input_errors) has passed
wallet/test/wallet_tests.cpp:772: info: check m_node.chain->broadcastTransaction(MakeTransactionRef(mempool_tx), DEFAULT_TRANSACTION_MAXFEE, false, error) has passed
wallet/test/wallet_tests.cpp:780: info: check addtx_count == 4 has passed
wallet/test/wallet_tests.cpp:783: info: check wallet->mapWallet.count(block_tx.GetHash()) == 1U has passed
wallet/test/wallet_tests.cpp:784: info: check wallet->mapWallet.count(mempool_tx.GetHash()) == 1U has passed
wallet/test/wallet_tests.cpp:696: Leaving test case "CreateWallet"; testing time: 201296644us
wallet/test/wallet_tests.cpp:791: Entering test case "ZapSelectTx"
wallet/test/wallet_tests.cpp:69: info: check SignTransaction(mtx, &keystore, coins, SIGHASH_ALL, input_errors) has passed
wallet/test/wallet_tests.cpp:810: info: check wallet->HasWalletSpend(prev_hash) has passed
wallet/test/wallet_tests.cpp:811: info: check wallet->mapWallet.count(block_hash) == 1u has passed
wallet/test/wallet_tests.cpp:814: info: check wallet->ZapSelectTx(vHashIn, vHashOut) == DBErrors::LOAD_OK has passed
wallet/test/wallet_tests.cpp:816: info: check !wallet->HasWalletSpend(prev_hash) has passed
wallet/test/wallet_tests.cpp:817: info: check wallet->mapWallet.count(block_hash) == 0u has passed
wallet/test/wallet_tests.cpp:791: Leaving test case "ZapSelectTx"; testing time: 200866629usCompare that to running the same tests after building with just bdb:
gmake clean
./configure --without-gui --with-sqlite=no
with wallet = yes
with sqlite = no
with bdb = yes
gmake -j8
src/test/test_bitcoin --run_test=wallet_tests --log_level=all
...
wallet/test/wallet_tests.cpp:696: Entering test case "CreateWallet"
wallet/test/wallet_tests.cpp:69: info: check SignTransaction(mtx, &keystore, coins, SIGHASH_ALL, input_errors) has passed
wallet/test/wallet_tests.cpp:69: info: check SignTransaction(mtx, &keystore, coins, SIGHASH_ALL, input_errors) has passed
wallet/test/wallet_tests.cpp:734: info: check m_node.chain->broadcastTransaction(MakeTransactionRef(mempool_tx), DEFAULT_TRANSACTION_MAXFEE, false, error) has passed
wallet/test/wallet_tests.cpp:740: info: check rescan_completed has passed
wallet/test/wallet_tests.cpp:741: info: check addtx_count == 2 has passed
wallet/test/wallet_tests.cpp:744: info: check wallet->mapWallet.count(block_tx.GetHash()) == 1U has passed
wallet/test/wallet_tests.cpp:745: info: check wallet->mapWallet.count(mempool_tx.GetHash()) == 1U has passed
wallet/test/wallet_tests.cpp:753: info: check addtx_count == 4 has passed
wallet/test/wallet_tests.cpp:767: info: check rescan_completed has passed
wallet/test/wallet_tests.cpp:69: info: check SignTransaction(mtx, &keystore, coins, SIGHASH_ALL, input_errors) has passed
wallet/test/wallet_tests.cpp:69: info: check SignTransaction(mtx, &keystore, coins, SIGHASH_ALL, input_errors) has passed
wallet/test/wallet_tests.cpp:772: info: check m_node.chain->broadcastTransaction(MakeTransactionRef(mempool_tx), DEFAULT_TRANSACTION_MAXFEE, false, error) has passed
wallet/test/wallet_tests.cpp:780: info: check addtx_count == 4 has passed
wallet/test/wallet_tests.cpp:783: info: check wallet->mapWallet.count(block_tx.GetHash()) == 1U has passed
wallet/test/wallet_tests.cpp:784: info: check wallet->mapWallet.count(mempool_tx.GetHash()) == 1U has passed
wallet/test/wallet_tests.cpp:696: Leaving test case "CreateWallet"; testing time: 2139054us
wallet/test/wallet_tests.cpp:791: Entering test case "ZapSelectTx"
wallet/test/wallet_tests.cpp:69: info: check SignTransaction(mtx, &keystore, coins, SIGHASH_ALL, input_errors) has passed
wallet/test/wallet_tests.cpp:810: info: check wallet->HasWalletSpend(prev_hash) has passed
wallet/test/wallet_tests.cpp:811: info: check wallet->mapWallet.count(block_hash) == 1u has passed
wallet/test/wallet_tests.cpp:814: info: check wallet->ZapSelectTx(vHashIn, vHashOut) == DBErrors::LOAD_OK has passed
wallet/test/wallet_tests.cpp:816: info: check !wallet->HasWalletSpend(prev_hash) has passed
wallet/test/wallet_tests.cpp:817: info: check wallet->mapWallet.count(block_hash) == 0u has passed
wallet/test/wallet_tests.cpp:791: Leaving test case "ZapSelectTx"; testing time: 1821042usIn the sqlite case,CreateWallet is taking ~100x longer: 201296644us vs 2139054us. ZapWalletTx is similar: 200866629us vs 1821042us. Full logs here.
I thought this might be some weird macOS behaviour, but I tested on Linux, and am also seeing up to 27x difference. CreateWallet: 27905993us vs 1092394us. ZapWalletTx: 28024310us vs 969432us. Full logs here.