Update libmultiprocess subtree to fix race conditions on disconnects by ryanofsky · Pull Request #34804 · bitcoin/bitcoin

ryanofsky · 2026-03-11T14:47:18Z

This is a draft because it depends on bitcoin-core/libmultiprocess#250

Includes:

doc: Bump version 8 > 9 bitcoin-core/libmultiprocess#246
proxy-types: add CustomHasField hook to map Cap'n Proto values to null C++ values bitcoin-core/libmultiprocess#242
type-map: Work around LLVM 22 "out of bounds index" error bitcoin-core/libmultiprocess#247
fixes for race conditions on disconnects bitcoin-core/libmultiprocess#249

The main change is bitcoin-core/libmultiprocess#249 which fixes 3 intermittent race conditions detected in bitcoin core CI and antithesis: #34711/#34756, #34777, and #34782.

The changes can be verified by running test/lint/git-subtree-check.sh src/ipc/libmultiprocess as described in developer notes and lint instructions

…cca9 2fb97e8cca9 race fix: m_on_cancel called after request finishes 846a43aafb4 test: m_on_cancel called after request finishes e69b6bf3f4e race fix: getParams() called after request cancel 75c5425173f test: getParams() called after request cancel f09731e242f race fix: worker thread destroyed before it is initialized 88cacd4239f test: worker thread destroyed before it is initialized 22bec918c97 Merge bitcoin-core/libmultiprocess#247: type-map: Work around LLVM 22 "out of bounds index" error 8a5e3ae6ed2 Merge bitcoin-core/libmultiprocess#242: proxy-types: add CustomHasField hook to map Cap'n Proto values to null C++ values e8d35246918 Merge bitcoin-core/libmultiprocess#246: doc: Bump version 8 > 9 97d877053b6 proxy-types: add CustomHasField hook for nullable decode paths 8c2f10252c9 refactor: add missing includes to mp/type-data.h b1638aceb40 doc: Bump version 8 > 9 f61af487217 type-map: Work around LLVM 22 "out of bounds index" error git-subtree-dir: src/ipc/libmultiprocess git-subtree-split: 2fb97e8cca9feb1df70cf29b2a9895bea2c4c49c

…ee-9

DrahtBot · 2026-03-11T14:48:00Z

The following sections might be updated with supplementary metadata relevant to reviewers and maintainers.

Reviews

See the guideline for information on the review process.
A summary of reviews will appear here.

LLM Linter (✨ experimental)

Possible typos and grammar issues:

neccessary -> necessary [spelling error in test comment: "Sleep is neccessary..." should be "necessary".]
execute an async requests -> executes an async request [subject-verb agreement and number: "after it execute an async requests" should be "after it executes an async request".]

^{2026-03-12 18:09:44}

ryanofsky · 2026-03-12T11:47:19Z

Rebased af35175 -> 6fd7194 (pr/subtree-9.1 -> pr/subtree-9.2, compare) adding new test from updated bitcoin-core/libmultiprocess#250

dergoegge · 2026-03-12T12:02:34Z

Tested this PR with Antithesis and the bugs it previously found seem to no longer reproduce.

fanquake · 2026-03-12T14:21:33Z

https://github.com/bitcoin/bitcoin/actions/runs/23000375935/job/66791524592?pr=34804#step:11:3292

WARNING: ThreadSanitizer: data race (pid=21189)
  Write of size 8 at 0x7208000030c8 by thread T2:
    #0 operator delete(void*, unsigned long) <null> (bitcoin-node+0x1f093c) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #1 kj::_::HeapDisposer<mp::ProxyServer<mp::ThreadMap>>::disposeImpl(void*) const /home/admin/actions-runner/_work/_temp/depends/x86_64-pc-linux-gnu/include/kj/memory.h:557:60 (bitcoin-node+0x8291a5) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #2 kj::Disposer::Dispose_<capnp::Capability::Server, true>::dispose(capnp::Capability::Server*, kj::Disposer const&) <null> (bitcoin-node+0xe2dc91) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #3 void kj::Disposer::dispose<capnp::Capability::Server>(capnp::Capability::Server*) const <null> (bitcoin-node+0xe2dbe5) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #4 kj::Own<capnp::Capability::Server, std::nullptr_t>::dispose() <null> (bitcoin-node+0xe5ed6e) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #5 kj::Own<capnp::Capability::Server, std::nullptr_t>::~Own() <null> (bitcoin-node+0xe2a695) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #6 kj::Maybe<kj::Own<capnp::Capability::Server, std::nullptr_t>>::~Maybe() <null> (bitcoin-node+0xe2d395) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #7 capnp::LocalClient::~LocalClient() <null> (bitcoin-node+0xe615f8) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #8 capnp::LocalClient::~LocalClient() <null> (bitcoin-node+0xe616f9) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #9 non-virtual thunk to capnp::LocalClient::~LocalClient() <null> (bitcoin-node+0xe617c9) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #10 kj::Refcounted::disposeImpl(void*) const <null> (bitcoin-node+0x122a954) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #11 kj::Disposer::Dispose_<capnp::ClientHook, true>::dispose(capnp::ClientHook*, kj::Disposer const&) <null> (bitcoin-node+0xe35971) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #12 void kj::Disposer::dispose<capnp::ClientHook>(capnp::ClientHook*) const <null> (bitcoin-node+0xe358c5) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #13 kj::Own<capnp::ClientHook, std::nullptr_t>::dispose() <null> (bitcoin-node+0xe3586e) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #14 kj::Own<capnp::ClientHook, std::nullptr_t>::~Own() <null> (bitcoin-node+0xe29585) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #15 capnp::_::(anonymous namespace)::RpcConnectionState::Export::~Export() rpc.c++ (bitcoin-node+0xe860dc) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #16 capnp::_::(anonymous namespace)::RpcConnectionState::releaseExport(unsigned int, unsigned int) rpc.c++ (bitcoin-node+0xe8e639) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #17 capnp::_::(anonymous namespace)::RpcConnectionState::releaseExports(kj::ArrayPtr<unsigned int>) rpc.c++ (bitcoin-node+0xe9bff5) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #18 capnp::_::(anonymous namespace)::RpcConnectionState::handleFinish(capnp::rpc::Finish::Reader const&)::'lambda'()::operator()() const rpc.c++ (bitcoin-node+0xee7e95) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #19 kj::_::Deferred<capnp::_::(anonymous namespace)::RpcConnectionState::handleFinish(capnp::rpc::Finish::Reader const&)::'lambda'()>::run() rpc.c++ (bitcoin-node+0xee7c9a) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #20 kj::_::Deferred<capnp::_::(anonymous namespace)::RpcConnectionState::handleFinish(capnp::rpc::Finish::Reader const&)::'lambda'()>::~Deferred() rpc.c++ (bitcoin-node+0xee6769) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #21 capnp::_::(anonymous namespace)::RpcConnectionState::handleFinish(capnp::rpc::Finish::Reader const&) rpc.c++ (bitcoin-node+0xe8d59d) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #22 capnp::_::(anonymous namespace)::RpcConnectionState::handleMessage(kj::Own<capnp::IncomingRpcMessage, std::nullptr_t>) rpc.c++ (bitcoin-node+0xe8a593) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #23 capnp::_::(anonymous namespace)::RpcConnectionState::messageLoop()::'lambda'(kj::Maybe<kj::Own<capnp::IncomingRpcMessage, std::nullptr_t>>&&)::operator()(kj::Maybe<kj::Own<capnp::IncomingRpcMessage, std::nullptr_t>>&&) const rpc.c++ (bitcoin-node+0xe8a1eb) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #24 bool kj::_::MaybeVoidCaller<kj::Maybe<kj::Own<capnp::IncomingRpcMessage, std::nullptr_t>>, bool>::apply<capnp::_::(anonymous namespace)::RpcConnectionState::messageLoop()::'lambda'(kj::Maybe<kj::Own<capnp::IncomingRpcMessage, std::nullptr_t>>&&)>(capnp::_::(anonymous namespace)::RpcConnectionState::messageLoop()::'lambda'(kj::Maybe<kj::Own<capnp::IncomingRpcMessage, std::nullptr_t>>&&)&, kj::Maybe<kj::Own<capnp::IncomingRpcMessage, std::nullptr_t>>&&) rpc.c++ (bitcoin-node+0xeefc35) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #25 kj::_::TransformPromiseNode<bool, kj::Maybe<kj::Own<capnp::IncomingRpcMessage, std::nullptr_t>>, capnp::_::(anonymous namespace)::RpcConnectionState::messageLoop()::'lambda'(kj::Maybe<kj::Own<capnp::IncomingRpcMessage, std::nullptr_t>>&&), capnp::_::(anonymous namespace)::RpcConnectionState::messageLoop()::'lambda'(kj::Exception&&)>::getImpl(kj::_::ExceptionOrValue&) rpc.c++ (bitcoin-node+0xeef841) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #26 kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&)::$_0::operator()() const async.c++ (bitcoin-node+0x10716f3) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #27 kj::Maybe<kj::Exception> kj::runCatchingExceptions<kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&)::$_0>(kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&)::$_0&&) async.c++ (bitcoin-node+0x1061bb7) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #28 kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&) <null> (bitcoin-node+0x1061aab) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #29 kj::_::TransformPromiseNodeBase::getDepResult(kj::_::ExceptionOrValue&) <null> (bitcoin-node+0x1061e62) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #30 kj::_::TransformPromiseNode<kj::_::Void, bool, capnp::_::(anonymous namespace)::RpcConnectionState::messageLoop()::'lambda'(bool), kj::_::PropagateException>::getImpl(kj::_::ExceptionOrValue&) rpc.c++ (bitcoin-node+0xef1bd3) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #31 kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&)::$_0::operator()() const async.c++ (bitcoin-node+0x10716f3) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #32 kj::Maybe<kj::Exception> kj::runCatchingExceptions<kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&)::$_0>(kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&)::$_0&&) async.c++ (bitcoin-node+0x1061bb7) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #33 kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&) <null> (bitcoin-node+0x1061aab) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #34 kj::TaskSet::Task::fire() <null> (bitcoin-node+0x108bf23) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #35 non-virtual thunk to kj::TaskSet::Task::fire() <null> (bitcoin-node+0x108c459) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #36 kj::EventLoop::turn() <null> (bitcoin-node+0x105c635) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #37 kj::_::waitImpl(kj::Own<kj::_::PromiseNode, kj::_::PromiseDisposer>&&, kj::_::ExceptionOrValue&, kj::WaitScope&, kj::SourceLocation)::$_2::operator()() const async.c++ (bitcoin-node+0x1070484) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #38 void kj::WaitScope::runOnStackPool<kj::_::waitImpl(kj::Own<kj::_::PromiseNode, kj::_::PromiseDisposer>&&, kj::_::ExceptionOrValue&, kj::WaitScope&, kj::SourceLocation)::$_2>(kj::_::waitImpl(kj::Own<kj::_::PromiseNode, kj::_::PromiseDisposer>&&, kj::_::ExceptionOrValue&, kj::WaitScope&, kj::SourceLocation)::$_2&&) async.c++ (bitcoin-node+0x105e5a7) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #39 kj::_::waitImpl(kj::Own<kj::_::PromiseNode, kj::_::PromiseDisposer>&&, kj::_::ExceptionOrValue&, kj::WaitScope&, kj::SourceLocation) <null> (bitcoin-node+0x105e121) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #40 kj::Promise<unsigned long>::wait(kj::WaitScope&, kj::SourceLocation) /home/admin/actions-runner/_work/_temp/depends/x86_64-pc-linux-gnu/include/kj/async-inl.h:1359:3 (bitcoin-node+0xe17187) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #41 mp::EventLoop::loop() /home/admin/actions-runner/_work/_temp/src/ipc/libmultiprocess/src/mp/proxy.cpp:244:68 (bitcoin-node+0xe11d71) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #42 ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'()::operator()() const /home/admin/actions-runner/_work/_temp/src/ipc/capnp/protocol.cpp:136:21 (bitcoin-node+0x819606) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #43 std::__1::__invoke_result_impl<void, ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'()>::type std::__1::__invoke[abi:dee220100]<ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'()>(ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'()&&) /cxx_build/include/c++/v1/__type_traits/invoke.h:90:27 (bitcoin-node+0x819606)
    #44 void std::__1::__thread_execute[abi:dee220100]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'(), 0ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'()>&, std::__1::__integer_sequence<unsigned long, 0ul>) /cxx_build/include/c++/v1/__thread/thread.h:161:3 (bitcoin-node+0x819606)
    #45 void* std::__1::__thread_proxy[abi:dee220100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'()>>(void*) /cxx_build/include/c++/v1/__thread/thread.h:169:3 (bitcoin-node+0x819606)

  Previous read of size 8 at 0x7208000030c8 by thread T11 (mutexes: write M0):
    #0 mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0::operator()() const /home/admin/actions-runner/_work/_temp/src/ipc/libmultiprocess/src/mp/proxy.cpp:421:19 (bitcoin-node+0xe15c65) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #1 std::__1::__invoke_result_impl<void, mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>::type std::__1::__invoke[abi:dee220100]<mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>(mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0&&) /cxx_build/include/c++/v1/__type_traits/invoke.h:90:27 (bitcoin-node+0xe15c65)
    #2 void std::__1::__thread_execute[abi:dee220100]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0, 0ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>&, std::__1::__integer_sequence<unsigned long, 0ul>) /cxx_build/include/c++/v1/__thread/thread.h:161:3 (bitcoin-node+0xe15c65)
    #3 void* std::__1::__thread_proxy[abi:dee220100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>>(void*) /cxx_build/include/c++/v1/__thread/thread.h:169:3 (bitcoin-node+0xe15c65)

  Mutex M0 (0x721c00023000) created at:
    #0 pthread_mutex_lock <null> (bitcoin-node+0x16c03b) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #1 __libcpp_mutex_lock /llvm-project/libcxx/include/__thread/support/pthread.h:95:10 (libc++.so.1+0x8e019) (BuildId: 73e2cbaab3a26ac81ab0ad8de91c2952a3321d81)
    #2 std::__1::mutex::lock() /llvm-project/libcxx/src/mutex.cpp:30:12 (libc++.so.1+0x8e019)
    #3 std::__1::unique_lock<std::__1::mutex>::unique_lock[abi:dee220100](std::__1::mutex&) /cxx_build/include/c++/v1/__mutex/unique_lock.h:44:11 (bitcoin-node+0xe15c19) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #4 mp::Lock::Lock(mp::Mutex&) /home/admin/actions-runner/_work/_temp/src/ipc/libmultiprocess/include/mp/util.h:173:45 (bitcoin-node+0xe15c19)
    #5 mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0::operator()() const /home/admin/actions-runner/_work/_temp/src/ipc/libmultiprocess/src/mp/proxy.cpp:419:14 (bitcoin-node+0xe15c19)
    #6 std::__1::__invoke_result_impl<void, mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>::type std::__1::__invoke[abi:dee220100]<mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>(mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0&&) /cxx_build/include/c++/v1/__type_traits/invoke.h:90:27 (bitcoin-node+0xe15c19)
    #7 void std::__1::__thread_execute[abi:dee220100]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0, 0ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>&, std::__1::__integer_sequence<unsigned long, 0ul>) /cxx_build/include/c++/v1/__thread/thread.h:161:3 (bitcoin-node+0xe15c19)
    #8 void* std::__1::__thread_proxy[abi:dee220100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>>(void*) /cxx_build/include/c++/v1/__thread/thread.h:169:3 (bitcoin-node+0xe15c19)

  Thread T2 'b-capnp-loop' (tid=21201, running) created by main thread at:
    #0 pthread_create <null> (bitcoin-node+0x16a32e) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #1 std::__1::__libcpp_thread_create[abi:dee220100](unsigned long*, void* (*)(void*), void*) /cxx_build/include/c++/v1/__thread/support/pthread.h:182:10 (bitcoin-node+0x819145) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #2 std::__1::thread::thread[abi:dee220100]<ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'(), 0>(ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'()&&) /cxx_build/include/c++/v1/__thread/thread.h:218:16 (bitcoin-node+0x819145)
    #3 ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*) /home/admin/actions-runner/_work/_temp/src/ipc/capnp/protocol.cpp:127:25 (bitcoin-node+0x819145)
    #4 ipc::capnp::(anonymous namespace)::CapnpProtocol::listen(int, char const*, interfaces::Init&) /home/admin/actions-runner/_work/_temp/src/ipc/capnp/protocol.cpp:87:9 (bitcoin-node+0x8184dd) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #5 ipc::(anonymous namespace)::IpcImpl::listenAddress(std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>&) /home/admin/actions-runner/_work/_temp/src/ipc/interfaces.cpp:111:21 (bitcoin-node+0x8148a0) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #6 AppInitMain(node::NodeContext&, interfaces::BlockAndHeaderTipInfo*) /home/admin/actions-runner/_work/_temp/src/init.cpp:1505:22 (bitcoin-node+0x2102a7) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #7 AppInit(node::NodeContext&) /home/admin/actions-runner/_work/_temp/src/bitcoind.cpp:242:43 (bitcoin-node+0x1f1e86) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #8 main /home/admin/actions-runner/_work/_temp/src/bitcoind.cpp:283:10 (bitcoin-node+0x1f1e86)

  Thread T11 (tid=21259, running) created by thread T2 at:
    #0 pthread_create <null> (bitcoin-node+0x16a32e) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #1 std::__1::__libcpp_thread_create[abi:dee220100](unsigned long*, void* (*)(void*), void*) /cxx_build/include/c++/v1/__thread/support/pthread.h:182:10 (bitcoin-node+0xe14411) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #2 std::__1::thread::thread[abi:dee220100]<mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0, 0>(mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0&&) /cxx_build/include/c++/v1/__thread/thread.h:218:16 (bitcoin-node+0xe14411)
    #3 mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>) /home/admin/actions-runner/_work/_temp/src/ipc/libmultiprocess/src/mp/proxy.cpp:416:17 (bitcoin-node+0xe14411)
    #4 mp::ThreadMap::Server::dispatchCallInternal(unsigned short, capnp::CallContext<capnp::AnyPointer, capnp::AnyPointer>) /home/admin/actions-runner/_work/_temp/build/src/ipc/libmultiprocess/include/mp/proxy.capnp.c++:602:9 (bitcoin-node+0xe0ea48) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #5 mp::ThreadMap::Server::dispatchCall(unsigned long, unsigned short, capnp::CallContext<capnp::AnyPointer, capnp::AnyPointer>) /home/admin/actions-runner/_work/_temp/build/src/ipc/libmultiprocess/include/mp/proxy.capnp.c++:591:14 (bitcoin-node+0xe0ea48)
    #6 virtual thunk to mp::ThreadMap::Server::dispatchCall(unsigned long, unsigned short, capnp::CallContext<capnp::AnyPointer, capnp::AnyPointer>) /home/admin/actions-runner/_work/_temp/build/src/ipc/libmultiprocess/include/mp/proxy.capnp.c++ (bitcoin-node+0xe0ea48)
    #7 capnp::LocalClient::callInternal(unsigned long, unsigned short, capnp::CallContextHook&) <null> (bitcoin-node+0xe6abed) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #8 capnp::LocalClient::call(unsigned long, unsigned short, kj::Own<capnp::CallContextHook, std::nullptr_t>&&, capnp::Capability::Client::CallHints)::'lambda'()::operator()() const <null> (bitcoin-node+0xe6a63d) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #9 kj::Promise<void> kj::_::MaybeVoidCaller<kj::_::Void, kj::Promise<void>>::apply<capnp::LocalClient::call(unsigned long, unsigned short, kj::Own<capnp::CallContextHook, std::nullptr_t>&&, capnp::Capability::Client::CallHints)::'lambda'()>(capnp::LocalClient::call(unsigned long, unsigned short, kj::Own<capnp::CallContextHook, std::nullptr_t>&&, capnp::Capability::Client::CallHints)::'lambda'()&, kj::_::Void&&) <null> (bitcoin-node+0xe72851) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #10 kj::_::TransformPromiseNode<kj::Promise<void>, kj::_::Void, capnp::LocalClient::call(unsigned long, unsigned short, kj::Own<capnp::CallContextHook, std::nullptr_t>&&, capnp::Capability::Client::CallHints)::'lambda'(), kj::_::PropagateException>::getImpl(kj::_::ExceptionOrValue&) <null> (bitcoin-node+0xe72451) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #11 kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&)::$_0::operator()() const async.c++ (bitcoin-node+0x10716f3) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #12 kj::Maybe<kj::Exception> kj::runCatchingExceptions<kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&)::$_0>(kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&)::$_0&&) async.c++ (bitcoin-node+0x1061bb7) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #13 kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&) <null> (bitcoin-node+0x1061aab) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #14 kj::_::ChainPromiseNode::fire() <null> (bitcoin-node+0x1063ac4) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #15 non-virtual thunk to kj::_::ChainPromiseNode::fire() <null> (bitcoin-node+0x10644d9) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #16 kj::EventLoop::turn() <null> (bitcoin-node+0x105c635) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #17 kj::_::waitImpl(kj::Own<kj::_::PromiseNode, kj::_::PromiseDisposer>&&, kj::_::ExceptionOrValue&, kj::WaitScope&, kj::SourceLocation)::$_2::operator()() const async.c++ (bitcoin-node+0x1070484) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #18 void kj::WaitScope::runOnStackPool<kj::_::waitImpl(kj::Own<kj::_::PromiseNode, kj::_::PromiseDisposer>&&, kj::_::ExceptionOrValue&, kj::WaitScope&, kj::SourceLocation)::$_2>(kj::_::waitImpl(kj::Own<kj::_::PromiseNode, kj::_::PromiseDisposer>&&, kj::_::ExceptionOrValue&, kj::WaitScope&, kj::SourceLocation)::$_2&&) async.c++ (bitcoin-node+0x105e5a7) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #19 kj::_::waitImpl(kj::Own<kj::_::PromiseNode, kj::_::PromiseDisposer>&&, kj::_::ExceptionOrValue&, kj::WaitScope&, kj::SourceLocation) <null> (bitcoin-node+0x105e121) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #20 kj::Promise<unsigned long>::wait(kj::WaitScope&, kj::SourceLocation) /home/admin/actions-runner/_work/_temp/depends/x86_64-pc-linux-gnu/include/kj/async-inl.h:1359:3 (bitcoin-node+0xe17187) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #21 mp::EventLoop::loop() /home/admin/actions-runner/_work/_temp/src/ipc/libmultiprocess/src/mp/proxy.cpp:244:68 (bitcoin-node+0xe11d71) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #22 ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'()::operator()() const /home/admin/actions-runner/_work/_temp/src/ipc/capnp/protocol.cpp:136:21 (bitcoin-node+0x819606) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604)
    #23 std::__1::__invoke_result_impl<void, ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'()>::type std::__1::__invoke[abi:dee220100]<ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'()>(ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'()&&) /cxx_build/include/c++/v1/__type_traits/invoke.h:90:27 (bitcoin-node+0x819606)
    #24 void std::__1::__thread_execute[abi:dee220100]<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'(), 0ul>(std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'()>&, std::__1::__integer_sequence<unsigned long, 0ul>) /cxx_build/include/c++/v1/__thread/thread.h:161:3 (bitcoin-node+0x819606)
    #25 void* std::__1::__thread_proxy[abi:dee220100]<std::__1::tuple<std::__1::unique_ptr<std::__1::__thread_struct, std::__1::default_delete<std::__1::__thread_struct>>, ipc::capnp::(anonymous namespace)::CapnpProtocol::startLoop(char const*)::'lambda'()>>(void*) /cxx_build/include/c++/v1/__thread/thread.h:169:3 (bitcoin-node+0x819606)

SUMMARY: ThreadSanitizer: data race (/home/admin/actions-runner/_work/_temp/build/bin/bitcoin-node+0x1f093c) (BuildId: 777a2f17a2bf7616985ece2bd101be9be19e9604) in operator delete(void*, unsigned long)

https://github.com/bitcoin/bitcoin/actions/runs/23000375935/job/66791524507?pr=34804#step:11:3525

==27808==ERROR: AddressSanitizer: heap-use-after-free on address 0x760561bf2468 at pc 0x646e8a5014c9 bp 0x75d55a31c930 sp 0x75d55a31c928
READ of size 8 at 0x760561bf2468 thread T11
    #0 0x646e8a5014c8 in mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0::operator()() const /home/admin/actions-runner/_work/_temp/src/ipc/libmultiprocess/src/mp/proxy.cpp:421:19
    #1 0x646e8a5014c8 in void std::__invoke_impl<void, mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>(std::__invoke_other, mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0&&) /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:61:14
    #2 0x646e8a5014c8 in std::__invoke_result<mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>::type std::__invoke<mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>(mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0&&) /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/invoke.h:96:14
    #3 0x646e8a5014c8 in void std::thread::_Invoker<std::tuple<mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>>::_M_invoke<0ul>(std::_Index_tuple<0ul>) /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_thread.h:292:13
    #4 0x646e8a5014c8 in std::thread::_Invoker<std::tuple<mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>>::operator()() /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_thread.h:299:11
    #5 0x646e8a5014c8 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<mp::ProxyServer<mp::ThreadMap>::makeThread(capnp::CallContext<mp::ThreadMap::MakeThreadParams, mp::ThreadMap::MakeThreadResults>)::$_0>>>::_M_run() /usr/lib/gcc/x86_64-linux-gnu/13/../../../../include/c++/13/bits/std_thread.h:244:13
    #6 0x79d563033db3  (/lib/x86_64-linux-gnu/libstdc++.so.6+0xecdb3) (BuildId: 753c6c8608b61d4e67be8f0c890e03e0aa046b8b)
    #7 0x646e8902e85a in asan_thread_start(void*) crtstuff.c
    #8 0x79d562ca5aa3  (/lib/x86_64-linux-gnu/libc.so.6+0x9caa3) (BuildId: 8e9fd827446c24067541ac5390e6f527fb5947bb)
    #9 0x79d562d32c6b  (/lib/x86_64-linux-gnu/libc.so.6+0x129c6b) (BuildId: 8e9fd827446c24067541ac5390e6f527fb5947bb)

0x760561bf2468 is located 8 bytes inside of 32-byte region [0x760561bf2460,0x760561bf2480)
freed by thread T2 (b-capnp-loop) here:
    #0 0x646e89075f66 in operator delete(void*, unsigned long) (/home/admin/actions-runner/_work/_temp/build/bin/bitcoin-node+0x1408f66) (BuildId: 89995cb68dd51cd207c97d4b8e372df7e8c677d2)
    #1 0x79d5635559d9  (/lib/x86_64-linux-gnu/libcapnp-rpc-1.0.1.so+0x489d9) (BuildId: 44717f0ad52d0e372420b8de2d7701bfba298f14)
    #2 0x79d563586a13  (/lib/x86_64-linux-gnu/libcapnp-rpc-1.0.1.so+0x79a13) (BuildId: 44717f0ad52d0e372420b8de2d7701bfba298f14)
    #3 0x79d5635872b6  (/lib/x86_64-linux-gnu/libcapnp-rpc-1.0.1.so+0x7a2b6) (BuildId: 44717f0ad52d0e372420b8de2d7701bfba298f14)
    #4 0x79d56358a505  (/lib/x86_64-linux-gnu/libcapnp-rpc-1.0.1.so+0x7d505) (BuildId: 44717f0ad52d0e372420b8de2d7701bfba298f14)
    #5 0x79d56359d42b  (/lib/x86_64-linux-gnu/libcapnp-rpc-1.0.1.so+0x9042b) (BuildId: 44717f0ad52d0e372420b8de2d7701bfba298f14)
    #6 0x79d5633e961c in kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&) (/lib/x86_64-linux-gnu/libkj-async-1.0.1.so+0x3e61c) (BuildId: 356b68ed6bfbcc99793859e96d335850186ca5da)
    #7 0x79d5633e9694 in kj::_::TransformPromiseNodeBase::getDepResult(kj::_::ExceptionOrValue&) (/lib/x86_64-linux-gnu/libkj-async-1.0.1.so+0x3e694) (BuildId: 356b68ed6bfbcc99793859e96d335850186ca5da)
    #8 0x79d56359ae20  (/lib/x86_64-linux-gnu/libcapnp-rpc-1.0.1.so+0x8de20) (BuildId: 44717f0ad52d0e372420b8de2d7701bfba298f14)
    #9 0x79d5633e961c in kj::_::TransformPromiseNodeBase::get(kj::_::ExceptionOrValue&) (/lib/x86_64-linux-gnu/libkj-async-1.0.1.so+0x3e61c) (BuildId: 356b68ed6bfbcc99793859e96d335850186ca5da)
    #10 0x79d5633e52e6  (/lib/x86_64-linux-gnu/libkj-async-1.0.1.so+0x3a2e6) (BuildId: 356b68ed6bfbcc99793859e96d335850186ca5da)
  Stack after return:      f5
  Stack use after scope:   f8
  Global redzone:          f9
  Global init order:       f6
  Poisoned by user:        f7
  Container overflow:      fc
  Array cookie:            ac
  Intra object redzone:    bb
  ASan internal:           fe
  Left alloca redzone:     ca
  Right alloca redzone:    cb
==27808==ABORTING

ryanofsky · 2026-03-12T16:40:04Z

Thanks fanquake looks like both of these errors are caused by the new hooks added for testng, so not very surprising. The tests were automatically generated and pretty close to being right but have a number of issues and I'm rewriting them.

ryanofsky · 2026-03-12T18:11:57Z

Rebased 6fd7194 -> 6350dc9 (pr/subtree-9.2 -> pr/subtree-9.3, compare) based on bitcoin-core/libmultiprocess#249 pr/disraces.3

Sjors · 2026-03-13T14:06:21Z

I think you need a functional test adjustment. Based on local TSan testing this seems to do the trick:

diff --git a/test/functional/interface_ipc.py b/test/functional/interface_ipc.py
index 16ec03079b..b10aeb8667 100755
--- a/test/functional/interface_ipc.py
+++ b/test/functional/interface_ipc.py
@@ -110,17 +110,17 @@ class IPCInterfaceTest(BitcoinTestFramework):
             with node.assert_debug_log(expected_msgs=["BlockTemplate.waitNext", "IPC server post request"], timeout=2):
                 promise = template.waitNext(ctx, waitoptions)
                 await asyncio.sleep(0.1)
-            disconnected_log_check.enter_context(node.assert_debug_log(expected_msgs=["IPC server: socket disconnected", "canceled while executing"], timeout=2))
+            disconnected_log_check.enter_context(node.assert_debug_log(expected_msgs=["IPC server: socket disconnected", "canceled "], timeout=2))
             del promise

         asyncio.run(capnp.run(async_routine()))

-        # Wait for socket disconnected log message, then generate a block to
-        # cause the waitNext() call to return a new template. Look for a
-        # canceled IPC log message after waitNext returns.
-        with node.assert_debug_log(expected_msgs=["interrupted (canceled)"], timeout=2):
-            disconnected_log_check.close()
-            self.generate(node, 1)
+        disconnected_log_check.close()
+        self.generate(node, 1)

     def run_thread_busy_test(self):
         """Test behavior when sending multiple calls to the same server thread

ryanofsky added 2 commits March 11, 2026 13:39

Merge commit '4522fc61cbc78c9685c0b0eb37cc343db0c5af56' into pr/subtr…

6350dc9

…ee-9

Sjors mentioned this pull request Mar 11, 2026

mining: add getTransactions(ByWitnessID) IPC methods #34020

Draft

ryanofsky mentioned this pull request Mar 11, 2026

Fix race conditions on disconnects bitcoin-core/libmultiprocess#250

Closed

ryanofsky force-pushed the pr/subtree-9 branch from af35175 to 6fd7194 Compare March 12, 2026 11:45

DrahtBot added the CI failed label Mar 12, 2026

fanquake mentioned this pull request Mar 12, 2026

ci: run some Bitcoin Core CI jobs bitcoin-core/libmultiprocess#253

Open

ryanofsky force-pushed the pr/subtree-9 branch from 6fd7194 to 6350dc9 Compare March 12, 2026 18:08

DrahtBot removed the CI failed label Mar 12, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Update libmultiprocess subtree to fix race conditions on disconnects#34804

Update libmultiprocess subtree to fix race conditions on disconnects#34804
ryanofsky wants to merge 2 commits intobitcoin:masterfrom
ryanofsky:pr/subtree-9

ryanofsky commented Mar 11, 2026 •

edited

Loading

Uh oh!

DrahtBot commented Mar 11, 2026 •

edited

Loading

Uh oh!

ryanofsky commented Mar 12, 2026

Uh oh!

dergoegge commented Mar 12, 2026

Uh oh!

fanquake commented Mar 12, 2026

Uh oh!

ryanofsky commented Mar 12, 2026

Uh oh!

ryanofsky commented Mar 12, 2026

Uh oh!

Sjors commented Mar 13, 2026 •

edited

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

ryanofsky commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

DrahtBot commented Mar 11, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Reviews

LLM Linter (✨ experimental)

Uh oh!

ryanofsky commented Mar 12, 2026

Uh oh!

dergoegge commented Mar 12, 2026

Uh oh!

fanquake commented Mar 12, 2026

Uh oh!

ryanofsky commented Mar 12, 2026

Uh oh!

ryanofsky commented Mar 12, 2026

Uh oh!

Sjors commented Mar 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ryanofsky commented Mar 11, 2026 •

edited

Loading

DrahtBot commented Mar 11, 2026 •

edited

Loading

Sjors commented Mar 13, 2026 •

edited

Loading