fix(NotificationCenter): lock order inversion deadlock #5159#5160
fix(NotificationCenter): lock order inversion deadlock #5159#5160
Conversation
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 22 out of 22 changed files in this pull request and generated 9 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
matejk
left a comment
There was a problem hiding this comment.
Some comments in the code. In addition: Would IOLock also deserve its own unit tests?
Foundation/include/Poco/IOLock.h
Outdated
| // | ||
| // Definition of the IOLock class. | ||
| // | ||
| // Copyright (c) 2004-2008, Applied Informatics Software Engineering GmbH., |
There was a problem hiding this comment.
Wrong (C) year. Blast from the past! :)
| RWLock::ScopedWriteLock lock(_mutex); | ||
| _observers.emplace_back(observer.clone()); | ||
| _observers.back()->start(); | ||
| AbstractObserverPtr pObserver = observer.clone(); |
There was a problem hiding this comment.
Why does the observer have to be cloned before lock?
There was a problem hiding this comment.
because this is precisely where the deadlock potential resides:
pObserver->start();| namespace Poco { | ||
|
|
||
|
|
||
| class Foundation_API IOLock |
There was a problem hiding this comment.
Perhaps IOResourcePin or IOResourceHold to distinguish from mutex-like locking?
| void IOLock::wait() | ||
| { | ||
| while (_inProgress.load()) | ||
| std::this_thread::sleep_for(std::chrono::microseconds(10)); |
There was a problem hiding this comment.
Use std::condition_variable instead of busy loop?
|
|
||
| bool IOLock::enter() | ||
| { | ||
| _inProgress.store(true); |
There was a problem hiding this comment.
Would it make sense to add detection if the second thread called enter()? debug assertion or exception?
There was a problem hiding this comment.
IOLock explicitly does not support multi-threaded I/O. It can be done, but I don't think the complications it brings are worth it, the class does one thing only - syncing between destruction in one thread and I/O in another. Whoever is doing l/O from multiple threads, I wish him good luck.
EDIT: to be more specific, whoever is doing I/O on a single channel, or polls, from multiple threads
There was a problem hiding this comment.
What I meant was to detect multiple threads using IOLock and prevent that (or log or throw and exception).
There was a problem hiding this comment.
I guess one thing that could be done is to keep thread references and add some sanity checks, but I would do it for debug build only and I really don't have time to do it. As far as I'm concerned, we made a knife, if someone gets cut, thats' on them.
|
@aleks-f , ready to merge? |
Branch Summary: 5159-nc-deadlock
This branch fixes several thread-safety issues and deadlocks reported in issues #5159, #5161, and #5162.
Closes #5159, #5161, and #5162.
Issues Addressed
New Components
IOLock (
Foundation/include/Poco/IOLock.h,Foundation/src/IOLock.cpp)A new lock-free synchronization primitive for coordinating I/O operations with close/shutdown:
IOLock- tracks whether an I/O operation is in progress and whether close has been requestedScopedIOLock- RAII wrapper that enters on construction and leaves on destructionstd::atomic<bool>for_closedand_inProgressflagsmarkClosed(),wait(),tryWait(), andclose()methodsScopedSocketReactor (
Net/include/Poco/Net/SocketReactor.h)A new RAII wrapper utility class that starts a SocketReactor in a background thread and automatically stops/joins on destruction:
Bug Fixes
NotificationCenter (#5159)
Pipe (#5161)
IOLocksynchronization for thread-safe read/write operationswait()for I/O completion now happens before closing file descriptorsPollSet (#5161)
IOLockto coordinatepoll()with destructorpoll()is running during destructionTest Improvements
New Tests
PipeTest- comprehensive test suite includingtestCloseRacefor concurrent close/readtestConcurrentHandlerRemovalin SocketReactorTest - tests concurrent handler removalCI Enhancements
Files Changed
IOLock.h,IOLock.cpp(new)PipeImpl_POSIX.h/cpp,PipeImpl_WIN32.h/cpp,Pipe.cppPollSet.cppSocketReactor.h,SocketReactor.cppPipeTest.cpp/h,SocketReactorTest.cpp/h,NotificationCenterTest.cpp/h.github/workflows/ci.ymlKey Code Changes
Pipe Close Sequence (Before)
Pipe Close Sequence (After)
Destructor Close Order (Before)
Destructor Close Order (After)