squid: mds/MDSDaemon: unlock mds_lock while shutting down Beacon and others#64886
Merged
neesingh-rh merged 1 commit intoceph:squidfrom Jan 19, 2026
Merged
squid: mds/MDSDaemon: unlock mds_lock while shutting down Beacon and others#64886neesingh-rh merged 1 commit intoceph:squidfrom
mds_lock while shutting down Beacon and others#64886neesingh-rh merged 1 commit intoceph:squidfrom
Conversation
This fixes a deadlock bug during MDS shutdown: - the "signal_handler" thread receives the shutdown signal and invokes MDSDaemon::suicide() while holding `mds_lock` - MDSDaemon::suicide() invokes Beacon::send_and_wait() while still holding `mds_lock` - meanwhile, all "ms_dispatch" threads get stuck waiting for `mds_lock`, for example in MDCache::upkeep_main() or MDSDaemon::ms_dispatch2() - Beacon::send_and_wait() waits for a `MSG_MDS_BEACON` packet to be dispatched (via `cvar` with a timeout) At this point, even if a `MSG_MDS_BEACON` packet is received by one of the worker threads, they will put it in the `DispatchQueue`, but no dispatcher thread will be able to handle it because they are all stuck. The cvar.wait_for() call in Beacon::send_and_wait() will therefore time out and the `MSG_MDS_BEACON` will never be processed. The proper solution is to unlock `mds_lock` to avoid the dispatchers from getting stuck. And in general, we should be holding a lock strictly only when it is needed and never do blocking calls while holding a lock. Fixes: https://tracker.ceph.com/issues/68760 Signed-off-by: Max Kellermann <max.kellermann@ionos.com> (cherry picked from commit c0fedb5)
Member
|
jenkins retest this please |
Member
|
This PR is under test in https://tracker.ceph.com/issues/73330. |
neesingh-rh
approved these changes
Jan 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
backport tracker: https://tracker.ceph.com/issues/72390
backport of #60326
parent tracker: https://tracker.ceph.com/issues/68760
this backport was staged using ceph-backport.sh version 16.0.0.6848
find the latest version at https://github.com/ceph/ceph/blob/main/src/script/ceph-backport.sh