Actions
Bug #73561
open[mgr/crash] Crash module failed with error "Module 'crash' has failed: dictionary changed size during iteration"
% Done:
0%
Source:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Description
In the crash module, a RuntimeError is encountered during the execution of the serve() method. This error occurs due to the _prune() method attempts to iterate over the self.crashes dictionary using a filter() operation via timestamp_filter(), while simultaneously modifying the dictionary by deleting entries. In Python, modifying a dictionary during iteration is unsafe and leads to a RuntimeError when its size changes.
MGR logs:
2025-10-12T05:34:13.061876914Z debug 2025-10-12T05:34:13.061+0000 7f06999c1640 -1 mgr.server reply reply (5) Input/output error Module 'crash' has experienced an error and cannot handle commands: dictionary changed size during iteration
2025-10-12T05:34:13.361732483Z debug 2025-10-12T05:34:13.361+0000 7f06a09cf640 0 log_channel(audit) log [DBG] : from='client.43773231 -' entity='client.crash' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch
2025-10-12T05:34:13.361732483Z debug 2025-10-12T05:34:13.361+0000 7f06999c1640 -1 mgr.server reply reply (5) Input/output error Module 'crash' has experienced an error and cannot handle commands: dictionary changed size during iteration
2025-10-12T05:34:13.583485958Z debug 2025-10-12T05:34:13.582+0000 7f06a09cf640 0 log_channel(audit) log [DBG] : from='client.43305448 -' entity='client.crash' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch
2025-10-12T05:34:13.583545572Z debug 2025-10-12T05:34:13.583+0000 7f06999c1640 -1 mgr.server reply reply (5) Input/output error Module 'crash' has experienced an error and cannot handle commands: dictionary changed size during iteration
2025-10-12T05:34:13.875979296Z debug 2025-10-12T05:34:13.875+0000 7f06a09cf640 0 log_channel(audit) log [DBG] : from='client.43773255 -' entity='client.crash' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch
2025-10-12T05:34:13.875979296Z debug 2025-10-12T05:34:13.875+0000 7f06999c1640 -1 mgr.server reply reply (5) Input/output error Module 'crash' has experienced an error and cannot handle commands: dictionary changed size during iteration
Updated by Prashant D 5 months ago
- Status changed from New to Fix Under Review
- Pull request ID set to 65975
Actions