Project

General

Profile

Actions

Bug #73561

open

[mgr/crash] Crash module failed with error "Module 'crash' has failed: dictionary changed size during iteration"

Added by Prashant D 5 months ago. Updated 5 months ago.

Status:
Fix Under Review
Priority:
Normal
Assignee:
Category:
-
Target version:
-
% Done:

0%

Source:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Tags (freeform):
Merge Commit:
Fixed In:
Released In:
Upkeep Timestamp:

Description

In the crash module, a RuntimeError is encountered during the execution of the serve() method. This error occurs due to the _prune() method attempts to iterate over the self.crashes dictionary using a filter() operation via timestamp_filter(), while simultaneously modifying the dictionary by deleting entries. In Python, modifying a dictionary during iteration is unsafe and leads to a RuntimeError when its size changes.

MGR logs:
2025-10-12T05:34:13.061876914Z debug 2025-10-12T05:34:13.061+0000 7f06999c1640 -1 mgr.server reply reply (5) Input/output error Module 'crash' has experienced an error and cannot handle commands: dictionary changed size during iteration
2025-10-12T05:34:13.361732483Z debug 2025-10-12T05:34:13.361+0000 7f06a09cf640  0 log_channel(audit) log [DBG] : from='client.43773231 -' entity='client.crash' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch
2025-10-12T05:34:13.361732483Z debug 2025-10-12T05:34:13.361+0000 7f06999c1640 -1 mgr.server reply reply (5) Input/output error Module 'crash' has experienced an error and cannot handle commands: dictionary changed size during iteration
2025-10-12T05:34:13.583485958Z debug 2025-10-12T05:34:13.582+0000 7f06a09cf640  0 log_channel(audit) log [DBG] : from='client.43305448 -' entity='client.crash' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch
2025-10-12T05:34:13.583545572Z debug 2025-10-12T05:34:13.583+0000 7f06999c1640 -1 mgr.server reply reply (5) Input/output error Module 'crash' has experienced an error and cannot handle commands: dictionary changed size during iteration
2025-10-12T05:34:13.875979296Z debug 2025-10-12T05:34:13.875+0000 7f06a09cf640  0 log_channel(audit) log [DBG] : from='client.43773255 -' entity='client.crash' cmd=[{"prefix": "crash post", "target": ["mon-mgr", ""]}]: dispatch
2025-10-12T05:34:13.875979296Z debug 2025-10-12T05:34:13.875+0000 7f06999c1640 -1 mgr.server reply reply (5) Input/output error Module 'crash' has experienced an error and cannot handle commands: dictionary changed size during iteration

Actions #1

Updated by Prashant D 5 months ago

  • Status changed from New to Fix Under Review
  • Pull request ID set to 65975
Actions

Also available in: Atom PDF