Project

General

Profile

Actions

Bug #70571

closed

Ceph mgr daemon "status" command does not exist on the admin socket in Ceph main

Added by Travis Nielsen about 1 year ago. Updated 5 months ago.

Status:
Resolved
Priority:
Normal
Category:
ceph cli
Target version:
% Done:

0%

Source:
Community (dev)
Backport:
squid,reef
Regression:
Yes
Severity:
1 - critical
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Crash signature (v1):
Crash signature (v2):
Tags (freeform):
backport_processed
Fixed In:
v20.0.0-761-g47cde58120
Released In:
v20.2.0~756
Upkeep Timestamp:
2025-11-01T01:32:09+00:00

Description

Starting on approximately March 3rd, the Rook daily CI running against Ceph main started failing.

The mgr daemon gets an error when Rook attempts to get the status from the mgr's admin socket:

  1. ceph --admin-daemon /run/ceph/ceph-mgr.a.asok status
    no valid command found; 10 closest matches:
    0
    1
    2
    abort
    assert
    config diff
    config diff get <var>
    config get <var>
    config help [<var>]
    config set <var> <val>...
    admin_socket: invalid command
    [root@rook-ceph-mgr-a-f4d7bfb5b-sjz8w ceph]# echo $?
    22

This status command had been working for many years, and just now stopped working in Ceph main.
The same status command works fine in the mon daemon pod from ceph main.
This failure has not yet been seen in other branches.

This is a blocker for Rook running against main.
This does not block development efforts or any releases,
but it is a clear blocker for the release of Tentacle,
and also could mask if there are any other issues blocking Rook in Ceph main.

See Rook issue https://github.com/rook/rook/issues/15509


Related issues 2 (0 open2 closed)

Copied to Ceph - Backport #70666: reef: Ceph mgr daemon "status" command does not exist on the admin socket in Ceph mainResolvedPatrick DonnellyActions
Copied to Ceph - Backport #70667: squid: Ceph mgr daemon "status" command does not exist on the admin socket in Ceph mainResolvedPatrick DonnellyActions
Actions #1

Updated by Travis Nielsen about 1 year ago

After a bit of digging through the ceph PRs merged recently, the merge of the [case-insensitive directory trees feature](https://tracker.ceph.com/issues/66373) is the only one that seems possible to affect the core.
- Merged to main on March 3rd: https://github.com/ceph/ceph/pull/60746
- Corresponds to the fails in main Rook started seeing on March 4th
- Merged to squid two days ago: https://github.com/ceph/ceph/pull/62095
- Corresponds to the fails in squid Rook started seeing yesterday

I'm not clear why this might be causing the mgr probe failure, but all other changes merged on the corresponding days look like they were only for rgw or dashboard.

Actions #3

Updated by Patrick Donnelly 12 months ago

  • Status changed from New to In Progress
  • Assignee set to Patrick Donnelly
  • Backport set to squid,reef
Actions #4

Updated by Patrick Donnelly 12 months ago

  • Status changed from In Progress to Fix Under Review
  • Pull request ID set to 62465
Actions #5

Updated by Patrick Donnelly 12 months ago

  • Status changed from Fix Under Review to Pending Backport
Actions #6

Updated by Upkeep Bot 12 months ago

  • Copied to Backport #70666: reef: Ceph mgr daemon "status" command does not exist on the admin socket in Ceph main added
Actions #7

Updated by Upkeep Bot 12 months ago

  • Copied to Backport #70667: squid: Ceph mgr daemon "status" command does not exist on the admin socket in Ceph main added
Actions #8

Updated by Upkeep Bot 12 months ago

  • Tags (freeform) set to backport_processed
Actions #9

Updated by Upkeep Bot 9 months ago

  • Merge Commit set to 47cde581208828d3d055accd6f3923e67256ed36
  • Fixed In set to v20.0.0-761-g47cde581208
  • Upkeep Timestamp set to 2025-07-08T18:06:53+00:00
Actions #10

Updated by Upkeep Bot 8 months ago

  • Fixed In changed from v20.0.0-761-g47cde581208 to v20.0.0-761-g47cde5812088
  • Upkeep Timestamp changed from 2025-07-08T18:06:53+00:00 to 2025-07-14T15:21:20+00:00
Actions #11

Updated by Upkeep Bot 8 months ago

  • Fixed In changed from v20.0.0-761-g47cde5812088 to v20.0.0-761-g47cde58120
  • Upkeep Timestamp changed from 2025-07-14T15:21:20+00:00 to 2025-07-14T20:45:53+00:00
Actions #12

Updated by Konstantin Shalygin 7 months ago

  • Status changed from Pending Backport to Resolved
Actions #13

Updated by Upkeep Bot 5 months ago

  • Released In set to v20.2.0~756
  • Upkeep Timestamp changed from 2025-07-14T20:45:53+00:00 to 2025-11-01T01:32:09+00:00
Actions

Also available in: Atom PDF