Ceph » mgr

/a/skanta-2025-07-04_23:32:34-rados-wip-bharath13-t[…]2025-07-04-0559-distro-default-smithi/8370622

Error in teuthology.log:

2025-07-05T03:25:49.688 INFO:teuthology.orchestra.run.smithi062.stderr:+ curl -s http://172.21.15.142:9095/api/v1/alerts
2025-07-05T03:25:49.692 INFO:teuthology.orchestra.run.smithi062.stderr:+ curl -s http://172.21.15.142:9095/api/v1/alerts
2025-07-05T03:25:49.692 INFO:teuthology.orchestra.run.smithi062.stderr:+ jq -e '.data | .alerts | .[] | select(.labels | .alertname == "CephMonDown") | .state == "firing"'
2025-07-05T03:25:50.273 DEBUG:teuthology.orchestra.run:got remote process result: 4
2025-07-05T03:25:50.274 INFO:teuthology.orchestra.run.smithi062.stdout:{"status":"success","data":{"alerts":[{"labels":{"alertname":"CephMgrPrometheusModuleInactive","cluster":"fa9941a0-594d-11f0-8720-adfe0268badd","instance":"ceph_cluster","job":"ceph","oid":"1.3.6.1.4.1.50495.1.2.1.6.2","severity":"critical","type":"ceph_default"},"annotations":{"description":"The mgr/prometheus module at ceph_cluster is unreachable. This could mean that the module has been disabled or the mgr daemon itself is down. Without the mgr/prometheus module metrics and alerts will no longer function. Open a shell to an admin node or toolbox pod and use 'ceph -s' to to determine whether the mgr is active. If the mgr is not active, restart it, otherwise you can determine module status with 'ceph mgr module ls'. If it is not listed as enabled, enable it with 'ceph mgr module enable prometheus'.","summary":"The mgr/prometheus module is not available"},"state":"firing","activeAt":"2025-07-05T03:22:03.245200013Z","value":"0e+00"}]}}
2025-07-05T03:25:50.275 ERROR:teuthology.run_tasks:Saw exception from tasks.

This error was also observed in this nightly run: https://pulpito.ceph.com/teuthology-2025-07-06_20:00:21-rados-main-distro-default-smithi/, but not the one before it: https://pulpito.ceph.com/teuthology-2025-06-29_20:00:18-rados-main-distro-default-smithi/. Looking at the PRs that were merged between these two runs, looks like https://github.com/ceph/ceph/pull/61468 might be the origin of the error.