Project

General

Profile

Actions
Copy link

Bug #72012

closed

Test failure: test_standby (tasks.mgr.test_prometheus.TestPrometheus)

Added by Shraddha Agrawal 9 months ago. Updated 7 months ago.

Status:
Resolved
Priority:
Normal
Assignee:
Nizamudeen A
Category:
-
Target version:
-
% Done:

0%

Source:
Backport:
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
64385
Tags (freeform):
Merge Commit:
e1c6a7d5243069e841a41eee5f23181dade125a7
Fixed In:
v20.3.0-1609-ge1c6a7d524
Released In:
Upkeep Timestamp:
2025-08-12T18:23:41+00:00

Description

/a/skanta-2025-07-04_23:32:34-rados-wip-bharath13-t[…]2025-07-04-0559-distro-default-smithi/8370609

This error was also observed in this nightly run: https://pulpito.ceph.com/teuthology-2025-07-06_20:00:21-rados-main-distro-default-smithi/, but not the one before it: https://pulpito.ceph.com/teuthology-2025-06-29_20:00:18-rados-main-distro-default-smithi/. Looking at the PRs that were merged between these two runs, looks like https://github.com/ceph/ceph/pull/61468 might be the origin of the error.

Related issues 1 (0 open — 1 closed)

Has duplicate mgr - Bug #72011: mgr/prometheus module at ceph_cluster is unreachableDuplicateNizamudeen A

Actions
Actions
Copy link
 #1

Updated by Shraddha Agrawal 9 months ago

  • Related to Bug #72011: mgr/prometheus module at ceph_cluster is unreachable added
Actions
Copy link
 #2

Updated by Lee Sanders 9 months ago

https://pulpito.ceph.com/skanta-2025-07-05_06:21:05-rados-wip-bharath15-testing-2025-07-04-1752-distro-default-smithi/8370753

/a/skanta-2025-07-05_06:21:05-rados-wip-bharath15-testing-2025-07-04-1752-distro-default-smithi/8370753

Actions
Copy link
 #3

Updated by Lee Sanders 9 months ago · Edited 

2025-07-05T08:01:00.194 INFO:teuthology.orchestra.run.smithi121.stderr:2025-07-05T08:01:00.191+0000 7f491c272640  1 -- 172.21.15.121:0/106362367 >> 172.21.15.121:0/106362367 conn(0x7f49140a1830 msgr2=0x7f4914055ce0 unknown :-1 s=STATE_NONE l=0).mark_down
2025-07-05T08:01:00.195 INFO:teuthology.orchestra.run.smithi121.stderr:2025-07-05T08:01:00.191+0000 7f491c272640  1 -- 172.21.15.121:0/106362367 shutdown_connections
2025-07-05T08:01:00.195 INFO:teuthology.orchestra.run.smithi121.stderr:2025-07-05T08:01:00.191+0000 7f491c272640  1 -- 172.21.15.121:0/106362367 wait complete.
2025-07-05T08:01:00.206 INFO:tasks.cephfs_test_runner:test_standby (tasks.mgr.test_prometheus.TestPrometheus) ... ERROR
2025-07-05T08:01:00.207 INFO:tasks.cephfs_test_runner:
2025-07-05T08:01:00.207 INFO:tasks.cephfs_test_runner:======================================================================
2025-07-05T08:01:00.207 INFO:tasks.cephfs_test_runner:ERROR: test_standby (tasks.mgr.test_prometheus.TestPrometheus)
2025-07-05T08:01:00.207 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2025-07-05T08:01:00.207 INFO:tasks.cephfs_test_runner:Traceback (most recent call last):
2025-07-05T08:01:00.208 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_f2fcb648c0c5e12591518d0a64d8148cb44113b8/qa/tasks/mgr/test_prometheus.py", line 34, in test_standby
2025-07-05T08:01:00.208 INFO:tasks.cephfs_test_runner:    original_uri = self._get_uri("prometheus")
2025-07-05T08:01:00.208 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_f2fcb648c0c5e12591518d0a64d8148cb44113b8/qa/tasks/mgr/mgr_test_case.py", line 211, in _get_uri
2025-07-05T08:01:00.208 INFO:tasks.cephfs_test_runner:    cls.wait_until_true(lambda: _get_or_none() is not None, 30)
2025-07-05T08:01:00.208 INFO:tasks.cephfs_test_runner:  File "/home/teuthworker/src/github.com_ceph_ceph-c_f2fcb648c0c5e12591518d0a64d8148cb44113b8/qa/tasks/ceph_test_case.py", line 339, in wait_until_true
2025-07-05T08:01:00.208 INFO:tasks.cephfs_test_runner:    raise TestTimeoutError("Timed out after {0}s and {1} retries".format(elapsed, retry_count))
2025-07-05T08:01:00.208 INFO:tasks.cephfs_test_runner:tasks.ceph_test_case.TestTimeoutError: Timed out after 30s and 0 retries
2025-07-05T08:01:00.208 INFO:tasks.cephfs_test_runner:
2025-07-05T08:01:00.208 INFO:tasks.cephfs_test_runner:----------------------------------------------------------------------
2025-07-05T08:01:00.208 INFO:tasks.cephfs_test_runner:Ran 2 tests in 158.463s
2025-07-05T08:01:00.208 INFO:tasks.cephfs_test_runner:
2025-07-05T08:01:00.208 INFO:tasks.cephfs_test_runner:FAILED (errors=1)
2025-07-05T08:01:00.209 INFO:tasks.cephfs_test_runner:
Actions
Copy link
 #4

Updated by Shraddha Agrawal 9 months ago

/a/skanta-2025-07-04_23:32:34-rados-wip-bharath13-testing-2025-07-04-0559-distro-default-smithi/8370609

Actions
Copy link
 #5

Updated by Nizamudeen A 9 months ago

  • Status changed from New to Fix Under Review
  • Assignee set to Nizamudeen A
  • Pull request ID set to 64385
Actions
Copy link
 #6

Updated by Laura Flores 9 months ago

  • Related to deleted (Bug #72011: mgr/prometheus module at ceph_cluster is unreachable)
Actions
Copy link
 #7

Updated by Laura Flores 9 months ago

  • Has duplicate Bug #72011: mgr/prometheus module at ceph_cluster is unreachable added
Actions
Copy link
 #8

Updated by Upkeep Bot 8 months ago

  • Merge Commit set to e1c6a7d5243069e841a41eee5f23181dade125a7
  • Fixed In set to v20.3.0-1609-ge1c6a7d524
  • Upkeep Timestamp set to 2025-07-15T13:36:56+00:00
Actions
Copy link
 #9

Updated by Naveen Naidu 8 months ago

/a/skanta-2025-07-11_12:12:50-rados-wip-bharath2-testing-2025-07-11-0437-distro-default-smithi/8381412

Actions
Copy link
 #10

Updated by Shraddha Agrawal 8 months ago

/a/skanta-2025-07-13_23:08:24-rados-wip-bharath4-testing-2025-07-13-0539-distro-default-smithi/8384542

Actions
Copy link
 #11

Updated by Upkeep Bot 7 months ago

  • Status changed from Fix Under Review to Resolved
  • Upkeep Timestamp changed from 2025-07-15T13:36:56+00:00 to 2025-08-12T18:23:41+00:00
Actions
Copy link

Also available in: Atom PDF