Bug #74565
openRocky10 - reimage timeout
0%
Description
/a/nmordech-2026-01-25_11:10:14-rados-wip-rocky10-branch-of-the-day-2026-01-23-1769128778-distro-default-trial/17037
2026-01-25T11:58:28.576 ERROR:teuthology.dispatcher.supervisor:Reimaging error. Unlocking machines...
Traceback (most recent call last):
File "/home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/teuthology/dispatcher/supervisor.py", line 233, in reimage
reimaged = lock_ops.reimage_machines(ctx, targets, job_config['machine_type'])
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/teuthology/lock/ops.py", line 366, in reimage_machines
with teuthology.parallel.parallel() as p:
File "/home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/teuthology/parallel.py", line 84, in __exit__
for result in self:
File "/home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/teuthology/parallel.py", line 98, in __next__
resurrect_traceback(result)
File "/home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/teuthology/parallel.py", line 30, in resurrect_traceback
raise exc.exc_info[1]
File "/home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/teuthology/parallel.py", line 23, in capture_traceback
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/teuthology/provision/__init__.py", line 49, in reimage
result = obj.create()
^^^^^^^^^^^^
File "/home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/teuthology/provision/fog.py", line 91, in create
self._wait_for_ready()
File "/home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/teuthology/provision/fog.py", line 286, in _wait_for_ready
while proceed():
^^^^^^^^^
File "/home/teuthworker/src/git.ceph.com_teuthology_c433f1062990a0488dc29a553589bc609a460691/teuthology/contextutil.py", line 134, in __call__
raise MaxWhileTries(error_msg)
teuthology.exceptions.MaxWhileTries: reached maximum tries (100) after waiting for 600 seconds
Updated by David Galloway about 2 months ago
- Status changed from New to In Progress
- Assignee set to David Galloway
Comparing a successful reimage: https://qa-proxy.ceph.com/teuthology/nmordech-2026-01-25_11:10:14-rados-wip-rocky10-branch-of-the-day-2026-01-23-1769128778-distro-default-trial/17037/console_logs/trial132_reimage.log
With the failed reimage: https://qa-proxy.ceph.com/teuthology/nmordech-2026-01-25_11:10:14-rados-wip-rocky10-branch-of-the-day-2026-01-23-1769128778-distro-default-trial/17037/console_logs/trial158_reimage.log
It looks like trial158 just didn't reboot... I'm not sure there's anything we can do but maybe update firmware.
Updated by Nitzan Mordechai about 2 months ago
https://pulpito.ceph.com/nmordech-2026-01-28_16:11:20-rados-wip-rocky10-branch-of-the-day-2026-01-23-1769128778-distro-default-trial/23238/
https://pulpito.ceph.com/nmordech-2026-01-28_16:11:20-rados-wip-rocky10-branch-of-the-day-2026-01-23-1769128778-distro-default-trial/23523/
https://pulpito.ceph.com/nmordech-2026-01-28_16:11:20-rados-wip-rocky10-branch-of-the-day-2026-01-23-1769128778-distro-default-trial/23359/
https://pulpito.ceph.com/nmordech-2026-01-28_16:11:20-rados-wip-rocky10-branch-of-the-day-2026-01-23-1769128778-distro-default-trial/23415/
https://pulpito.ceph.com/nmordech-2026-01-28_16:11:20-rados-wip-rocky10-branch-of-the-day-2026-01-23-1769128778-distro-default-trial/23523/