Actions
Bug #52116
closedkubeadm task fails with error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
Status:
Resolved
Priority:
High
Assignee:
-
Category:
mgr/rook
Target version:
-
% Done:
0%
Source:
Backport:
pacific
Regression:
No
Severity:
3 - minor
Reviewed:
Affected Versions:
ceph-qa-suite:
Pull request ID:
Tags (freeform):
Merge Commit:
Fixed In:
v17.0.0-6791-g84479e03a7
Released In:
v17.2.0~1224
Upkeep Timestamp:
2025-07-14T16:44:14+00:00
Description
2021-08-06T17:08:58.760 INFO:teuthology.orchestra.run.smithi187.stdout:[kubelet-check] It seems like the kubelet isn't running or healthy.
2021-08-06T17:08:58.761 INFO:teuthology.orchestra.run.smithi187.stdout:[kubelet-check] The HTTP call equal to 'curl -sSL http://localhost:10248/healthz' failed with error: Get "http://localhost:10248/healthz": dial tcp [::1]:10248: connect: connection refused.
2021-08-06T17:08:58.761 INFO:teuthology.orchestra.run.smithi187.stdout:
2021-08-06T17:08:58.762 INFO:teuthology.orchestra.run.smithi187.stdout: Unfortunately, an error has occurred:
2021-08-06T17:08:58.762 INFO:teuthology.orchestra.run.smithi187.stdout: timed out waiting for the condition
2021-08-06T17:08:58.762 INFO:teuthology.orchestra.run.smithi187.stdout:
2021-08-06T17:08:58.763 INFO:teuthology.orchestra.run.smithi187.stdout: This error is likely caused by:
2021-08-06T17:08:58.763 INFO:teuthology.orchestra.run.smithi187.stdout: - The kubelet is not running
2021-08-06T17:08:58.763 INFO:teuthology.orchestra.run.smithi187.stdout: - The kubelet is unhealthy due to a misconfiguration of the node in some way (required cgroups disabled)
2021-08-06T17:08:58.763 INFO:teuthology.orchestra.run.smithi187.stdout:
2021-08-06T17:08:58.764 INFO:teuthology.orchestra.run.smithi187.stdout: If you are on a systemd-powered system, you can try to troubleshoot the error with the following commands:
2021-08-06T17:08:58.764 INFO:teuthology.orchestra.run.smithi187.stdout: - 'systemctl status kubelet'
2021-08-06T17:08:58.765 INFO:teuthology.orchestra.run.smithi187.stdout: - 'journalctl -xeu kubelet'
2021-08-06T17:08:58.765 INFO:teuthology.orchestra.run.smithi187.stdout:
2021-08-06T17:08:58.765 INFO:teuthology.orchestra.run.smithi187.stdout: Additionally, a control plane component may have crashed or exited when started by the container runtime.
2021-08-06T17:08:58.765 INFO:teuthology.orchestra.run.smithi187.stdout: To troubleshoot, list all containers using your preferred container runtimes CLI.
2021-08-06T17:08:58.765 INFO:teuthology.orchestra.run.smithi187.stdout:
2021-08-06T17:08:58.766 INFO:teuthology.orchestra.run.smithi187.stdout: Here is one example how you may list all Kubernetes containers running in docker:
2021-08-06T17:08:58.766 INFO:teuthology.orchestra.run.smithi187.stdout: - 'docker ps -a | grep kube | grep -v pause'
2021-08-06T17:08:58.766 INFO:teuthology.orchestra.run.smithi187.stdout: Once you have found the failing container, you can inspect its logs with:
2021-08-06T17:08:58.766 INFO:teuthology.orchestra.run.smithi187.stdout: - 'docker logs CONTAINERID'
2021-08-06T17:08:58.767 INFO:teuthology.orchestra.run.smithi187.stdout:
2021-08-06T17:08:58.767 DEBUG:teuthology.orchestra.run:got remote process result: 1
2021-08-06T17:08:58.768 INFO:teuthology.orchestra.run.smithi187.stderr:error execution phase wait-control-plane: couldn't initialize a Kubernetes cluster
2021-08-06T17:08:58.769 INFO:teuthology.orchestra.run.smithi187.stderr:To see the stack trace of this error execute with --v=5 or higher
2021-08-06T17:08:58.769 ERROR:tasks.kubeadm:Command failed on smithi187 with status 1: 'sudo kubeadm init --node-name smithi187 --token abcdef.8trb50ra0rrc1b01 --pod-network-cidr 10.253.208.0/21'
Traceback (most recent call last):
File "/home/teuthworker/src/github.com_ceph_ceph-c_3c0f8c8164075af7aac4d1f2805d3f4580709461/qa/tasks/kubeadm.py", line 252, in kubeadm_init_join
bootstrap_remote.run(args=cmd)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_04c2febe7099917d97a71271f17abb5710030132/teuthology/orchestra/remote.py", line 509, in run
r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
File "/home/teuthworker/src/git.ceph.com_git_teuthology_04c2febe7099917d97a71271f17abb5710030132/teuthology/orchestra/run.py", line 455, in run
r.wait()
File "/home/teuthworker/src/git.ceph.com_git_teuthology_04c2febe7099917d97a71271f17abb5710030132/teuthology/orchestra/run.py", line 161, in wait
self._raise_for_status()
File "/home/teuthworker/src/git.ceph.com_git_teuthology_04c2febe7099917d97a71271f17abb5710030132/teuthology/orchestra/run.py", line 183, in _raise_for_status
node=self.hostname, label=self.label
teuthology.exceptions.CommandFailedError: Command failed on smithi187 with status 1: 'sudo kubeadm init --node-name smithi187 --token abcdef.8trb50ra0rrc1b01 --pod-network-cidr 10.253.208.0/21'
/a/yuriw-2021-08-06_16:31:19-rados-wip-yuri-master-8.6.21-distro-basic-smithi/6324419
Updated by Neha Ojha over 4 years ago
- Backport set to pacific
/a/sseshasa-2021-08-06_04:49:51-rados-wip-sseshasa2-testing-2021-08-04-1847-pacific-distro-basic-smithi/6323276
Updated by Joseph Sawaya over 4 years ago
- Status changed from New to Pending Backport
Updated by Deepika Upadhyay over 4 years ago
- Priority changed from Normal to High
Updated by Deepika Upadhyay over 4 years ago
ceph/teuthology-archive/yuriw-2021-10-02_15:03:31-rados-wip-yuri2-testing-2021-10-01-0902-pacific-distro-basic-smithi/641
7863/teuthology.log
Updated by Laura Flores over 4 years ago
teuthology/yuriw-2021-11-08_15:10:38-rados-wip-yuri8-testing-2021-11-02-1009-pacific-distro-basic-smithi/6491072/teuthology.log
Updating this Tracker to keep a history of the issue. This occurred in a recent Pacific Teuthology run.
Updated by Neha Ojha over 4 years ago
Pacific backport: https://github.com/ceph/ceph/pull/43937
Updated by Sebastian Wagner about 4 years ago
- Status changed from Pending Backport to Resolved
Updated by Laura Flores over 3 years ago
- Related to Bug #58258: rook: kubelet fails from connection refused added
Updated by Upkeep Bot 8 months ago
- Merge Commit set to 84479e03a7e91e6a2fd28d220542c64317b31922
- Fixed In set to v17.0.0-6791-g84479e03a7
- Released In set to v17.2.0~1224
- Upkeep Timestamp set to 2025-07-14T16:44:14+00:00
Actions