qa/tests: try using `cephadm` by yuriw · Pull Request #35622 · ceph/ceph

yuriw · 2020-06-17T17:22:12Z

Signed-off-by: Yuri Weinstein yweinste@redhat.com

Checklist

References tracker ticket
Updates documentation if necessary
Includes tests for new functionality or reproducer for bug

Show available Jenkins commands

jenkins retest this please
jenkins test classic perf
jenkins test crimson perf
jenkins test signed
jenkins test make check
jenkins test make check arm64
jenkins test submodules
jenkins test dashboard
jenkins test dashboard backend
jenkins test docs
jenkins render docs
jenkins test ceph-volume all
jenkins test ceph-volume tox

yuriw · 2020-06-17T17:36:36Z

http://pulpito.ceph.com/teuthology-2020-06-17_17:24:17-rados:basic-master-distro-basic-smithi/

http://pulpito.ceph.com/teuthology-2020-06-17_19:29:49-rados:basic-master-distro-basic-smithi/

sebastian-philipp · 2020-06-18T09:26:49Z

2020-06-17T17:50:38.789 INFO:teuthology.orchestra.run.smithi117.stderr:Error ENOENT: ('name %s already in use', 'b')
debug 2020-06-17T17:50:38.782+0000 7f8276554700 -1 log_channel(cephadm) log [ERR] : _Promise failed
Traceback (most recent call last):
  File "/usr/share/ceph/mgr/orchestrator/_interface.py", line 275, in _finalize
    next_result = self._on_complete(self._value)
  File "/usr/share/ceph/mgr/cephadm/module.py", line 131, in <lambda>
    return CephadmCompletion(on_complete=lambda _: f(*args, **kwargs))
  File "/usr/share/ceph/mgr/cephadm/module.py", line 2004, in add_mon
    return self._add_daemon('mon', spec, self.mon_service.create)
  File "/usr/share/ceph/mgr/cephadm/module.py", line 1957, in _add_daemon
    create_func, config_func)
  File "/usr/share/ceph/mgr/cephadm/module.py", line 1973, in _create_daemons
    forcename=name)
  File "/usr/share/ceph/mgr/cephadm/module.py", line 570, in get_unique_name
    raise orchestrator.OrchestratorValidationError('name %s already in use', forcename)
orchestrator._interface.OrchestratorValidationError: ('name %s already in use', 'b')
debug 2020-06-17T17:50:38.782+0000 7f8276554700 -1 mgr.server reply reply (2) No such file or directory ('name %s already in use', 'b')
2020-06-17T17:50:39.397 DEBUG:teuthology.orchestra.run:got remote process result: 2
2020-06-17T17:50:39.398 ERROR:teuthology.contextutil:Saw exception from nested tasks
Traceback (most recent call last):
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 32, in nested
    vars.append(enter())
  File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__
    return next(self.gen)
  File "/home/teuthworker/src/github.com_yuriw_ceph_wip-yuriw-rados-master/qa/tasks/cephadm.py", line 502, in ceph_mons
    remote.shortname + ':' + ctx.ceph[cluster_name].mons[mon] + '=' + id_,
  File "/home/teuthworker/src/github.com_yuriw_ceph_wip-yuriw-rados-master/qa/tasks/cephadm.py", line 47, in _shell
    **kwargs
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 206, in run
    r = self._runner(client=self.ssh, name=self.shortname, **kwargs)
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 475, in run
    r.wait()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 162, in wait
    self._raise_for_status()
  File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/run.py", line 184, in _raise_for_status
    node=self.hostname, label=self.label
teuthology.exceptions.CommandFailedError: Command failed on smithi117 with status 2: 'sudo /home/ubuntu/cephtest/cephadm --image quay.ceph.io/ceph-ci/ceph:9df7e03345a31351fa0be5602855e9fdcda5e993 shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyrin

One idea would be to use the qa/suites/rados/cephadm/smoke/fixed-2.yaml cluster.

sebastian-philipp · 2020-06-24T12:25:53Z

@yuriw any update on this?

yuriw · 2020-06-24T15:16:36Z

@sebastian-philipp not yet @JoshD was reviewing

tchaikov

could you please remove the merge commit from this PR?

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

This sets up the prerequisites for many tests. Signed-off-by: Josh Durgin <jdurgin@redhat.com>

yuriw · 2020-07-01T22:08:21Z

testing on @jdurgin commit
http://pulpito.ceph.com/teuthology-2020-07-01_22:05:32-rados:basic-master-distro-basic-smithi/

Bootstrapping with roleless mode would otherwise potentially use a different host than first_mon/first_mgr, and when the daemons are later deployed on all hosts, skipping first_mon/first_mgr, we'd be trying to create the same mon/mgr twice, e.g. mon.smithi028 and failing because it already existed. Signed-off-by: Josh Durgin <jdurgin@redhat.com>

yuriw · 2020-07-02T01:41:04Z

that ^ passed

and this is cephadm
https://pulpito.ceph.com/teuthology-2020-07-02_01:38:17-rados:cephadm-master-distro-basic-smithi

dismissed

tchaikov · 2020-07-11T06:56:14Z

qa/tasks/cephadm.py

            '/etc/ceph/{}.client.admin.keyring'.format(cluster_name),
            '--output-pub-ssh-key', '{}/{}.pub'.format(testdir, cluster_name),
+            '--mon-id', first_mon,
+            '--mgr-id', first_mgr,


2020-07-11T05:43:03.562 INFO:tasks.cephadm:Teardown complete 2020-07-11T05:43:03.562 ERROR:teuthology.run_tasks:Saw exception from tasks. Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-07-11-1109/qa/tasks/cephadm.py", line 368, in ceph_bootstrap '--mgr-id', first_mgr, UnboundLocalError: local variable 'first_mgr' referenced before assignment During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/run_tasks.py", line 93, in run_tasks manager.__enter__() File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-07-11-1109/qa/tasks/cephadm.py", line 1217, in task lambda: cephfs_setup(ctx=ctx, config=config), File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 55, in nested raise exc[1] File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__ self.gen.throw(type, value, traceback) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-07-11-1109/qa/tasks/cephadm.py", line 1095, in initialize_config yield File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/contextutil.py", line 31, in nested vars.append(enter()) File "/usr/lib/python3.6/contextlib.py", line 81, in __enter__ return next(self.gen) File "/home/teuthworker/src/git.ceph.com_ceph-c_wip-kefu-testing-2020-07-11-1109/qa/tasks/cephadm.py", line 471, in ceph_bootstrap ctx.daemons.get_daemon(type_, id_, cluster).stop() File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/daemon/cephadmunit.py", line 149, in stop self.remote.sh(self.stop_cmd) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 248, in sh proc = self.run(**kwargs) File "/home/teuthworker/src/git.ceph.com_git_teuthology_master/teuthology/orchestra/remote.py", line 204, in run r = self._runner(client=self.ssh, name=self.shortname, **kwargs)

https://pulpito.ceph.com/kchai-2020-07-11_05:07:55-rados-wip-kefu-testing-2020-07-11-1109-distro-basic-smithi/

hm. --mon-id and --mgr-id only make sense in roleless mode. We#re leaving the roleless mode when we require the -mon-id param.

I'd say either we have to use the non-roleless mode or we have to fix the tests.

ok this change doesn't really make sense. can you revert this mon-ip thingy, please?

stale · 2020-09-12T13:03:37Z

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

sebastian-philipp · 2020-09-12T19:24:48Z

unstale.

stale · 2020-11-15T18:56:09Z

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

yuriw requested a review from jdurgin June 17, 2020 17:22

batrick added cephadm needs-review tests labels Jun 17, 2020

tchaikov added the needs-qa label Jun 18, 2020

sebastian-philipp mentioned this pull request Jun 18, 2020

mgr/cephadm: fix broken "already in use" exception message #35643

Merged

3 tasks

sebastian-philipp added wip-swagner3-testing and removed wip-swagner3-testing labels Jun 19, 2020

tchaikov added wip-kefu-testing and removed wip-kefu-testing labels Jun 23, 2020

tchaikov previously requested changes Jun 27, 2020

View reviewed changes

yuriw force-pushed the wip-yuriw-rados-master branch from f2462d0 to e02cf17 Compare June 29, 2020 16:01

qa/tests: try using cephadm

e1dd7ae

Signed-off-by: Yuri Weinstein <yweinste@redhat.com>

yuriw force-pushed the wip-yuriw-rados-master branch from e02cf17 to e1dd7ae Compare July 1, 2020 21:50

qa/tasks/cephadm: create an rbd pool and cephfs like the ceph task

a3bb599

This sets up the prerequisites for many tests. Signed-off-by: Josh Durgin <jdurgin@redhat.com>

yuriw requested a review from a team as a code owner July 1, 2020 21:53

jdurgin requested a review from sebastian-philipp July 2, 2020 01:39

tchaikov added the wip-kefu-testing label Jul 10, 2020

tchaikov requested changes Jul 11, 2020

View reviewed changes

tchaikov removed needs-qa wip-kefu-testing labels Jul 11, 2020

stale bot added the stale label Sep 12, 2020

stale bot removed the stale label Sep 12, 2020

stale bot added the stale label Nov 15, 2020

yuriw added the DNM label Dec 14, 2020

stale bot removed the stale label Dec 14, 2020

yuriw closed this Dec 14, 2020

Conversation

yuriw commented Jun 17, 2020

Checklist

Uh oh!

yuriw commented Jun 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

sebastian-philipp commented Jun 18, 2020

Uh oh!

sebastian-philipp commented Jun 24, 2020

Uh oh!

yuriw commented Jun 24, 2020

Uh oh!

tchaikov left a comment

Choose a reason for hiding this comment

Uh oh!

yuriw commented Jul 1, 2020

Uh oh!

yuriw commented Jul 2, 2020

Uh oh!

tchaikov Jul 11, 2020

Choose a reason for hiding this comment

Uh oh!

sebastian-philipp Jul 14, 2020

Choose a reason for hiding this comment

Uh oh!

sebastian-philipp Sep 12, 2020

Choose a reason for hiding this comment

Uh oh!

stale bot commented Sep 12, 2020

Uh oh!

sebastian-philipp commented Sep 12, 2020

Uh oh!

stale bot commented Nov 15, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

yuriw commented Jun 17, 2020 •

edited

Loading