mgr/orchestrator: Add test orchestrator by sebastian-philipp · Pull Request #25236 · ceph/ceph

sebastian-philipp · 2018-11-23T13:58:12Z

To be able to run the cli without an external orchestrator.
Run the CLI in Teuthology.

Signed-off-by: Sebastian Wagner sebastian.wagner@suse.com

TODO

Make the call to ceph-volume actually work. See [RFC] ceph-volume: Make c-v invocable without installing it #25237
Includes tests for new functionality or reproducer for bug

tchaikov · 2018-11-25T10:41:56Z

retest this please

sebastian-philipp · 2018-12-10T13:41:17Z

@alfredodeza This PR now contains the changes needed to create the virtual environment as discussed in #25237

sebastian-philipp · 2018-12-10T14:48:40Z

src/pybind/mgr/test_orchestrator/module.py

+                devs = []
+                for device in json.loads(out):
+                    dev = orchestrator.InventoryDevice()
+                    if 'partitions' not in device["sys_api"]:


This filters our partitions in the JSON output. Relates to #25390. Can be removed afterwards

#25390 is merged now, so I guess this can be removed?

sebastian-philipp · 2018-12-18T12:34:19Z

@alfredodeza ping?

jtlayton

Looks reasonable to me. I like the idea of a test orchestrator for this and it may be nice for doing orchestrator/dashboard integration too.

tserong

Instead of setting up a virtualenv, is there some way ceph-volume could be installed to build/bin, along with the ceph cli tool and the other binaries? Because that'd be kinda neat :-)

tserong · 2018-12-20T04:18:45Z

src/pybind/mgr/test_orchestrator/module.py

+        self.execute_cb = execute_cb
+        self.complete_cb = complete_cb
+
+        # Executed means I executed my k8s API call, it may or may


Probably souldn't mention k8s

tserong · 2018-12-20T04:19:02Z

src/pybind/mgr/test_orchestrator/module.py

+
+def deferred_read(f):
+    """
+    Decorator to make RookOrchestrator methods return


s/RookOrchestrator/TestOrchestrator/

tserong · 2018-12-20T04:26:31Z

src/pybind/mgr/test_orchestrator/module.py

+                devs = []
+                for device in json.loads(out):
+                    dev = orchestrator.InventoryDevice()
+                    if 'partitions' not in device["sys_api"]:


#25390 is merged now, so I guess this can be removed?

jan--f · 2018-12-19T12:46:40Z

src/pybind/mgr/orchestrator.py


        :param node_filter:
        :return: list of InventoryNode
+        :rtype: InventoryNode


Is this supposed to be part of this PR? Also the TestOrchestrator seems to return [InventoryNode]?

removed form this PR. Thanks!

sebastian-philipp · 2018-12-20T08:01:11Z

@tserong

Instead of setting up a virtualenv, is there some way ceph-volume could be installed to build/bin, along with the ceph cli tool and the other binaries? Because that'd be kinda neat :-)

Actually, you don't need to install it at all. You can invoke c-v directly from the source folder. #25237 is the only thing missing here.

tserong · 2018-12-20T09:05:21Z

Actually, you don't need to install it at all. You can invoke c-v directly from the source folder. #25237 is the only thing missing here.

I guess I meant: be able to invoke it from the build/bin directory, the same way you can invoke build/bin/ceph currently. Would ceph-mgr be able to find ceph-volume then, in that directory, without having to activate a virtualenv internally?

sebastian-philipp · 2018-12-20T09:51:01Z

2018-12-20 10:22:17,176.176 INFO:__main__:test_status (tasks.mgr.test_orchestrator_cli.TestDashboard) ... ok
2018-12-20 10:22:17,176.176 INFO:__main__:Stopped test: test_status (tasks.mgr.test_orchestrator_cli.TestDashboard) in 9.691915s
2018-12-20 10:22:17,176.176 INFO:__main__:
2018-12-20 10:22:17,176.176 INFO:__main__:----------------------------------------------------------------------
2018-12-20 10:22:17,177.177 INFO:__main__:Ran 3 tests in 66.019s
2018-12-20 10:22:17,177.177 INFO:__main__:
2018-12-20 10:22:17,177.177 INFO:__main__:OK

1. To be able to run the cli without an external orchestrator. 2. Run the CLI in Teuthology. Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

Adds new add_custom_target `ceph-volume-venv-setup` that creates a virtual environment, and calls `python setup.py develop` Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

sebastian-philipp · 2018-12-20T10:03:37Z

@liewegas . I just added the missing qa/suites/rados/mgr/tasks/orchestrator_cli.yaml.

* refs/pull/25236/head: ceph-volume: Add venv to vstart cmake target mgr/orchestrator: Add test orchestrator Reviewed-by: Jeff Layton <jlayton@redhat.com>

sebastian-philipp added the orchestrator label Nov 23, 2018

sebastian-philipp requested review from alfredodeza, liewegas and travisn November 23, 2018 13:58

sebastian-philipp mentioned this pull request Nov 23, 2018

[RFC] ceph-volume: Make c-v invocable without installing it #25237

Closed

sebastian-philipp added the DNM label Nov 26, 2018

sebastian-philipp force-pushed the orchestrator-test-orchestrator branch from a014376 to 67fc10d Compare December 10, 2018 13:37

ceph-jenkins added the ceph-volume label Dec 10, 2018

sebastian-philipp mentioned this pull request Dec 10, 2018

ceph-volume: Fix TypeError: join() takes exactly one argument (2 given) #25469

Merged

sebastian-philipp commented Dec 10, 2018

View reviewed changes

sebastian-philipp requested a review from jan--f December 10, 2018 14:48

sebastian-philipp added needs-review and removed DNM labels Dec 11, 2018

This was referenced Dec 11, 2018

pybind/mgr: Unified bits of volumes and orchestrator #25492

Merged

mgr/orchestrator: Add JSON output to CLI commands #25340

Merged

sebastian-philipp requested review from ifed01 and removed request for ifed01 December 18, 2018 14:02

sebastian-philipp added tests needs-qa labels Dec 18, 2018

sebastian-philipp requested a review from jtlayton December 18, 2018 16:07

sebastian-philipp mentioned this pull request Dec 18, 2018

mgr/orchestrator: fix _list_services display #25610

Merged

3 tasks

tserong self-requested a review December 19, 2018 09:36

jtlayton approved these changes Dec 19, 2018

View reviewed changes

liewegas added the wip-sage3-testing label Dec 19, 2018

tserong reviewed Dec 20, 2018

View reviewed changes

jan--f reviewed Dec 20, 2018

View reviewed changes

sebastian-philipp force-pushed the orchestrator-test-orchestrator branch from 67fc10d to fe1f713 Compare December 20, 2018 09:48

sebastian-philipp added 2 commits December 20, 2018 10:56

mgr/orchestrator: Add test orchestrator

933b2cf

1. To be able to run the cli without an external orchestrator. 2. Run the CLI in Teuthology. Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

ceph-volume: Add venv to vstart cmake target

3164551

Adds new add_custom_target `ceph-volume-venv-setup` that creates a virtual environment, and calls `python setup.py develop` Signed-off-by: Sebastian Wagner <sebastian.wagner@suse.com>

sebastian-philipp force-pushed the orchestrator-test-orchestrator branch from fe1f713 to 3164551 Compare December 20, 2018 09:58

liewegas merged commit 3164551 into ceph:master Dec 20, 2018

liewegas added a commit that referenced this pull request Dec 20, 2018

Merge PR #25236 into master

bb804eb

* refs/pull/25236/head: ceph-volume: Add venv to vstart cmake target mgr/orchestrator: Add test orchestrator Reviewed-by: Jeff Layton <jlayton@redhat.com>

Conversation

sebastian-philipp commented Nov 23, 2018 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

TODO

Uh oh!

tchaikov commented Nov 25, 2018

Uh oh!

sebastian-philipp commented Dec 10, 2018

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sebastian-philipp commented Dec 18, 2018

Uh oh!

jtlayton left a comment

Choose a reason for hiding this comment

Uh oh!

tserong left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sebastian-philipp commented Dec 20, 2018

Uh oh!

tserong commented Dec 20, 2018

Uh oh!

sebastian-philipp commented Dec 20, 2018

Uh oh!

sebastian-philipp commented Dec 20, 2018

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

sebastian-philipp commented Nov 23, 2018 •

edited

Loading