mgr_util: add CephfsClient implementation by jan--f · Pull Request #32319 · ceph/ceph

jan--f · 2019-12-18T10:43:30Z

This pulls parts of the VolumesClient implementation into mgr_util to
make the CephFS specific pieces available to other mgr modules. To
reduce code dumplication the VolumeClient now extends the CephfsClient
class to add the volume specific methods.

Signed-off-by: Jan Fajerski jfajerski@suse.com

I did some of this originally in #29489. This is an update with some changes the VolumeClient received in the mean time. One change is that the connection_pool_wrap is now a static method (it doesn't need to be an object method) and that it doesn't replace the fs_name with a fs_handle but returns an object that should behave just like the fs_name string but also contains the handle.
This is more of an RFC, but I think its useful to have a basic CephfsClient implementation in mgr_util.

Checklist

References tracker ticket
Updates documentation if necessary
Includes tests for new functionality or reproducer for bug

Show available Jenkins commands

jenkins retest this please
jenkins test crimson perf
jenkins test signed
jenkins test make check
jenkins test make check arm64
jenkins test submodules
jenkins test dashboard
jenkins test dashboard backend
jenkins test docs
jenkins render docs
jenkins test ceph-volume all
jenkins test ceph-volume tox

sebastian-philipp · 2019-12-18T15:18:55Z

Already fixed with #32597

mind adding

diff --git a/src/pybind/mgr/tox.ini b/src/pybind/mgr/tox.ini
index 7cce321c68..396ed3c7c5 100644
--- a/src/pybind/mgr/tox.ini
+++ b/src/pybind/mgr/tox.ini
@@ -12,4 +12,4 @@ basepython = python3
 deps =
     -r requirements.txt
     mypy
-commands = mypy --config-file=../../mypy.ini orchestrator.py cephadm/module.py rook/module.py ansible/module.py
\ No newline at end of file
+commands = mypy --config-file=../../mypy.ini orchestrator.py mgr_util.py cephadm/module.py rook/module.py ansible/module.py
\ No newline at end of file

to your PR?

batrick · 2019-12-18T20:48:33Z

deferring to @vshankar on this one

vshankar · 2019-12-19T04:47:49Z

Thanks @jan--f -- I was planning to work on this today!

src/pybind/mgr/volumes/fs/volume.py

sebastian-philipp · 2020-01-13T10:10:13Z

jenkins test make check

sebastian-philipp · 2020-01-13T11:40:21Z

mgr_util.py: note: In member "disconnect" of class "Connection":
mgr_util.py:110: error: Item "None" of "Optional[Any]" has no attribute "get_addrs"
mgr_util.py:111: error: Item "None" of "Optional[Any]" has no attribute "shutdown"
mgr_util.py: note: In member "abort" of class "Connection":
mgr_util.py:118: error: Item "None" of "Optional[Any]" has no attribute "abort_conn"
mgr_util.py: note: In member "run" of class "RTimer":
mgr_util.py:127: error: "RTimer" has no attribute "finished"
mgr_util.py:128: error: "RTimer" has no attribute "finished"
mgr_util.py:128: error: "RTimer" has no attribute "interval"
mgr_util.py:129: error: "RTimer" has no attribute "function"
mgr_util.py:129: error: "RTimer" has no attribute "args"
mgr_util.py:129: error: "RTimer" has no attribute "kwargs"
mgr_util.py:130: error: "RTimer" has no attribute "finished"
mgr_util.py: note: In member "shutdown" of class "CephfsClient":
mgr_util.py:222: error: "CephfsClient" has no attribute "purge_queue"

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

vshankar

needs rebase.

@jan--f note that with the recent change to mgr/volumes plugin, I removed the dependency on connection_pool_wrap decorator. I'm ok with still having it if there are users.

src/pybind/mgr/mgr_util.py

jan--f · 2020-01-23T14:00:15Z

Updated things according to recent changes in the volumes implementation

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

src/pybind/mgr/mgr_util.py

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f · 2020-03-12T14:03:39Z

ping @vshankar

jan--f · 2020-03-18T10:56:26Z

@vshankar Anything still blocking this?

vshankar · 2020-03-18T11:17:58Z

@vshankar Anything still blocking this?

nothing -- got busy elsewhere. Will schedule test soon.

vshankar · 2020-03-19T04:15:23Z

http://pulpito.ceph.com/vshankar-2020-03-18_16:53:39-fs-wip-vshankar-testing-2020-03-18-095216-testing-basic-smithi/

failures are not related to this pr

vshankar

LGTM

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f added cephfs Ceph File System pybind labels Dec 18, 2019

jan--f requested review from ajarr, batrick and vshankar December 18, 2019 10:43

batrick added the needs-review label Dec 18, 2019

vshankar self-assigned this Dec 19, 2019

vshankar reviewed Jan 7, 2020

View reviewed changes

src/pybind/mgr/volumes/fs/volume.py Show resolved Hide resolved

jan--f pushed a commit to jan--f/ceph that referenced this pull request Jan 16, 2020

WIP: adjust interface according to discussion in ceph#32319

10dd8c5

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

vshankar requested changes Jan 17, 2020

View reviewed changes

src/pybind/mgr/mgr_util.py Outdated Show resolved Hide resolved

src/pybind/mgr/mgr_util.py Outdated Show resolved Hide resolved

src/pybind/mgr/mgr_util.py Outdated Show resolved Hide resolved

sebastian-philipp added the needs-rebase label Jan 22, 2020

jan--f force-pushed the pybind-mgr-util-cephfs-con-pool branch from 11f9425 to df03d2f Compare January 23, 2020 13:59

jan--f requested a review from vshankar January 23, 2020 13:59

jan--f removed the needs-rebase label Jan 23, 2020

jan--f pushed a commit to jan--f/ceph that referenced this pull request Jan 24, 2020

WIP: adjust interface according to discussion in ceph#32319

966d58d

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f pushed a commit to jan--f/ceph that referenced this pull request Jan 24, 2020

WIP: adjust interface according to discussion in ceph#32319

b114b38

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f pushed a commit to jan--f/ceph that referenced this pull request Jan 24, 2020

WIP: adjust interface according to discussion in ceph#32319

283a634

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

sebastian-philipp reviewed Jan 27, 2020

View reviewed changes

src/pybind/mgr/mgr_util.py Show resolved Hide resolved

sebastian-philipp suggested changes Jan 27, 2020

View reviewed changes

src/pybind/mgr/mgr_util.py Outdated Show resolved Hide resolved

vshankar reviewed Jan 27, 2020

View reviewed changes

src/pybind/mgr/mgr_util.py Outdated Show resolved Hide resolved

jan--f force-pushed the pybind-mgr-util-cephfs-con-pool branch from df03d2f to 87726d7 Compare February 10, 2020 14:01

jan--f pushed a commit to jan--f/ceph that referenced this pull request Feb 10, 2020

WIP: adjust interface according to discussion in ceph#32319

5043d40

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f requested a review from vshankar February 11, 2020 09:35

jan--f pushed a commit to jan--f/ceph that referenced this pull request Feb 13, 2020

WIP: adjust interface according to discussion in ceph#32319

90a01e4

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f added the needs-qa label Mar 12, 2020

sebastian-philipp mentioned this pull request Mar 17, 2020

doc: Add a generated reference of all mon commands. #33886

Merged

5 tasks

vshankar added wip-vshankar-testing and removed wip-vshankar-testing labels Mar 18, 2020

vshankar approved these changes Mar 19, 2020

View reviewed changes

jan--f removed needs-qa needs-review labels Mar 20, 2020

sebastian-philipp merged commit 8c681d2 into ceph:master Mar 20, 2020

jan--f pushed a commit to jan--f/ceph that referenced this pull request Apr 2, 2020

WIP: adjust interface according to discussion in ceph#32319

b409c50

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f pushed a commit to jan--f/ceph that referenced this pull request Apr 2, 2020

wip rebase on ceph#32319

d7a4183

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

sebastian-philipp mentioned this pull request Apr 2, 2020

pybind/mgr: Fix run_tox.sh mgr -- cephadm #34374

Merged

3 tasks

vshankar pushed a commit to vshankar/ceph that referenced this pull request Apr 14, 2020

WIP: adjust interface according to discussion in ceph#32319

73f377f

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

vshankar pushed a commit to vshankar/ceph that referenced this pull request Apr 14, 2020

wip rebase on ceph#32319

8529094

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f pushed a commit to jan--f/ceph that referenced this pull request Apr 16, 2020

WIP: adjust interface according to discussion in ceph#32319

89005ab

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f pushed a commit to jan--f/ceph that referenced this pull request Apr 16, 2020

wip rebase on ceph#32319

1d0a92d

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f pushed a commit to jan--f/ceph that referenced this pull request Apr 16, 2020

WIP: adjust interface according to discussion in ceph#32319

2e41f41

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f pushed a commit to jan--f/ceph that referenced this pull request Apr 16, 2020

wip rebase on ceph#32319

46c7258

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

vshankar pushed a commit to ceph/ceph-ci that referenced this pull request Apr 28, 2020

WIP: adjust interface according to discussion in ceph/ceph#32319

ff31412

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f pushed a commit to jan--f/ceph that referenced this pull request Apr 30, 2020

WIP: adjust interface according to discussion in ceph#32319

2d47864

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f pushed a commit to jan--f/ceph that referenced this pull request Apr 30, 2020

wip rebase on ceph#32319

9d8241d

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

vshankar pushed a commit to vshankar/ceph that referenced this pull request Apr 30, 2020

WIP: adjust interface according to discussion in ceph#32319

8e3fc9a

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

vshankar pushed a commit to vshankar/ceph that referenced this pull request Apr 30, 2020

wip rebase on ceph#32319

5ae1d1e

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f pushed a commit to jan--f/ceph that referenced this pull request May 4, 2020

WIP: adjust interface according to discussion in ceph#32319

98e0d39

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f pushed a commit to jan--f/ceph that referenced this pull request May 4, 2020

wip rebase on ceph#32319

8850283

Signed-off-by: Jan Fajerski <jfajerski@suse.com>

jan--f mentioned this pull request Sep 14, 2020

octopus: cephfs: backport snap-schedule module #37142

Closed

kotreshhr mentioned this pull request Feb 28, 2022

octopus: mgr/volumes: A few volumes plugin backport #44800

Merged

14 tasks

Conversation

jan--f commented Dec 18, 2019

Checklist

Uh oh!

sebastian-philipp commented Dec 18, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

batrick commented Dec 18, 2019

Uh oh!

vshankar commented Dec 19, 2019

Uh oh!

Uh oh!

sebastian-philipp commented Jan 13, 2020

Uh oh!

sebastian-philipp commented Jan 13, 2020

Uh oh!

vshankar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jan--f commented Jan 23, 2020

Uh oh!

Uh oh!

Uh oh!

Uh oh!

jan--f commented Mar 12, 2020

Uh oh!

jan--f commented Mar 18, 2020

Uh oh!

vshankar commented Mar 18, 2020

Uh oh!

vshankar commented Mar 19, 2020

Uh oh!

vshankar left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

sebastian-philipp commented Dec 18, 2019 •

edited

Loading