Skip to content

mgr/volumes/nfs: Check cluster exists before creating exports and fetch exports from export objects if mgr restarted#35417

Merged
batrick merged 24 commits intoceph:masterfrom
varshar16:wip-make-exports-persistent
Jul 1, 2020
Merged

mgr/volumes/nfs: Check cluster exists before creating exports and fetch exports from export objects if mgr restarted#35417
batrick merged 24 commits intoceph:masterfrom
varshar16:wip-make-exports-persistent

Conversation

@varshar16
Copy link
Contributor

@varshar16 varshar16 commented Jun 5, 2020 •
edited
Loading

This patch series does the following changes:

  • Restrict cluster id to only [A-Za-z0-9-_.] characters
  • On restarting mgr, exports are fetched from export rados objects
  • Check cluster availability in export command methods
  • Add get export interface
  • Add ls export interface
  • Add nfs cluster ls interface
  • Add idempotency test for nfs commands and all the new changes

Fixes: https://tracker.ceph.com/issues/45740
Fixes: https://tracker.ceph.com/issues/45744
Fixes: https://tracker.ceph.com/issues/45741

Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard backend
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox

@varshar16
Copy link
Contributor Author

varshar16 commented Jun 5, 2020

Testing with vstart requires this PR #35344

@varshar16 varshar16 force-pushed the wip-make-exports-persistent branch from c432e26 to 166bbcb Compare June 8, 2020 09:52
@varshar16
Copy link
Contributor Author

varshar16 commented Jun 8, 2020

Fixed mypy errors.

@sebastian-philipp sebastian-philipp added the wip-swagner-testing My Teuthology tests label Jun 8, 2020
@sebastian-philipp
Copy link
Contributor

sebastian-philipp commented Jun 8, 2020

green: http://pulpito.ceph.com/swagner-2020-06-08_13:53:34-rados:cephadm-wip-swagner-testing-2020-06-08-1252-distro-basic-smithi/

@sebastian-philipp sebastian-philipp removed the wip-swagner-testing My Teuthology tests label Jun 8, 2020
ajarr
ajarr reviewed Jun 8, 2020
View reviewed changes
Copy link
Contributor

@ajarr ajarr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested locally. Export persistence across ganesha restart and cluster check works

@varshar16 varshar16 force-pushed the wip-make-exports-persistent branch 3 times, most recently from a17f81f to 702cb79 Compare June 9, 2020 15:39
@varshar16
Copy link
Contributor Author

varshar16 commented Jun 9, 2020

Added tests to check idempotency of nfs commands and export persistence on restarting mgr.

@varshar16
Copy link
Contributor Author

varshar16 commented Jun 10, 2020

Tests passed: http://qa-proxy.ceph.com/teuthology/varsha-2020-06-10_07:03:45-rados-wip-make-exports-persistent-distro-basic-smithi/

@varshar16 varshar16 mentioned this pull request Jun 11, 2020
Closed
@sebastian-philipp sebastian-philipp removed their request for review June 11, 2020 07:37
@sebastian-philipp
Copy link
Contributor

sebastian-philipp commented Jun 11, 2020

ack.

@jtlayton
Copy link
Contributor

jtlayton commented Jun 11, 2020

Looks reasonable overall aside from the protocols and transport versions being converted to singletons from a list.

ajarr
ajarr requested changes Jun 11, 2020
View reviewed changes
@varshar16 varshar16 force-pushed the wip-make-exports-persistent branch from 702cb79 to a657597 Compare June 11, 2020 16:41
@ajarr
Copy link
Contributor

ajarr commented Jun 14, 2020 •
edited
Loading

@varshar16 please pick the following commits from #35414
6c9161a
f8711d2
8967a3c

the 3 commit were rebased on top of this work. I can then abandon #35414

@ajarr ajarr mentioned this pull request Jun 14, 2020
Closed
3 tasks
varshar16 and others added 24 commits June 30, 2020 23:17
@varshar16
mgr/volumes/nfs: Fetch exports in persistent way if mgr is restarted
162a571 
Fixes: https://tracker.ceph.com/issues/45740
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
mgr/volumes/nfs: Check if cluster exists before creating exports
c13a40b 
Fixes: https://tracker.ceph.com/issues/45740
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
qa/tasks/cephfs: Add idempotency test for nfs commands
1a55888 
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
qa/tasks/cephfs: Add test to check if export is available on restarti…
3b554f2 
…ng mgr

Signed-off-by: Varsha Rao <varao@redhat.com>
@ajarr @varshar16
mgr/volume/nfs: allow only [A-Za-z0-9-_.] characters
7ae24c2 
... for ganesha cluster names.

Fixes: https://tracker.ceph.com/issues/45744
Signed-off-by: Ramana Raja <rraja@redhat.com>
@ajarr @varshar16
mgr/volumes/nfs: list exports within a ganesha cluster
6f8d20f 
List the pseudo paths of the exports within a ganesha cluster
with `nfs export ls <cluster ID>` command.

List all the details of the exports within a ganesha cluster
with `nfs export ls <cluster ID> --detailed` command.

Partially-fixes: https://tracker.ceph.com/issues/45741
Signed-off-by: Ramana Raja <rraja@redhat.com>
@ajarr @varshar16
mgr/volumes/nfs: get export in a nfs cluster
7dcefb1 
Fixes: https://tracker.ceph.com/issues/45741
Signed-off-by: Ramana Raja <rraja@redhat.com>
@varshar16
mgr/volumes/nfs: Remove 'ganesha-' prefix from orch returned service id
555da00 
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
mgr/volumes/nfs: Add nfs cluster ls command
b77c3ee 
This commands provides list of deployed nfs clusters.

Fixes: https://tracker.ceph.com/issues/45742
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
qa/tasks/cephfs: Add test for nfs cluster ls command
eb2cd16 
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
mgr/volumes/nfs: Misc changes to export get and ls command
310c7fb 
This patch does following changes:
* Add extra line before every class definition
* Reorder export dictionary
* Use _fetch_export() in get export command
* Remove cluster_id key from export dictionary if single export is deleted

Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
mgr/volumes/nfs: Check cluster availability in export command methods
9d83045 
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
mgr/volumes/nfs: Set pool_namespace and cluster_id through decorator
2692a94 
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
pybind/mgr/volumes/nfs: Reorder FSExport and NFSCluster class methods
ed20bd8 
To maintain consistency and improve readability.

Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
qa/tasks/cephfs: Add test for listing exports
9418b42 
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
qa/tasks/cephfs: Add tests for invalid cluster id, export type and mo…
285ab1b 
…dify list cluster

Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
qa/tasks/cephfs: Add tests for export create with non-existing fsname…
eab60b5 
… and cluster id

Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
doc/cephfs: Add list and get export usage
55fd634 
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
qa/tasks/cephfs/test_nfs: Add description for each test
dfa3ab4 
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
qa/tasks/cephfs/test_nfs: Add test for export user id
1aa2a66 
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
qa/tasks/cephfs/test_nfs: Add test for get and list detailed export
5f506e3 
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
qa/tasks/test_nfs: Add todo note for cluster update test
d7396fd 
Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16
pybind/mgr/volumes/nfs: Fix flake8 errors
4da7581 
Signed-off-by: Varsha Rao <varao@redhat.com>
@batrick @varshar16
volumes/fs/nfs: Don't parse rados export objects during FSExport init
4ebb258 
Instead we can do it if any export commands are used. This fixes ImportError
raised when FSExport is initialized with cephadm module disabled.

Signed-off-by: Varsha Rao <varao@redhat.com>
@varshar16 varshar16 force-pushed the wip-make-exports-persistent branch from fd9bceb to 4ebb258 Compare June 30, 2020 17:48
@batrick
Copy link
Member

batrick commented Jun 30, 2020

jenkins test make check

@batrick
Copy link
Member

batrick commented Jun 30, 2020

make check failure not caused by this PR

@batrick
Copy link
Member

batrick commented Jul 1, 2020

https://pulpito.ceph.com/pdonnell-2020-07-01_02:02:30-rados-wip-pdonnell-testing-20200630.195421-distro-basic-smithi/

@batrick batrick merged commit 2555ed1 into ceph:master Jul 1, 2020
@varshar16 varshar16 deleted the wip-make-exports-persistent branch August 5, 2020 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@batrick batrick batrick approved these changes

@ajarr ajarr ajarr approved these changes

@mgfritch mgfritch Awaiting requested review from mgfritch

@jtlayton jtlayton Awaiting requested review from jtlayton

+1 more reviewer

@sebastian-philipp sebastian-philipp sebastian-philipp left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

cephadm cephfs Ceph File System needs-review wip-varsha-testing

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@varshar16 @sebastian-philipp @jtlayton @ajarr @batrick