Skip to content

reef: mgr/nfs: disallow non-existent paths when creating export#50808

Merged
yuriw merged 14 commits intoceph:reeffrom
dparmar18:wip-59251-reef
May 16, 2023
Merged

reef: mgr/nfs: disallow non-existent paths when creating export#50808
yuriw merged 14 commits intoceph:reeffrom
dparmar18:wip-59251-reef

Conversation

@dparmar18
Copy link
Contributor

@dparmar18 dparmar18 commented Mar 31, 2023

Backport tracker: https://tracker.ceph.com/issues/59251, https://tracker.ceph.com/issues/59248 and https://tracker.ceph.com/issues/59245
Backport of #49460
Parent tracker: https://tracker.ceph.com/issues/58228, https://tracker.ceph.com/issues/58744 and https://tracker.ceph.com/issues/58758

Contribution Guidelines

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

this helper instantiates CephfsClient, however this was
initially planned in ExportMgr class in export.py but
due to make check failure where main python thread
experienced a dead lock which after several efforts
pointed at instantiation of CephfsClient in ExportMgr
was problematic, it was decided in order to achieve
singleton behavior, func has been added inside this
helper func that restricts instantiation using functool's
lru_cache.

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 1356992)
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 6334e10)
Fixes: https://tracker.ceph.com/issues/58228
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 992c686)
Fixes: https://tracker.ceph.com/issues/58228
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit b0bb250)
Fixes: https://tracker.ceph.com/issues/58228
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit afaf251)
Also adds a function _nfs_complete_cmd() that returns process obj so that stdout/stderr
can be used for evaluation(_nfs_cmd() uses raw_cluster_cmd() that returns just stdout
and it became difficult to time cluster creation errors in _test_create_cluster()).

It takes sometime to update the cluster data, therefore running the command set
(check nfs server status -> nfs cluster create test -> check cluster status) in
a loop (max six iteration with sleep of 5 secs at each iteration) fixes the issue.

Fixes: https://tracker.ceph.com/issues/58744
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 7cb3e58)
…terid'

Fixes: https://tracker.ceph.com/issues/58758
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 3b3fa71)
Comment in the code says to wait for two minutes as cluster
creation takes time but actually it's waiting for thirteen
minutes, it's not required to wait this long, i think a minute
here is more than enough, also switched to using safe_while().

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 503f867)
_get_port_ip_info() fails to fetch port and ip due to empty 'backend' key:

2023-02-24T20:49:09.084 DEBUG:teuthology.orchestra.run.smithi042:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph nfs cluster info test
2023-02-24T20:49:09.471 INFO:teuthology.orchestra.run.smithi042.stdout:{
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:  "test": {
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:    "backend": [],
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:    "virtual_ip": null
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:  }
2023-02-24T20:49:09.472 INFO:teuthology.orchestra.run.smithi042.stdout:}

it then raises:

2023-02-24T20:49:10.323 INFO:tasks.cephfs_test_runner:    info_output = json.loads(self._nfs_cmd('cluster', 'info', self.cluster_id))['test']['backend'][0]
2023-02-24T20:49:10.323 INFO:tasks.cephfs_test_runner:IndexError: list index out of range

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 310286f)
@dparmar18 dparmar18 requested a review from a team March 31, 2023 08:15
@rishabh-d-dave rishabh-d-dave added this to the reef milestone Mar 31, 2023
@vshankar vshankar added the DNM label Apr 12, 2023
@vshankar vshankar changed the title reef: mgr/nfs: disallow non-existent paths when creating export [DNM] reef: mgr/nfs: disallow non-existent paths when creating export Apr 12, 2023
@dparmar18
Copy link
Contributor Author

dparmar18 commented Apr 25, 2023

@vshankar this one needs commits from #51005 (after merged into main) before moving ahead right?

@vshankar
Copy link
Contributor

@vshankar this one needs commits from #51005 (after merged into main) before moving ahead right?

yes.

- Renamed to cephfs_path_is_dir

- Removed exception handling to prevent redundant log statements like:
   "No such file or directory error in stat: b'/mnt/testdir_symlink': No such file or directory [Errno 2]"

  Exceptions handled inside caller eliminates this redundancy

- Set modifier flag AT_SYMLINK_NOFOLLOW

- Removed string "{path} is not a dir" when raising NotADirectoryError
  Rationale: will be handled in export.py

- change mock to cephfs_path_is_dir

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit f3d7370)
dparmar18 added 4 commits May 2, 2023 15:20
in create_cephfs_export()

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit d759fe4)
It actually didn't test the invalid path but still ended with
ENOENT(which is expected in case path is invalid) as the test
didn't create a fs, and it failed saying "FS nfs-cephfs not found"
which too raises ENOENT and thus it always passed.

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 5cc0857)
- test_nfs_export_creation_at_filepath:
ENOTDIR is raised instead of EINVAL which is better
aligned with the nature of the failure

- test_nfs_export_creation_at_symlink:
ENOTDIR is raised instead of ENOENT since the code
can now check if the path is symlink but won't follow
it.

Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 0c89625)
Signed-off-by: Dhairya Parmar <dparmar@redhat.com>
(cherry picked from commit 7a6ab31)
@dparmar18 dparmar18 changed the title [DNM] reef: mgr/nfs: disallow non-existent paths when creating export reef: mgr/nfs: disallow non-existent paths when creating export May 2, 2023
@dparmar18 dparmar18 removed the DNM label May 2, 2023
@ljflores
Copy link
Member

ljflores commented May 3, 2023

jenkins test api

@ljflores
Copy link
Member

ljflores commented May 3, 2023

jenkins test make check

@ljflores
Copy link
Member

ljflores commented May 3, 2023

Rados suite review: https://pulpito.ceph.com/?branch=wip-yuri6-testing-2023-04-26-1247-reef

Failures, unrelated:
1. https://tracker.ceph.com/issues/59049
2. https://tracker.ceph.com/issues/59057
3. https://tracker.ceph.com/issues/59142
4. https://tracker.ceph.com/issues/58797

Details:
1. WaitReplicas::react(const DigestUpdate&): Unexpected DigestUpdate event - Ceph - RADOS
2. rados/test_envlibrados_for_rocksdb.sh: No rule to make target 'rocksdb_env_librados_test' on centos 8 - Ceph - RADOS
3. mgr/dashboard: fix e2e for dashboard v3 - Ceph - Mgr - Dashboard
4. scrub/osd-scrub-dump.sh: TEST_recover_unexpected fails from "ERROR: Unexpectedly low amount of scrub reservations seen during test" - Ceph - RADOS

@yuriw yuriw merged commit 3300e36 into ceph:reef May 16, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants