qa/cephfs: add helper methods in CephFSTestCase by rishabh-d-dave · Pull Request #50569 · ceph/ceph

rishabh-d-dave · 2023-03-17T06:54:30Z

Contribution Guidelines

To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

Checklist

Tracker (select at least one)
- References tracker ticket
- Very recent bug; references commit where it was introduced
- New feature (ticket optional)
- Doc update (no ticket needed)
- Code cleanup (no ticket needed)
Component impact
- Affects Dashboard, opened tracker ticket
- Affects Orchestrator, opened tracker ticket
- No impact that needs to be tracked
Documentation (select at least one)
- Updates relevant documentation
- No doc update is appropriate
Tests (select at least one)
- Includes unit test(s)
- Includes integration test(s)
- Includes bug reproducer
- No tests

Show available Jenkins commands

jenkins retest this please
jenkins test classic perf
jenkins test crimson perf
jenkins test signed
jenkins test make check
jenkins test make check arm64
jenkins test submodules
jenkins test dashboard
jenkins test dashboard cephadm
jenkins test api
jenkins test docs
jenkins render docs
jenkins test ceph-volume all
jenkins test ceph-volume tox
jenkins test windows

qa/tasks/cephfs/cephfs_test_case.py

rishabh-d-dave · 2023-03-23T08:47:58Z

Improved commit message for commit 6a1283c and also rebased the PR branch.

rishabh-d-dave · 2023-03-28T11:45:02Z

http://pulpito.front.sepia.ceph.com/rishabh-2023-03-28_10:29:50-fs-wip-rishabh-cephman-6-testing-default-smithi
http://pulpito.front.sepia.ceph.com/rishabh-2023-03-28_10:29:50-fs-wip-rishabh-cephman-6-testing-default-smithi

vshankar · 2023-03-29T14:46:48Z

@rishabh-d-dave Wound't it make sense to have a singleton MonManger object in CephFSTestCase and everything else using this singleton object rather than instantiating all over the place?

rishabh-d-dave · 2023-03-29T15:02:33Z

@rishabh-d-dave Wound't it make sense to have a singleton MonManger object in CephFSTestCase and everything else using this singleton object rather than instantiating all over the place?

That will certainly reduce total number of CephManage instances and save some space. I'll make this change as per our discussion we hade verbally.

vshankar · 2023-04-18T08:53:35Z

@rishabh-d-dave Wound't it make sense to have a singleton MonManger object in CephFSTestCase and everything else using this singleton object rather than instantiating all over the place?

That will certainly reduce total number of CephManage instances and save some space. I'll make this change as per our discussion we hade verbally.

@rishabh-d-dave You did reach out to me on IRC saying that this effort is probably not worth doing. Is that still true?

rishabh-d-dave · 2023-04-18T09:16:57Z

@vshankar

@rishabh-d-dave Wound't it make sense to have a singleton MonManger object in CephFSTestCase and everything else using this singleton object rather than instantiating all over the place?

That will certainly reduce total number of CephManage instances and save some space. I'll make this change as per our discussion we hade verbally.

@rishabh-d-dave You did reach out to me on IRC saying that this effort is probably not worth doing. Is that still true?

Yes, I think so now too. A singleton design pattern for class CephManager will save a very tiny amount of space in memory which doesn't matter. We should proceed to test this PR without it.

rishabh-d-dave · 2023-04-18T09:17:19Z

jenkins test api

vshankar · 2023-04-19T02:25:17Z

@vshankar

@rishabh-d-dave Wound't it make sense to have a singleton MonManger object in CephFSTestCase and everything else using this singleton object rather than instantiating all over the place?

That will certainly reduce total number of CephManage instances and save some space. I'll make this change as per our discussion we hade verbally.

@rishabh-d-dave You did reach out to me on IRC saying that this effort is probably not worth doing. Is that still true?

Yes, I think so now too. A singleton design pattern for class CephManager will save a very tiny amount of space in memory which doesn't matter. We should proceed to test this PR without it.

The intent was not about saving memory - the singleton design avoids cluttering various classes and structures with redundant class objects.

Anyway, I won't come in between much on this.

github-actions · 2023-04-20T15:30:25Z

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

vshankar · 2023-04-28T07:19:54Z

OK -- @rishabh-d-dave mentioned that the way forward isn't clear. What I meant was although singleton would be have been better, I'm fine with not implementing that for now.

@rishabh-d-dave Please do the needful to get this merged.

Add run_ceph_cmd(), get_ceph_cmd_stdout() and get_ceph_cmd_result() to class Filesystem so that running Ceph command is easier. This affects not only methods inside class Filesystem but also methods elsewhere that uses instance of class Filesystem to run Ceph commands. Instead of "self.fs.mon_manager.raw_cluster_cmd()" writing "self.fs.run_ceph_cmd()" will suffice. Signed-off-by: Rishabh Dave <ridave@redhat.com>

In filesystem.py and wherever instance of class Filesystem are used, use run_ceph_cmd() instead of get_ceph_cluster_stdout() when output of Ceph command is not required. Signed-off-by: Rishabh Dave <ridave@redhat.com>

rishabh-d-dave · 2023-07-08T16:00:39Z

jenkins test make check

rishabh-d-dave · 2023-07-08T16:00:51Z

jenkins test api

rishabh-d-dave · 2023-07-10T08:17:22Z

jenkins test api

rishabh-d-dave · 2023-07-10T08:17:29Z

jenkins test make check

rishabh-d-dave · 2023-07-10T08:17:48Z

Failure looks unrelated - https://jenkins.ceph.com/job/ceph-pull-requests/117973/

rishabh-d-dave · 2023-07-11T10:45:32Z

https://jenkins.ceph.com/job/ceph-api/57964/

rishabh-d-dave · 2023-07-11T10:45:39Z

jenkins test api

rishabh-d-dave · 2023-07-11T15:40:07Z

Although API tests failed, the failue neither looks to be related to this PR now is it same as before - https://jenkins.ceph.com/job/ceph-api/58053/.

rishabh-d-dave · 2023-07-11T15:40:20Z

jenkins test api

dparmar18 · 2023-07-11T19:10:46Z

Although API tests failed, the failue neither looks to be related to this PR now is it same as before - https://jenkins.ceph.com/job/ceph-api/58053/.

api tests are failing due to this https://tracker.ceph.com/issues/61907

EDIT: Yup, I can see the warnings in the cluster log from the above the api tests link https://jenkins.ceph.com/job/ceph-api/58053/artifact/build/out/cluster.mon.a.log

rishabh-d-dave · 2023-07-12T06:59:41Z

Although API tests failed, the failue neither looks to be related to this PR now is it same as before - https://jenkins.ceph.com/job/ceph-api/58053/.

api tests are failing due to this https://tracker.ceph.com/issues/61907

EDIT: Yup, I can see the warnings in the cluster log from the above the api tests link https://jenkins.ceph.com/job/ceph-api/58053/artifact/build/out/cluster.mon.a.log

Thanks @dparmar18

rishabh-d-dave

Tests were successful - https://tracker.ceph.com/projects/cephfs/wiki/Main#12-July-2023

rishabh-d-dave · 2023-07-12T18:37:22Z

Tests were successful - https://tracker.ceph.com/projects/cephfs/wiki/Main#12-July-2023

@vshankar As discussed previously the PR has been tested and results have been posted. Please approve the PR so that I can be merged. :)

vshankar

LGTM

rishabh-d-dave · 2023-12-15T19:43:00Z

Somewhat related: #52924 and #52042.

Due to ceph#50569 not being backported Signed-off-by: Venky Shankar <vshankar@redhat.com>

rishabh-d-dave added cephfs Ceph File System tests needs-review labels Mar 17, 2023

rishabh-d-dave requested a review from a team March 17, 2023 06:54

github-actions bot added the nfs label Mar 17, 2023

rishabh-d-dave force-pushed the CephManager-in-CephFSTestCase branch 2 times, most recently from 93cbae1 to 8e1000f Compare March 17, 2023 16:53

vshankar reviewed Mar 18, 2023

View reviewed changes

qa/tasks/cephfs/cephfs_test_case.py Outdated Show resolved Hide resolved

rishabh-d-dave force-pushed the CephManager-in-CephFSTestCase branch from 8e1000f to c7dd3b5 Compare March 23, 2023 08:46

rishabh-d-dave force-pushed the CephManager-in-CephFSTestCase branch 4 times, most recently from 3f65179 to df93492 Compare March 28, 2023 08:32

rishabh-d-dave force-pushed the CephManager-in-CephFSTestCase branch from df93492 to 27e9df1 Compare March 29, 2023 12:41

vshankar requested a review from a team April 18, 2023 09:19

github-actions bot added the needs-rebase label Apr 20, 2023

rishabh-d-dave force-pushed the CephManager-in-CephFSTestCase branch from 27e9df1 to 161a1c3 Compare April 27, 2023 12:05

github-actions bot removed the needs-rebase label Apr 27, 2023

rishabh-d-dave force-pushed the CephManager-in-CephFSTestCase branch 2 times, most recently from 3160145 to dd3fcfc Compare April 30, 2023 10:53

rishabh-d-dave added 2 commits June 28, 2023 17:38

qa/cephfs: use run_ceph_cmd() when cmd output is not needed

0a781ef

In filesystem.py and wherever instance of class Filesystem are used, use run_ceph_cmd() instead of get_ceph_cluster_stdout() when output of Ceph command is not required. Signed-off-by: Rishabh Dave <ridave@redhat.com>

rishabh-d-dave force-pushed the CephManager-in-CephFSTestCase branch from 26ee97c to 0a781ef Compare June 28, 2023 12:08

rishabh-d-dave commented Jul 12, 2023

View reviewed changes

vshankar approved these changes Jul 13, 2023

View reviewed changes

rishabh-d-dave added the wip-rishabh-testing Rishabh's testing label label Jul 13, 2023

rishabh-d-dave merged commit f0588bd into ceph:main Jul 13, 2023

rishabh-d-dave deleted the CephManager-in-CephFSTestCase branch July 13, 2023 07:22

adk3798 mentioned this pull request Jul 19, 2023

qa/tasks/test_nfs: have TestNFS inherit from CephFSTestCase #52556

Closed

14 tasks

This was referenced Aug 5, 2023

qa/cephfs: fix test_disallow_monitor_managed_snaps_for_fs_pools #52709

Merged

mon,osd,mds: allow fs authorize to update caps #41779

Merged

vshankar mentioned this pull request Feb 29, 2024

reef: qa: inherit RunCephCmd in CephTestCase instead of CephFSTestCase #55780

Closed

vshankar added a commit to vshankar/ceph that referenced this pull request Feb 29, 2024

reef: use raw_cluster_cmd instead of run_ceph_cmd

0363418

Due to ceph#50569 not being backported Signed-off-by: Venky Shankar <vshankar@redhat.com>

vshankar mentioned this pull request Feb 29, 2024

reef: use raw_cluster_cmd instead of run_ceph_cmd #55836

Merged

14 tasks

vshankar added a commit to vshankar/ceph that referenced this pull request Feb 29, 2024

reef: use raw_cluster_cmd instead of run_ceph_cmd

105e28e

Due to ceph#50569 not being backported Signed-off-by: Venky Shankar <vshankar@redhat.com>

neesingh-rh mentioned this pull request Mar 15, 2024

reef: mgr/volumes: support to reject CephFS clones if cloner threads are not available #55692

Merged

rishabh-d-dave mentioned this pull request Mar 26, 2024

reef: qa/cephfs: add helper methods for running ceph cmd #56485

Closed

14 tasks

Conversation

rishabh-d-dave commented Mar 17, 2023

Contribution Guidelines

Checklist

Uh oh!

Uh oh!

rishabh-d-dave commented Mar 23, 2023

Uh oh!

rishabh-d-dave commented Mar 28, 2023

Uh oh!

vshankar commented Mar 29, 2023

Uh oh!

rishabh-d-dave commented Mar 29, 2023

Uh oh!

vshankar commented Apr 18, 2023

Uh oh!

rishabh-d-dave commented Apr 18, 2023

Uh oh!

rishabh-d-dave commented Apr 18, 2023

Uh oh!

vshankar commented Apr 19, 2023

Uh oh!

github-actions bot commented Apr 20, 2023

Uh oh!

vshankar commented Apr 28, 2023

Uh oh!

rishabh-d-dave commented Jul 8, 2023

Uh oh!

rishabh-d-dave commented Jul 8, 2023

Uh oh!

rishabh-d-dave commented Jul 10, 2023

Uh oh!

rishabh-d-dave commented Jul 10, 2023

Uh oh!

rishabh-d-dave commented Jul 10, 2023

Uh oh!

rishabh-d-dave commented Jul 11, 2023

Uh oh!

rishabh-d-dave commented Jul 11, 2023

Uh oh!

rishabh-d-dave commented Jul 11, 2023

Uh oh!

rishabh-d-dave commented Jul 11, 2023

Uh oh!

dparmar18 commented Jul 11, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rishabh-d-dave commented Jul 12, 2023

Uh oh!

rishabh-d-dave left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

rishabh-d-dave commented Jul 12, 2023 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vshankar left a comment

Choose a reason for hiding this comment

Uh oh!

rishabh-d-dave commented Dec 15, 2023

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dparmar18 commented Jul 11, 2023 •

edited

Loading

rishabh-d-dave left a comment •

edited

Loading

rishabh-d-dave commented Jul 12, 2023 •

edited

Loading