qa/cephfs: improvements in caps_helper#50882
Conversation
afe5cb7 to
9853a7b
Compare
9853a7b to
eeefbb2
Compare
Use Python type list instead of tuple since it get's necessary to modify members of this sequence. Signed-off-by: Rishabh Dave <ridave@redhat.com>
c809357 to
340395f
Compare
changes to caps_helper.CapTester.run_cap_tests(). TO BE SQUASHED when PR ceph#50882 merges. Signed-off-by: Rishabh Dave <ridave@redhat.com>
changes to caps_helper.CapTester.run_cap_tests(). TO BE SQUASHED when PR ceph#50882 merges. Signed-off-by: Rishabh Dave <ridave@redhat.com>
b9b099e to
5ef46dd
Compare
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com>
5ef46dd to
00bea65
Compare
|
jenkins test make check arm64 |
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com>
00bea65 to
78bb795
Compare
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com>
78bb795 to
7d6c03b
Compare
vshankar
left a comment
There was a problem hiding this comment.
Also fix the make check failure:
flake8: commands[0] /home/jenkins-build/build/workspace/ceph-pull-requests/qa> flake8 --select=F,E9 --exclude=venv,.tox
./tasks/cephfs/caps_helper.py:8:1: F401 'tasks.cephfs.cephfs_test_case.CephFSTestCase' imported but unused
qa/tasks/cephfs/caps_helper.py
Outdated
|
|
||
| if len(caps) == 1: | ||
| return _gen_mon_cap_str(caps[0]) | ||
| return _gen_mon_cap_str(*caps[0]) |
There was a problem hiding this comment.
explain a bit in the commit message about the bug.
There was a problem hiding this comment.
I avoided it because this meant explaining a Python concept.
There was a problem hiding this comment.
I've elaborated on the Python concept in the commit message now.
There was a problem hiding this comment.
OK, thx. So, the caps didn't get generated as expected? Didn't that cause a test case failure?
There was a problem hiding this comment.
It does. I've spotted this once in integration test runs.
There was a problem hiding this comment.
This is where the bug was introduced - 969a93d. Here's the teuthology runs - #50497 (comment) And the testing included PR #50212 along with PR #50497.
There was a problem hiding this comment.
This is where the bug was introduced - 969a93d.
I don't think so. This commit uses unpack_tuple which does not have the bug - callers for _gen_mds_cap_str() are:
if len(caps) == 1:
return _gen_mds_cap_str(caps[0])
and
for i, c in enumerate(caps):
mds_cap += _gen_mds_cap_str(c)
if i != len(caps) - 1:
mds_cap += ', '
which seems fine.
Here's the teuthology runs - #50497 (comment) And the testing included PR #41779 along with PR #50497.
Isn't 87025d1 (PR #50664) where the bug got introduced? The test run for this is: http://pulpito.front.sepia.ceph.com/rishabh-2023-04-05_11:02:03-fs:functional-wip-rishabh-fs-auth-subcmd-distro-default-smithi/ - branch: wip-rishabh-fs-auth-subcmd which has the following code:
87025d15858a (Rishabh Dave 2023-03-29 20:48:53 +0530 71) def _gen_mds_cap_str(perm, fsname=None, cephfs_mntpt='/'):
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 72) mds_cap = f'allow {perm}'
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 73) if fsname:
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 74) mds_cap += f' fsname={fsname}'
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 75) if cephfs_mntpt != '/':
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 76) if cephfs_mntpt[0] == '/':
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 77) cephfs_mntpt = cephfs_mntpt[1:]
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 78) mds_cap += f' path={cephfs_mntpt}'
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 79) return mds_cap
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 80)
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 81) if len(caps) == 1:
c8c29aaeb643 (Rishabh Dave 2023-04-07 20:43:46 +0530 82) return _gen_mds_cap_str(*caps[0])
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 83)
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 84) mds_cap = ''
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 85) for i, c in enumerate(caps):
87025d15858a (Rishabh Dave 2023-03-29 20:48:53 +0530 86) mds_cap += _gen_mds_cap_str(*c)
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 87) if i != len(caps) - 1:
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 88) mds_cap += ', '
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 89)
969a93d0dcf1 (Rishabh Dave 2023-03-13 16:16:13 +0530 90) return mds_cap
where:
c8c29aa (Rishabh Dave 2023-04-07 20:43:46 +0530 82) return _gen_mds_cap_str(*caps[0])
is part of https://github.com/ceph/ceph/pull/41779/commits
and"
87025d1 (Rishabh Dave 2023-03-29 20:48:53 +0530 86) mds_cap += _gen_mds_cap_str(*c)
is a part of https://github.com/ceph/ceph/pull/50664/commits
It looks like the above PRs (#41779 and #50664) were included in the test branch, however, only #50664 was merged. @rishabh-d-dave ?
There was a problem hiding this comment.
Isn't 87025d1 (PR #50664) where the bug got introduced? The test run for this is: http://pulpito.front.sepia.ceph.com/rishabh-2023-04-05_11:02:03-fs:functional-wip-rishabh-fs-auth-subcmd-distro-default-smithi/ - branch: wip-rishabh-fs-auth-subcmd which has the following code:
Right. I had forgotten this commit when I replied to you.
where:
c8c29aa (Rishabh Dave 2023-04-07 20:43:46 +0530 82) return _gen_mds_cap_str(*caps[0])
c8c29aa was not present on PR #41779. It was added to only recently when this PR was created.
and"
87025d1 (Rishabh Dave 2023-03-29 20:48:53 +0530 86) mds_cap += _gen_mds_cap_str(*c)
is a part of https://github.com/ceph/ceph/pull/50664/commits
It looks like the above PRs (#41779 and #50664) were included in the test branch, however, only #50664 was merged. @rishabh-d-dave ?
Yes.
There was a problem hiding this comment.
c8c29aa was not present on PR #41779. It was added to only recently when this PR was created.
commit 39ce51a53cf9c8602adc89354214146e6ac505c3 (HEAD -> wip-rishabh-fs-auth-subcmd, rishabh-d-dave/fs-auth-subcmd, ci/wip-rishabh-fs-auth-subcmd)
Author: Rishabh Dave <ridave@redhat.com>
Date: Fri Apr 7 23:51:26 2023 +0530
mon/AuthMonitor: log when parsing caps fails
Signed-off-by: Rishabh Dave <ridave@redhat.com>
commit c8c29aaeb643ca4dbf66ea20b23ed3b81e0bc08f
Author: Rishabh Dave <ridave@redhat.com>
Date: Fri Apr 7 20:43:46 2023 +0530
qa/cephfs/caps_helper: fix a bug in methods that generate cap string
Signed-off-by: Rishabh Dave <ridave@redhat.com>
The commit was a part of the test branch - might not have been in the PR, but surely in the test branch, which is a bit more concerning then (cherry-picked?).
Anyway - please add Introduced-by: <commit> in the commit message.
There was a problem hiding this comment.
The commit was a part of the test branch - might not have been in the PR, but surely in the test branch, which is a bit more concerning then (cherry-picked?).
The testing branch was updated after the testing took place.
The commit has been updated - 64e3dd7.
As per our conversation on IRC I've moved this commit to separate PR - #51127.
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com>
7d6c03b to
90f4878
Compare
Signed-off-by: Rishabh Dave <ridave@redhat.com>
7dac9b6 to
95c6daa
Compare
Class CapTester contains two distinct immiscible group of methods: one that tests MON caps and other that tests MDS caps. When using CapTester for the former reason the instantiation neither needs mount object and the path where files for testing will be created nor it needs to run the method that creates files for testing rw permissions. When using this class for latter the case is the exact opposite. Create 2 separate classes for each of these purpose and class that inherits both of these classes so that instantiating the class becomes as simple as it can be. Signed-off-by: Rishabh Dave <ridave@redhat.com>
|
jenkins test make check arm64 |
|
jenkins test make check |
Looks fine. And the PRs in this test branch are this change and #51127? Just confirmings. |
Yes. |
changes to caps_helper.CapTester.run_cap_tests(). TO BE SQUASHED when PR ceph#50882 merges. Signed-off-by: Rishabh Dave <ridave@redhat.com>
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com>
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com>
qa/cephfs: improvements in caps_helper Reviewed-by: Venky Shankar <vshankar@redhat.com>
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit e8bdf94)
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit e8bdf94)
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit e8bdf94)
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit e8bdf94)
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit e8bdf94)
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit e8bdf94)
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit e8bdf94)
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit e8bdf94)
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit e8bdf94) Conflicts: qa/tasks/cephfs/caps_helper.py - Conflict occured because code on Reef branch changed due to merging of commit 59c9104.
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit e8bdf94) Conflicts: qa/tasks/cephfs/caps_helper.py - Conflict occured because code on Reef branch changed due to merging of commit 59c9104.
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit e8bdf94) Conflicts: qa/tasks/cephfs/caps_helper.py - Conflict occured because code on Reef branch changed due to merging of commit 59c9104. (cherry picked from commit 2f85621) Conflicts: qa/tasks/cephfs/caps_helper.py - CephFSMount.read_file() doesn't accept sudo_read in downstream yet Resolves: rhbz#2149717
Contribution Guidelines
To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
Checklist
Show available Jenkins commands
jenkins retest this pleasejenkins test classic perfjenkins test crimson perfjenkins test signedjenkins test make checkjenkins test make check arm64jenkins test submodulesjenkins test dashboardjenkins test dashboard cephadmjenkins test apijenkins test docsjenkins render docsjenkins test ceph-volume alljenkins test ceph-volume toxjenkins test windows