reef: cephfs: add command "ceph fs swap"#54942
Merged
rishabh-d-dave merged 19 commits intoceph:reeffrom Apr 12, 2024
Merged
Conversation
935e420 to
831e6dc
Compare
|
This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved |
831e6dc to
efa7667
Compare
|
This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved |
efa7667 to
1bc737e
Compare
|
This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved |
Contributor
|
@rishabh-d-dave pls rebase |
Contributor
@rishabh-d-dave ping? |
1bc737e to
1e91e9d
Compare
Contributor
|
jenkins retest this please |
Use Python type list instead of tuple since it get's necessary to modify members of this sequence. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit 1a7ca48)
Inheritting CephFSTestCase in CapTester just for methods assertEqual() and assertIn() from class unittest.TestCase is odd and heavy-weight. Don't inherit CephFSTestCase and use simple assert instead. Reference: ceph#50882 (comment). To avoid code duplication, a couple of similar methods have been added instead. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit e8bdf94) Conflicts: qa/tasks/cephfs/caps_helper.py - Conflict occured because code on Reef branch changed due to merging of commit 59c9104.
test_mutlifs_auth.TestMDSCaps._create_client() not only creates a client but also generate caps strings for the client as per the parameter this method receives and and then writes the keyring to all remote machines. This creates confusion when reading code on test methods in TestMDSCaps. Let's re-arrange this code such that this confusion is avoided. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit 265abc7)
b70fb9e to
02b384d
Compare
02b384d to
9865900
Compare
Improvement #1: CapTester.write_test_files() not only creates the test file but also does the following for every mount object it receives in parameters - * carefully produces the path for the test file as per parameters received * generates the unique data for each test file on a CephFS mount * creates a data structure -- list of lists -- that holds all this information along with mount object itself for each mount object so that tests can be conducted at a later point Untangle this mess of code by splitting this method into 3 separate methods - 1. To produce the path for test file (as per user's need). 2. To generate the data that will be written into the test file. 3. To actually create the test file on CephFS. Improvement ceph#2: Remove the internal data structure used for testing -- self.test_set -- and use separate class attributes to store all the data required for testing instead of a tuple. This serves two purpose - One, it makes it easy to manipulate all this data from helper methods and during debugging session, especially while using a PDB session. And two, make it impossible to have multiple mounts/multiple "test sets" within same CapTester instance for the sake of simplicity. Users can instead create two instances of CapTester instances if needed. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit 008dbe9) Conflicts: qa/tasks/cephfs/caps_helper.py - Conflict occurred because this file went under modification due to commit 59c9104. qa/tasks/cephfs/test_admin.py - Conflict occurred because code region located around the patch has changed.
This method checks if the output of the command "ceph fs ls" for client ID it receives is same as the output printed for client.admin. Don't do so, limit the test to only checking if "ceph fs ls --id client.x -k keyring_file" prints fs name for which client.x has permissions. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit ad68a55)
Move get_mon_cap_from_keyring() and get_fsnmes_from_moncap() from class CapTester to main namespace of caps_helper.py so that they can be imported freely and reused by tests. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit ea9f13e)
Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit 95c6daa)
Class CapTester contains two distinct immiscible group of methods: one that tests MON caps and other that tests MDS caps. When using CapTester for the former reason the instantiation neither needs mount object and the path where files for testing will be created nor it needs to run the method that creates files for testing rw permissions. When using this class for latter the case is the exact opposite. Create 2 separate classes for each of these purpose and class that inherits both of these classes so that instantiating the class becomes as simple as it can be. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit f0ffade) Conflicts: - qa/tasks/cephfs/caps_helper.py: This file went through changes due to merging of commit 59c9104.
Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit be78b3e)
Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit 69e4c9e) Conflicts: * qa/tasks/cephfs/test_admin.py The commit needs an update so that importing "sleep" from "time" instead of importing "time" fully works on Reef branch too.
When assert fails for equality of two variables and when both the variables are printed in error message, print each variable on a new line. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit 6ac58b0)
When two values (say x and y) are being printed because assert for equality of both failed (assert x == y), print both the values on a new line. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit edec8f3)
Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit bd4cb58)
Commands issued by negtest_ceph_cmd() aren't printed because log level (due to code for teuthology) changes from DEBUG to INFO in case of some files. This patch ensures that users can see commands being executed regardless of whether log level is changed or not. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit 03df86b)
Add a FS command that enables users to swap names of two file systems in a single PAXOS transaction. Add an option to this command that swaps FSCIDS along with FS names. This commands also updates the application pool tags and fails when mirroring is enabled on either or both FSs. Fixes: https://tracker.ceph.com/issues/58129 Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit 7170314) Conflicts: doc/man/8/ceph.rst Conflict occurred because Ceph man page contians lesser amout of CephFS commands in it in Reef branch src/mds/FSMap.h * Methods like set_fscid() can be cherry-picked as it is but due to different lines around, patch couldn't be applied as it is. * Methods like get_fscid(), get_mds_map() are absent in Reef branch. They have been kept/added during this conflict resolution.
Create an alias so "APP_NAME_CEPHFS" can be written instead of "pg_pool_t::APPLICATION_NAME_CEPHFS". Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit 4a12f6e) Conflicts: src/mon/FSCommands.cc * Conflict occured because variable in Reef is "fs" and in Main branch is "fsp".
When working with large group tests (18 in this case), it gets very tedious to debug and fix tests when all 18 have to be run again for every mistake. Cheap fix for this to split these 18 tests into several classes. But when modification are made to the feature, all these 18 tests needs to exercised and previous solution forces developer to intitiate all these test classes to run one by one. Best of both worlds can be achieved if we split tests into group but move all these related group to a new file. Signed-off-by: Rishabh Dave <ridave@redhat.com> (cherry picked from commit 9c547ad)
9865900 to
2c4f3f1
Compare
rishabh-d-dave
commented
Apr 12, 2024
Contributor
Author
rishabh-d-dave
left a comment
There was a problem hiding this comment.
QA run was successful - https://tracker.ceph.com/projects/cephfs/wiki/Reef#12-April-2024
Member
|
Follow-up fix: #57373 Unfortunately the test run for the |
Member
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
backport tracker: https://tracker.ceph.com/issues/63834
backport of #50212
parent tracker: https://tracker.ceph.com/issues/58129
Also backports because these commits were necessary to have before PR #50212 could be backported - #50497, #50498, #50665, #50882
this backport was staged using ceph-backport.sh version 16.0.0.6848
find the latest version at https://github.com/ceph/ceph/blob/main/src/script/ceph-backport.sh