Skip to content

qa/cephfs: upgrade xfstests_dev.py for FUSE mounts#44240

Merged
vshankar merged 1 commit intoceph:mainfrom
rishabh-d-dave:xfstests-dev-ceph-fuse-support
Feb 27, 2023
Merged

qa/cephfs: upgrade xfstests_dev.py for FUSE mounts#44240
vshankar merged 1 commit intoceph:mainfrom
rishabh-d-dave:xfstests-dev-ceph-fuse-support

Conversation

@rishabh-d-dave
Copy link
Contributor

@rishabh-d-dave rishabh-d-dave commented Dec 7, 2021 •
edited
Loading

Add support for testing CephFS mounted with FUSE using xfstests-dev.

Fixes: https://tracker.ceph.com/issues/55354
Related commit from xfstests-dev repo: 1dac95cb0d6e97dfa336deab6d13c81aed531666

Depends on changes from: PR #45036, #45950

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox

@rishabh-d-dave rishabh-d-dave added cephfs Ceph File System DNM tests labels Dec 7, 2021
@rishabh-d-dave rishabh-d-dave requested a review from a team December 7, 2021 14:27
batrick
batrick previously requested changes Jan 6, 2022
View reviewed changes
Copy link
Member

@batrick batrick left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

qa/cephfs: minor fix in xfstests_dev.py is missing signed-off-by.

Otherwise makes sense to me. Please run through teuthology and link to results.

@vshankar vshankar requested a review from a team April 18, 2022 05:49
@vshankar
Copy link
Contributor

vshankar commented Apr 18, 2022

needs tracker.

@rishabh-d-dave rishabh-d-dave force-pushed the xfstests-dev-ceph-fuse-support branch 2 times, most recently from 6f5203c to e550945 Compare April 19, 2022 09:21
@rishabh-d-dave rishabh-d-dave requested a review from batrick April 19, 2022 09:34
@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Apr 19, 2022 •
edited
Loading

DNM because it depends on PR #45036, PR #45950, besides the patch that makes xfstest-dev repo support ceph-fuse as well.

The commit "qa: add a method to get keyring for a client_id " is also present on PR #41779 and the commit originally was located at PR #42335.

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Apr 26, 2022

jenkins test make check

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Apr 26, 2022

jenkins test api

@rishabh-d-dave rishabh-d-dave force-pushed the xfstests-dev-ceph-fuse-support branch 2 times, most recently from 141703c to 0313636 Compare May 8, 2022 17:30
@rishabh-d-dave rishabh-d-dave added the wip-rishabh-testing Rishabh's testing label label May 8, 2022
@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented May 9, 2022

Tests with teuthology ran successfully - http://pulpito.front.sepia.ceph.com/rishabh-2022-05-08_17:30:43-fs-wip-rishabh-fs-auth-subcmd-distro-basic-smithi/

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented May 9, 2022

make check job failure is due to the DNM commit that enables testing this PR with for-next xfstests-dev repo branch.

@rishabh-d-dave rishabh-d-dave removed the wip-rishabh-testing Rishabh's testing label label May 9, 2022
@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented May 10, 2022 •
edited
Loading

The related commit in xfstests-dev repo has been merged in master too now (earlier it was only merged in for-next branch). I re-tested this PR after removing the extra code that ran this PR against that for-next branch.

The tests ran fine - http://pulpito.front.sepia.ceph.com/rishabh-2022-05-09_17:20:03-fs-wip-rishabh-fs-auth-subcmd-distro-basic-smithi/

EDIT: I ran the test with old branch, re-running now.

@rishabh-d-dave rishabh-d-dave force-pushed the xfstests-dev-ceph-fuse-support branch from 0313636 to 8b0e635 Compare May 10, 2022 05:16
@rishabh-d-dave rishabh-d-dave added the wip-rishabh-testing Rishabh's testing label label May 10, 2022
@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented May 10, 2022

The test passed with teuthology (and master branch of xfstests-dev repo) -

2022-05-10T05:39:31.887 INFO:tasks.cephfs.test_acls:client is fuse mounted
2022-05-10T05:39:31.887 INFO:teuthology.orchestra.run:Running command with timeout 30
2022-05-10T05:39:31.888 DEBUG:teuthology.orchestra.run.smithi133:running tests for ACLs from xfstests-dev> (cd /tmp/tmp.yrQBLd1rmZxfstests-dev && exec sudo ./check generic/099)
2022-05-10T05:39:32.034 INFO:teuthology.orchestra.run.smithi133.stdout:FSTYP         -- ceph-fuse
2022-05-10T05:39:32.035 INFO:teuthology.orchestra.run.smithi133.stdout:PLATFORM      -- Linux/x86_64 smithi133 4.18.0-348.20.1.el8_5.x86_64 #1 SMP Tue Mar 8 12:56:54 EST 2022
2022-05-10T05:39:32.035 INFO:teuthology.orchestra.run.smithi133.stdout:
2022-05-10T05:39:32.313 INFO:teuthology.orchestra.run.smithi133.stdout:generic/099	 0s
2022-05-10T05:39:32.317 INFO:teuthology.orchestra.run.smithi133.stdout:Ran: generic/099
2022-05-10T05:39:32.318 INFO:teuthology.orchestra.run.smithi133.stdout:Passed all 1 tests
2022-05-10T05:39:32.319 INFO:teuthology.orchestra.run.smithi133.stdout:
2022-05-10T05:39:32.337 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi133.stderr:ceph-fuse[101298]: fuse finished with error 0 and tester_r 0
2022-05-10T05:39:32.344 INFO:tasks.cephfs.test_acls:Command return value: 0

  suite_branch: xfstests-dev-ceph-fuse-support
  suite_path: /home/teuthworker/src/github.com_rishabh-d-dave_ceph_8b0e635bdf546c1eaa1b9ee9e40884270f0df118/qa
  suite_relpath: qa
  suite_repo: https://github.com/rishabh-d-dave/ceph
  suite_sha1: 8b0e635bdf546c1eaa1b9ee9e40884270f0df118

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented May 10, 2022 •
edited
Loading

This PR's ready for merge, it's only waiting for PRs #45036 and PR #45950 to merge now.

@rishabh-d-dave rishabh-d-dave removed the wip-rishabh-testing Rishabh's testing label label May 10, 2022
@github-actions
Copy link

github-actions bot commented Jul 20, 2022

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Jul 21, 2022

jenkins test windows

@github-actions
Copy link

github-actions bot commented Aug 19, 2022

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

1 similar comment
@github-actions
Copy link

github-actions bot commented Dec 6, 2022

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Dec 13, 2022

jenkins test make check

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Dec 13, 2022

jenkins test api

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Dec 14, 2022

Tests ran fine with teuthology - http://pulpito.front.sepia.ceph.com/rishabh-2022-12-13_22:07:07-fs:functional-wip-rishabh-xfstests3-distro-default-smithi

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Dec 14, 2022

CI job "ceph API tests" failed for an unrelated reason -

rm -rf -- /home/jenkins-build/build/workspace/ceph-api/build/dev/mon.c 
mkdir -p /home/jenkins-build/build/workspace/ceph-api/build/dev/mon.c 
/home/jenkins-build/build/workspace/ceph-api/build/bin/ceph-mon --mkfs -c /home/jenkins-build/build/workspace/ceph-api/build/ceph.conf -i c --monmap=/tmp/ceph_monmap.210800 --keyring=/home/jenkins-build/build/workspace/ceph-api/build/keyring 
rm -- /tmp/ceph_monmap.210800 
/home/jenkins-build/build/workspace/ceph-api/build/bin/ceph-mon -i a -c /home/jenkins-build/build/workspace/ceph-api/build/ceph.conf 
2022-12-13T20:38:30.988+0000 7f1f4c9fcc80 -1  Processor -- bind unable to bind to v2:172.21.2.9:40151/0: (98) Address already in use
2022-12-13T20:38:30.988+0000 7f1f4c9fcc80 -1  Processor -- bind was unable to bind. Trying again in 5 seconds 
2022-12-13T20:38:35.988+0000 7f1f4c9fcc80 -1  Processor -- bind unable to bind to v2:172.21.2.9:40151/0: (98) Address already in use
2022-12-13T20:38:35.988+0000 7f1f4c9fcc80 -1  Processor -- bind was unable to bind. Trying again in 5 seconds 
2022-12-13T20:38:40.988+0000 7f1f4c9fcc80 -1  Processor -- bind unable to bind to v2:172.21.2.9:40151/0: (98) Address already in use
2022-12-13T20:38:40.988+0000 7f1f4c9fcc80 -1  Processor -- bind was unable to bind after 3 attempts: (98) Address already in use
2022-12-13T20:38:40.988+0000 7f1f4c9fcc80 -1 unable to bind monitor to [v2:172.21.2.9:40151/0,v1:172.21.2.9:40152/0]
Using guessed paths /home/jenkins-build/build/workspace/ceph-api/build/lib/ ['/home/jenkins-build/build/workspace/ceph-api/qa', '/home/jenkins-build/build/workspace/ceph-api/build/lib/cython_modules/lib.3', '/home/jenkins-build/build/workspace/ceph-api/src/pybind']
Traceback (most recent call last):
  File "/home/jenkins-build/build/workspace/ceph-api/build/../qa/tasks/vstart_runner.py", line 1510, in <module>
    exec_test()
  File "/home/jenkins-build/build/workspace/ceph-api/build/../qa/tasks/vstart_runner.py", line 1366, in exec_test
    remote.run(args=args, env=vstart_env, timeout=(3 * 60))
  File "/home/jenkins-build/build/workspace/ceph-api/build/../qa/tasks/vstart_runner.py", line 417, in run
    return self._do_run(**kwargs)
  File "/home/jenkins-build/build/workspace/ceph-api/build/../qa/tasks/vstart_runner.py", line 456, in _do_run
    proc.wait()
  File "/home/jenkins-build/build/workspace/ceph-api/build/../qa/tasks/vstart_runner.py", line 258, in wait
    raise CommandFailedError(self.args, self.exitstatus)
teuthology.exceptions.CommandFailedError: Command failed with status 1: '../src/vstart.sh -n --nolockdep'


TEST FAILED

https://jenkins.ceph.com/job/ceph-api/48618/https://jenkins.ceph.com/job/ceph-api/48618

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Dec 14, 2022

jenkins test api

vshankar
vshankar reviewed Dec 14, 2022
View reviewed changes
qa/tasks/cephfs/xfstests_dev.py
# generate local.config for FUSE cephfs mounts
def _gen_conf_for_fuse_mnt(self):
mon_sock = self.fs.mon_manager.get_msgrv1_mon_socks()[0]
test_dev = 'ceph-fuse'
Copy link
Contributor

@vshankar vshankar Dec 14, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@rishabh-d-dave I think @lxbsz is correct. setting test_dev to ceph-fuse is confusing.

vshankar
vshankar requested changes Dec 16, 2022
View reviewed changes
Copy link
Contributor

@vshankar vshankar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Mostly looks fine.

@vshankar
Copy link
Contributor

vshankar commented Dec 16, 2022

Waiting for @rishabh-d-dave to confirm about using TEST_DEV.

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Jan 31, 2023

Rebased the PR branch to latest main.

@github-actions
Copy link

github-actions bot commented Feb 22, 2023

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Feb 23, 2023

Rebased and resolved the conflict.

@github-actions
Copy link

github-actions bot commented Feb 23, 2023

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Feb 23, 2023

Rebased and resolved the conflict again. No other changes.

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Feb 23, 2023

@vshankar

Waiting for @rishabh-d-dave to confirm about using TEST_DEV.

I tried running test_acls.py without setting TEST_DEV and with setting TEST_DEV to something other than ceph-fuse. In both the cases, the command ./check generic/099 fails. Thus, TEST_DEV=ceph-fuse is must in xfstests-dev's local.confing. I hope I am answering the question.

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Feb 23, 2023 •
edited
Loading

About last push: added a minor fix.

@vshankar
Copy link
Contributor

vshankar commented Feb 24, 2023

@vshankar

Waiting for @rishabh-d-dave to confirm about using TEST_DEV.

I tried running test_acls.py without setting TEST_DEV and with setting TEST_DEV to something other than ceph-fuse. In both the cases, the command ./check generic/099 fails. Thus, TEST_DEV=ceph-fuse is must in xfstests-dev's local.confing. I hope I am answering the question.

ACK. Please include a comment since its not obvious.

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Feb 24, 2023

@vshankar

I tried running test_acls.py without setting TEST_DEV and with setting TEST_DEV to something other than ceph-fuse. In both the cases, the command ./check generic/099 fails. Thus, TEST_DEV=ceph-fuse is must in xfstests-dev's local.confing. I hope I am answering the question.

ACK. Please include a comment since its not obvious.

Done - https://github.com/ceph/ceph/compare/699a61938f9c78675ebfdf5295df4fab25519222..0d4ffbf57a75a03e54ad7b27bf3b423ff50ff90c.

@vshankar
Copy link
Contributor

vshankar commented Feb 24, 2023

jenkins test make check

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Feb 24, 2023

jenkins test make check

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Feb 24, 2023

Tests ran successfully -
http://pulpito.front.sepia.ceph.com/rishabh-2023-02-24_05:21:23-fs-wip-rishabh-fs-qa-workunit-quota.sh-4-distro-default-smithi/detail
http://pulpito.front.sepia.ceph.com/rishabh-2023-02-24_05:22:31-fs-wip-rishabh-fs-qa-workunit-quota.sh-4-distro-default-smithi/detail

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Feb 24, 2023

make check failure isn't related - https://jenkins.ceph.com/job/ceph-pull-requests/111225/

@rishabh-d-dave
Copy link
Contributor Author

rishabh-d-dave commented Feb 24, 2023

jenkins test make check

@rishabh-d-dave
qa/cephfs: upgrade xfstests_dev.py for FUSE mounts
b88e917 
Add support for testing CephFS mounted with FUSE using xfstests-dev.

Fixes: https://tracker.ceph.com/issues/55354
Signed-off-by: Rishabh Dave <ridave@redhat.com>
@vshankar vshankar mentioned this pull request Feb 27, 2023
Merged
14 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vshankar vshankar vshankar approved these changes

@lxbsz lxbsz lxbsz approved these changes

@batrick batrick Awaiting requested review from batrick

@jtlayton jtlayton Awaiting requested review from jtlayton

@luis-henrix luis-henrix Awaiting requested review from luis-henrix

Assignees

@vshankar vshankar

Labels

cephfs Ceph File System tests wip-rishabh-testing Rishabh's testing label

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@rishabh-d-dave @vshankar @batrick @lxbsz