Skip to content

qa: postgresql test suite workunit#46821

Merged
vshankar merged 11 commits intoceph:mainfrom
batrick:i55470
Oct 3, 2022
Merged

qa: postgresql test suite workunit#46821
vshankar merged 11 commits intoceph:mainfrom
batrick:i55470

Conversation

@batrick
Copy link
Member

@batrick batrick commented Jun 22, 2022

This PR also adds snap schedules for fs:workload to exercise snapshots during the postgres task. That will also affect other workunits.

Contribution Guidelines

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

@batrick batrick added the cephfs Ceph File System label Jun 22, 2022
@github-actions github-actions bot added the tests label Jun 22, 2022
@batrick batrick marked this pull request as draft June 22, 2022 18:40
@batrick
Copy link
Member Author

batrick commented Jun 22, 2022

Still iterating on tests to reproduce problem in https://tracker.ceph.com/issues/54546

@batrick batrick force-pushed the i55470 branch 4 times, most recently from ffb1d71 to 4652a70 Compare June 23, 2022 15:37
batrick added a commit to batrick/ceph that referenced this pull request Jun 23, 2022
* refs/pull/46821/head:
	qa: add testing for postgresql databases on CephFS
	qa: add snap-schedule task
	qa: reorganize scrub/workunit tasks
	qa: allow specifying mount options for fuse
	qa: unify mountpoint config behavior
	qa: allow check_counter to look at nested keys
	qa: add missing or correct .qa links
@batrick batrick force-pushed the i55470 branch 3 times, most recently from 114eea0 to 17b3f80 Compare June 30, 2022 00:49
@github-actions
Copy link

github-actions bot commented Jul 4, 2022

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@batrick
Copy link
Member Author

batrick commented Aug 11, 2022

jenkins test make check arm64

@batrick batrick force-pushed the i55470 branch 4 times, most recently from aafe1de to f97d4c5 Compare September 9, 2022 14:36
batrick added a commit to batrick/ceph that referenced this pull request Sep 9, 2022
* refs/pull/46821/head:
	qa: add testing for postgresql databases on CephFS
	qa: fs:workload add journal flush task
	qa: add snap-schedule task
	qa: reorganize scrub/workunit tasks
	qa: raise error if fwd_scrub discovers damage
	qa: allow specifying mount options for fuse
	qa: unify mountpoint config behavior
	qa: allow check_counter to look at nested keys
	qa: log mounts file if not mounted
	qa: add missing or correct .qa links
batrick added a commit to batrick/ceph that referenced this pull request Sep 17, 2022
* refs/pull/46821/head:
	qa: add testing for postgresql databases on CephFS
	qa: fs:workload add journal flush task
	qa: add snap-schedule task
	qa: reorganize scrub/workunit tasks
	qa: raise error if fwd_scrub discovers damage
	qa: allow specifying mount options for fuse
	qa: unify mountpoint config behavior
	qa: allow check_counter to look at nested keys
	qa: log mounts file if not mounted
	qa: add missing or correct .qa links
batrick added a commit to batrick/ceph that referenced this pull request Sep 19, 2022
* refs/pull/46821/head:
	qa: add testing for postgresql databases on CephFS
	qa: fs:workload add journal flush task
	qa: add snap-schedule task
	qa: reorganize scrub/workunit tasks
	qa: raise error if fwd_scrub discovers damage
	qa: allow specifying mount options for fuse
	qa: unify mountpoint config behavior
	qa: allow check_counter to look at nested keys
	qa: log mounts file if not mounted
	qa: add missing or correct .qa links
batrick added a commit to batrick/ceph that referenced this pull request Sep 20, 2022
* refs/pull/46821/head:
	qa: add testing for postgresql databases on CephFS
	qa: fs:workload add journal flush task
	qa: add snap-schedule task
	qa: reorganize scrub/workunit tasks
	qa: raise error if fwd_scrub discovers damage
	qa: allow specifying mount options for fuse
	qa: unify mountpoint config behavior
	qa: allow check_counter to look at nested keys
	qa: log mounts file if not mounted
	qa: add missing or correct .qa links
batrick added a commit to batrick/ceph that referenced this pull request Sep 21, 2022
* refs/pull/46821/head:
	qa: add testing for postgresql databases on CephFS
	qa: fs:workload add journal flush task
	qa: add snap-schedule task
	qa: reorganize scrub/workunit tasks
	qa: raise error if fwd_scrub discovers damage
	qa: allow specifying mount options for fuse
	qa: unify mountpoint config behavior
	qa: allow check_counter to look at nested keys
	qa: log mounts file if not mounted
	qa: add missing or correct .qa links
@batrick
Copy link
Member Author

batrick commented Sep 23, 2022

batrick added a commit to batrick/ceph that referenced this pull request Sep 26, 2022
* refs/pull/46821/head:
	qa: add testing for postgresql databases on CephFS
	qa: fs:workload add journal flush task
	qa: add snap-schedule task
	qa: reorganize scrub/workunit tasks
	qa: raise error if fwd_scrub discovers damage
	qa: allow specifying mount options for fuse
	qa: unify mountpoint config behavior
	qa: allow check_counter to look at nested keys
	qa: log mounts file if not mounted
	qa: add missing or correct .qa links
@batrick
Copy link
Member Author

batrick commented Sep 26, 2022

Now shows type of dentry damage (we're looking for "dentry" damage):

https://pulpito.ceph.com/pdonnell-2022-09-26_16:35:43-fs-wip-pdonnell-testing-20220923.171109-distro-default-smithi/7044452/

    find qa/suites/ -type d -execdir ln -sfT ../.qa/ {}/.qa \;

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
For debugging.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Only fuse would support the "mountpoint" yaml config. Move it to
mount.py so it works consistently with the kernel mount. Ditto for the
cephfs mount point.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Also, fix the type default to None. A list does not work in the code but
the default is not used in QA.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
test_client_recovery was also using mntopts to specify additional
options to ceph-fuse. Because the two prior commits unify the behavior
of ceph-fuse and the kernel mount so that the "-o" option is available
for both, that changes breaks this test. Add a special set of args
available only for fuse (there is no equivalent on the kernel).

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
So we can enforce an ordering of tasks and ensure that the fs/subvolume
is available before the task runs.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
To exercise snapshot creation and deletion during workloads.

Fixes: https://tracker.ceph.com/issues/23724
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Idea here is to force the MDS to flush metadata mutations out of the
journal. This may help expose any type of corruption seen with postgres.

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
Fixes: https://tracker.ceph.com/issues/55470
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
@batrick
Copy link
Member Author

batrick commented Sep 27, 2022

@vshankar vshankar merged commit 8981012 into ceph:main Oct 3, 2022
@batrick batrick deleted the i55470 branch October 14, 2022 19:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants