qa: add killpoint testing for dirfrags by batrick · Pull Request #48130 · ceph/ceph

batrick · 2022-09-15T20:55:36Z

Successor to #35178

Contribution Guidelines

To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

Checklist

Tracker (select at least one)
- References tracker ticket
- Very recent bug; references commit where it was introduced
- New feature (ticket optional)
- Doc update (no ticket needed)
- Code cleanup (no ticket needed)
Component impact
- Affects Dashboard, opened tracker ticket
- Affects Orchestrator, opened tracker ticket
- No impact that needs to be tracked
Documentation (select at least one)
- Updates relevant documentation
- No doc update is appropriate
Tests (select at least one)
- Includes unit test(s)
- Includes integration test(s)
- Includes bug reproducer
- No tests

Show available Jenkins commands

jenkins retest this please
jenkins test classic perf
jenkins test crimson perf
jenkins test signed
jenkins test make check
jenkins test make check arm64
jenkins test submodules
jenkins test dashboard
jenkins test dashboard cephadm
jenkins test api
jenkins test docs
jenkins render docs
jenkins test ceph-volume all
jenkins test ceph-volume tox
jenkins test windows

vshankar

@batrick Should commit ba4b329 ("mds: optimize MDBalancer code path config access") be a separate PR? Seems like it could be a backport candidate.

src/mds/MDCache.cc

batrick · 2022-10-24T15:27:42Z

@batrick Should commit ba4b329 ("mds: optimize MDBalancer code path config access") be a separate PR? Seems like it could be a backport candidate.

I don't think the performance aspect is really significant. It was useful for config changes (although IIRC legacy config still tracks changes).

I think we can backport this set to Quincy or even Pacific so no particular reason to split it out IMO.

vshankar · 2022-10-26T06:35:41Z

@batrick Should commit ba4b329 ("mds: optimize MDBalancer code path config access") be a separate PR? Seems like it could be a backport candidate.

I don't think the performance aspect is really significant. It was useful for config changes (although IIRC legacy config still tracks changes).

I think we can backport this set to Quincy or even Pacific so no particular reason to split it out IMO.

Fair enough.

batrick · 2024-06-04T20:20:56Z

This PR is under test in https://tracker.ceph.com/issues/66358.

batrick · 2024-06-05T01:02:45Z

jenkins test dashboard cephadm

batrick · 2024-06-05T01:02:52Z

jenkins test make check

batrick · 2024-06-05T01:02:59Z

jenkins test make check arm64

batrick · 2024-06-05T18:52:50Z

jenkins test dashboard cephadm

batrick · 2024-06-06T17:51:31Z

/teuthology/pdonnell-2024-06-05_18:51:33-fs-wip-pdonnell-testing-20240605.005022-debug-distro-default-smithi/7742414/

Defined a set of killpoints for testing directory fragmentation during cluster recovery after failure. Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com> Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

This change is necessary because the new way of setting config is to use the ceph config command or the asok interface rather than the old way which involved editing the ceph.conf and restarting the daemons to reflect the changes. Have updated the code to support runtime config changes. Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com> Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

These configs may be set using: ceph: cluster-config: entity: foo: bar same as the current: ceph: config: entity: foo: bar The configs will be set in parallel using the `ceph config set` command. The main benefit here is to avoid using the ceph.conf to set configs which cannot be overriden using subsequent `ceph config` command. The only way to override is to change the ceph.conf in the test (yuck) or the admin socket (which gets reset when the daemon restarts). Finally, we can now exploit the `ceph config reset` command will let us trivially rollback config changes after a test completes. That is exposed as the `ctx.config_epoch` variable. Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

So that these configs can be easily overriden by tests. Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

This is faster and catches config changes made outside this interface. Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

Fixes: https://tracker.ceph.com/issues/7320 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com> Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>

batrick · 2024-06-06T17:59:06Z

Minor rebase. Needs approval + merge @vshankar

batrick · 2024-06-06T18:01:21Z

jenkins test dashboard cephadm

batrick · 2024-06-06T18:26:52Z

jenkins test dashboard cephadm

batrick · 2024-06-07T00:47:49Z

jenkins test api

batrick · 2024-06-07T00:47:54Z

jenkins test dashboard cephadm

batrick · 2024-06-07T12:41:54Z

jenkins test dashboard cephadm

batrick · 2024-06-07T12:41:59Z

jenkins test api

vshankar · 2024-06-07T14:21:42Z

jenkins test api

batrick · 2024-06-07T16:39:06Z

jenkins test api

batrick · 2024-06-07T16:39:11Z

jenkins test dashboard cephadm

batrick · 2024-06-07T16:40:54Z

jenkins test api

batrick · 2024-06-08T00:30:59Z

jenkins test api

batrick · 2024-06-08T00:31:04Z

jenkins test dashboard cephadm

batrick added cephfs Ceph File System tests needs-review labels Sep 15, 2022

github-actions bot added the common label Sep 15, 2022

batrick mentioned this pull request Sep 15, 2022

mds: Add killpoints for directory fragmentation #35178

Closed

3 tasks

batrick marked this pull request as ready for review September 15, 2022 22:12

batrick force-pushed the i7320 branch 4 times, most recently from 8b3a707 to 88ff337 Compare September 16, 2022 13:36

batrick force-pushed the i7320 branch from 88ff337 to c01a257 Compare October 3, 2022 15:03

batrick requested a review from a team October 3, 2022 17:04

vshankar reviewed Oct 17, 2022

View reviewed changes

src/mds/MDCache.cc Outdated Show resolved Hide resolved

batrick force-pushed the i7320 branch 3 times, most recently from c2ab198 to cf2fe4d Compare October 24, 2022 20:32

github-actions bot added the needs-rebase label Oct 26, 2022

batrick force-pushed the i7320 branch from cf2fe4d to d7cea97 Compare October 27, 2022 15:06

github-actions bot removed the needs-rebase label Oct 27, 2022

vshankar added wip-vshankar-testing wip-vshankar-testing1 and removed wip-vshankar-testing labels Nov 29, 2022

github-actions bot added the needs-rebase label Dec 22, 2022

vshankar removed the wip-vshankar-testing1 label Dec 28, 2022

batrick force-pushed the i7320 branch from d7cea97 to a51f00f Compare January 3, 2023 17:42

github-actions bot removed the needs-rebase label Jan 3, 2023

batrick force-pushed the i7320 branch from a51f00f to 60bcec6 Compare January 3, 2023 18:53

sidharthanup and others added 9 commits June 6, 2024 13:57

mds: add killpoints for directory fragmentation

8fbefb0

Defined a set of killpoints for testing directory fragmentation during cluster recovery after failure. Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com> Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa/cephfs: set confs using cluster-conf

d7e49c8

So that these configs can be easily overriden by tests. Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa/tasks/ceph_test_case: rollback configs using config reset

3bfb320

This is faster and catches config changes made outside this interface. Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: restore default for config to split exports

82c43d0

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: move some configs to cluster-conf

5601f91

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: stringify arguments to setfattr

94a8113

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>

qa: add killpoint testing for dirfrags

d38dfda

Fixes: https://tracker.ceph.com/issues/7320 Signed-off-by: Patrick Donnelly <pdonnell@redhat.com> Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>

vshankar approved these changes Jun 7, 2024

View reviewed changes

idryomov mentioned this pull request Jul 9, 2024

qa/tasks/ceph: fix "ceph mgr module enable" command #58487

Merged

14 tasks

Conversation

batrick commented Sep 15, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Contribution Guidelines

Checklist

Uh oh!

vshankar left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

batrick commented Oct 24, 2022

Uh oh!

vshankar commented Oct 26, 2022

Uh oh!

batrick commented Jun 4, 2024

Uh oh!

batrick commented Jun 5, 2024

Uh oh!

batrick commented Jun 5, 2024

Uh oh!

batrick commented Jun 5, 2024

Uh oh!

batrick commented Jun 5, 2024

Uh oh!

batrick commented Jun 6, 2024

Uh oh!

batrick commented Jun 6, 2024

Uh oh!

batrick commented Jun 6, 2024

Uh oh!

batrick commented Jun 6, 2024

Uh oh!

batrick commented Jun 7, 2024

Uh oh!

batrick commented Jun 7, 2024

Uh oh!

batrick commented Jun 7, 2024

Uh oh!

batrick commented Jun 7, 2024

Uh oh!

vshankar commented Jun 7, 2024

Uh oh!

batrick commented Jun 7, 2024

Uh oh!

batrick commented Jun 7, 2024

Uh oh!

batrick commented Jun 7, 2024

Uh oh!

batrick commented Jun 8, 2024

Uh oh!

batrick commented Jun 8, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

batrick commented Sep 15, 2022 •

edited

Loading