Skip to content

qa: restrict 'perf dump' on active mds only#59383

Merged
mchangir merged 1 commit intoceph:mainfrom
joscollin:wip-B67360-counters-set-on-mds-failed
Dec 6, 2024
Merged

qa: restrict 'perf dump' on active mds only#59383
mchangir merged 1 commit intoceph:mainfrom
joscollin:wip-B67360-counters-set-on-mds-failed

Conversation

@joscollin
Copy link
Member

@joscollin joscollin commented Aug 21, 2024 •
edited
Loading

Fixes: https://tracker.ceph.com/issues/67360

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows
  • jenkins test rook e2e

@joscollin joscollin marked this pull request as draft August 21, 2024 15:15
@github-actions github-actions bot added the tests label Aug 21, 2024
@joscollin
Copy link
Member Author

joscollin commented Aug 22, 2024

jenkins test make check

@joscollin
Copy link
Member Author

joscollin commented Aug 22, 2024

jenkins test make check arm64

@joscollin
Copy link
Member Author

joscollin commented Aug 23, 2024

jenkins test make check

@joscollin
Copy link
Member Author

joscollin commented Aug 25, 2024

jenkins test make check arm64

@joscollin joscollin force-pushed the wip-B67360-counters-set-on-mds-failed branch from 9292e38 to b1e7b54 Compare August 27, 2024 06:06
@joscollin
Copy link
Member Author

joscollin commented Aug 27, 2024

jenkins test make check

@joscollin joscollin marked this pull request as ready for review August 27, 2024 09:18
@joscollin joscollin added the cephfs Ceph File System label Aug 27, 2024
@joscollin joscollin changed the title qa: restrict 'perf dump' on active mds qa: restrict 'perf dump' on active mdses only Aug 27, 2024
@joscollin joscollin changed the title qa: restrict 'perf dump' on active mdses only qa: restrict 'perf dump' on active mds only Aug 27, 2024
@joscollin
Copy link
Member Author

joscollin commented Aug 27, 2024

https://pulpito.ceph.com/jcollin-2024-08-26_12:35:33-fs:workload-wip-mchangir-uninline-debug-distro-default-smithi/

@joscollin
Copy link
Member Author

joscollin commented Aug 27, 2024

jenkins test make check

batrick
batrick requested changes Aug 27, 2024
View reviewed changes
@vshankar vshankar requested a review from a team August 28, 2024 05:48
@joscollin joscollin force-pushed the wip-B67360-counters-set-on-mds-failed branch from b1e7b54 to 3d7f384 Compare August 28, 2024 11:40
@vshankar
Copy link
Contributor

vshankar commented Aug 28, 2024

This change looks fine when check-counter the MDS states are stable, i.e., when there aren't frequent transitions b/w active to standby, etc.. But check-counters is also included in fs:thrash.

@joscollin
Copy link
Member Author

joscollin commented Aug 28, 2024

This change looks fine when check-counter the MDS states are stable, i.e., when there aren't frequent transitions b/w active to standby, etc.. But check-counters is also included in fs:thrash.

@batrick Then I should check the status each time in the loop to cover this^.

@vshankar
Copy link
Contributor

vshankar commented Aug 28, 2024

This change looks fine when check-counter the MDS states are stable, i.e., when there aren't frequent transitions b/w active to standby, etc.. But check-counters is also included in fs:thrash.

@batrick Then I should check the status each time in the loop to cover this^.

That just narrows the race windows IMO.

@batrick
Copy link
Member

batrick commented Aug 28, 2024

This change looks fine when check-counter the MDS states are stable, i.e., when there aren't frequent transitions b/w active to standby, etc.. But check-counters is also included in fs:thrash.

@batrick Then I should check the status each time in the loop to cover this^.

That just narrows the race windows IMO.

If that's a concern, then use mds.<gid> with the ceph tell command so that the command will fail if there is a failover since the fs dump was fetched.

@joscollin joscollin force-pushed the wip-B67360-counters-set-on-mds-failed branch 3 times, most recently from 5b9aa7e to 1c74c08 Compare August 29, 2024 12:02
@joscollin
Copy link
Member Author

joscollin commented Aug 29, 2024

Test passed: https://pulpito.ceph.com/jcollin-2024-08-29_10:48:29-fs:workload-wip-mchangir-uninline-debug-distro-default-smithi/

vshankar
vshankar reviewed Aug 29, 2024
View reviewed changes
@joscollin
Copy link
Member Author

joscollin commented Aug 30, 2024

jenkins test make check

@joscollin joscollin force-pushed the wip-B67360-counters-set-on-mds-failed branch from 1c74c08 to c096967 Compare September 2, 2024 03:21
vshankar
vshankar requested changes Sep 2, 2024
View reviewed changes
Copy link
Contributor

@vshankar vshankar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://github.com/ceph/ceph/pull/59383/files#r1740357995

@joscollin
Copy link
Member Author

joscollin commented Sep 10, 2024

jenkins test windows

@joscollin
Copy link
Member Author

joscollin commented Sep 10, 2024

jenkins test make check arm64

@joscollin
qa: restrict 'perf dump' on active mds
7a952dc 
Fixes: https://tracker.ceph.com/issues/67360
Signed-off-by: Jos Collin <jcollin@redhat.com>
@joscollin joscollin force-pushed the wip-B67360-counters-set-on-mds-failed branch from c096967 to 7a952dc Compare September 12, 2024 03:59
@joscollin joscollin requested a review from vshankar September 16, 2024 05:06
@batrick
Copy link
Member

batrick commented Oct 9, 2024

jenkins test make check arm64

@batrick
Copy link
Member

batrick commented Oct 15, 2024

jenkins test make check arm64

@vshankar
Copy link
Contributor

vshankar commented Oct 16, 2024

This PR is under test in https://tracker.ceph.com/issues/68578.

vshankar
vshankar reviewed Oct 25, 2024
View reviewed changes
@mchangir mchangir merged commit 7fdc8a7 into ceph:main Dec 6, 2024
@joscollin joscollin deleted the wip-B67360-counters-set-on-mds-failed branch December 6, 2024 07:41
This was referenced Dec 6, 2024
Open
Merged
@vshankar
Copy link
Contributor

vshankar commented Dec 16, 2024

Thx for merging @mchangir

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vshankar vshankar vshankar approved these changes

@batrick batrick batrick approved these changes

Assignees

No one assigned

Labels

cephfs Ceph File System needs-qa ready-to-merge tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@joscollin @vshankar @batrick @mchangir