Skip to content

mds: Add killpoints for directory fragmentation#35178

Closed
sidharthanup wants to merge 3 commits intoceph:mainfrom
sidharthanup:wip-dirfrag-killpoints
Closed

mds: Add killpoints for directory fragmentation#35178
sidharthanup wants to merge 3 commits intoceph:mainfrom
sidharthanup:wip-dirfrag-killpoints

Conversation

@sidharthanup
Copy link
Contributor

@sidharthanup sidharthanup commented May 21, 2020 •
edited
Loading

Defined Killpoints for testing completion/recovery of directory fragmentation during failure.

Fixes: https://tracker.ceph.com/issues/7320
Signed-off-by: Sidharth Anupkrishnan sanupkri@redhat.com

Checklist

  • References tracker ticket
  • Updates documentation if necessary
  • Includes tests for new functionality or reproducer for bug
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard backend
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox

@sidharthanup sidharthanup force-pushed the wip-dirfrag-killpoints branch 2 times, most recently from f6dfcb1 to f6e9cab Compare May 26, 2020 11:32
@sidharthanup sidharthanup added the cephfs Ceph File System label May 26, 2020
@sidharthanup sidharthanup force-pushed the wip-dirfrag-killpoints branch from f6e9cab to 8f3af60 Compare May 26, 2020 12:16
batrick
batrick requested changes May 29, 2020
View reviewed changes
@sidharthanup sidharthanup force-pushed the wip-dirfrag-killpoints branch from 8f3af60 to e3765b1 Compare June 8, 2020 13:00
@sidharthanup sidharthanup force-pushed the wip-dirfrag-killpoints branch 3 times, most recently from 35039f2 to ea91b44 Compare June 25, 2020 17:47
@batrick
Copy link
Member

batrick commented Jul 8, 2020

retest this please

@batrick
Copy link
Member

batrick commented Jul 31, 2020

@sidharthanup please rebase.

Any update on this? Ready for another round of review?

@sidharthanup sidharthanup force-pushed the wip-dirfrag-killpoints branch 2 times, most recently from 3ba10ff to a7a58a4 Compare August 6, 2020 13:14
@sidharthanup
Copy link
Contributor Author

sidharthanup commented Aug 6, 2020

@batrick Just cleaned it up and pushed an updated set of killpoints which I think is more appropriate. Please do review.

@sidharthanup sidharthanup force-pushed the wip-dirfrag-killpoints branch 5 times, most recently from 3a62cbe to 5a229b8 Compare August 10, 2020 15:26
@sidharthanup
Copy link
Contributor Author

sidharthanup commented Aug 10, 2020

@batrick https://pulpito.ceph.com/sidharthanup-2020-08-10_15:27:17-fs-wip-dirfrag-killpoints-distro-basic-smithi/ . Tests pass. I don't think I see anything funny happening. Please take a look.

@batrick
Copy link
Member

batrick commented Aug 11, 2020

jenkins test make check

batrick
batrick requested changes Aug 11, 2020
View reviewed changes
batrick
batrick requested changes Aug 11, 2020
View reviewed changes
@sidharthanup sidharthanup force-pushed the wip-dirfrag-killpoints branch 2 times, most recently from a7917bb to 29e8119 Compare August 13, 2020 16:57
batrick
batrick requested changes Aug 13, 2020
View reviewed changes
@ukernel
Copy link
Contributor

ukernel commented Aug 17, 2020 •
edited
Loading

add kill points to MDCache::_fragment_logged, MDCache::_fragment_committed and MDCache::_fragment_old_purged

@sidharthanup
Copy link
Contributor Author

sidharthanup commented Aug 17, 2020

@ukernel Thanks! I've started testing it locally. Will push those changes.

@sidharthanup sidharthanup force-pushed the wip-dirfrag-killpoints branch from 29e8119 to 7702415 Compare August 18, 2020 12:31
@batrick
Copy link
Member

batrick commented Dec 9, 2020

@sidharthanup please run through QA again when finished.

@sidharthanup sidharthanup force-pushed the wip-dirfrag-killpoints branch 2 times, most recently from 16bdb75 to cdf2d2e Compare December 9, 2020 21:32
@sidharthanup
Copy link
Contributor Author

sidharthanup commented Dec 9, 2020

Ack I forgot to push some of the latest fixes.

@sidharthanup
Copy link
Contributor Author

sidharthanup commented Dec 9, 2020

jenkins test make check

@sidharthanup sidharthanup force-pushed the wip-dirfrag-killpoints branch from cdf2d2e to e893b18 Compare December 9, 2020 21:46
@sidharthanup
Copy link
Contributor Author

sidharthanup commented Dec 17, 2020

@batrick Pls review again. https://pulpito.ceph.com/sidharthanup-2020-12-17_14:01:08-fs-wip-dirfrag-killpoints17-distro-basic-smithi/ seems to be hitting the same assertion failure as before.

@batrick
Copy link
Member

batrick commented Dec 19, 2020

@batrick Pls review again. https://pulpito.ceph.com/sidharthanup-2020-12-17_14:01:08-fs-wip-dirfrag-killpoints17-distro-basic-smithi/ seems to be hitting the same assertion failure as before.

It looks like the begin.yaml changes were lost?

#35178 (comment)

Please let me know what you fixed/changed.

@sidharthanup sidharthanup force-pushed the wip-dirfrag-killpoints branch from e893b18 to d4cfc79 Compare December 21, 2020 12:40
@sidharthanup
Copy link
Contributor Author

sidharthanup commented Dec 21, 2020 •
edited
Loading

Uh oh.. Not sure why the begin.yaml update and the frag_enable deletes got lost. Have reupdated. Waiting for teuthology.

@sidharthanup sidharthanup force-pushed the wip-dirfrag-killpoints branch 2 times, most recently from 0a77f96 to 2e9d4cc Compare December 22, 2020 21:24
@batrick
Copy link
Member

batrick commented Dec 23, 2020

jenkins test make check

@github-actions
Copy link

github-actions bot commented Dec 24, 2020

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@sidharthanup sidharthanup force-pushed the wip-dirfrag-killpoints branch 2 times, most recently from 87c598f to 8792a94 Compare January 4, 2021 10:23
@github-actions
Copy link

github-actions bot commented Jan 16, 2021

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@batrick
Copy link
Member

batrick commented May 4, 2021

Please rebase

sidharthanup
sidharthanup commented May 5, 2021
View reviewed changes
@sidharthanup
mds: Add killpoints for directory fragmentation
fe1e591 
Defined a set of killpoints for testing directory fragmentation during cluster recovery after failure.
Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
@sidharthanup
mds: Optimize Balancer code path to use newer config fetch and also c…
d072711 
…ache the fetched values when there is a hot code path

This change is necessary because the new way of setting config is to use the ceph config command or the asok interfave rahter than the old way which involved editing the ceph.conf and restarting the daemons to reflect the changes. Have updated the code to support runtime config changes.
Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
@sidharthanup
qa: Add testing of the dirfrag killpoints
41730f2 
Signed-off-by: Sidharth Anupkrishnan <sanupkri@redhat.com>
@github-actions
Copy link

github-actions bot commented Jun 23, 2021

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@stale
Copy link

stale bot commented Jan 9, 2022

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

1 similar comment
@github-actions
Copy link

github-actions bot commented Sep 13, 2022

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@batrick batrick mentioned this pull request Sep 15, 2022
Merged
14 tasks
@batrick
Copy link
Member

batrick commented Sep 15, 2022

Now: #48130

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@batrick batrick batrick requested changes

Assignees

@sidharthanup sidharthanup

Labels

cephfs Ceph File System common needs-rebase stale tests

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@sidharthanup @batrick @ukernel