quincy: rgw: make incomplete multipart upload part of bucket check efficient#57244
quincy: rgw: make incomplete multipart upload part of bucket check efficient#57244ivancich wants to merge 2 commits intoceph:quincyfrom
Conversation
Previously the incomplete multipart portion of bucket check would list all entries in the _multipart_ namespace across all shards and then analyze them in memory before taking further action. Since all index entries for a given multipart upload are all on the same shard by design, we can work on this asynchronously shard by shard. Furthermore since all entries for a given multipart upload are sequential in the bucket index, we can use a small window to analyze each of the uploads. This should make the operation quicker and use much less memory in the worst cases. Signed-off-by: J. Eric Ivancich <ivancich@redhat.com> (cherry picked from commit ebe9893) Conflicts: src/rgw/driver/rados/rgw_bucket.{cc,h} - located in src/rgw - small adjustments due to zipper progress
…m bi As one of the steps in `radosgw-admin bucket check --fix ...` it looks for bucket index entries for incomplete multipart uploads that do not have a corresponding ".meta" entry in the same bucket index. It then intends to delete those entries, however the function that it calls to perform the bucket index deletions was flawed and did not direct the removals to the appropriate shard(s), but instead a non-existant oid. This commit determines the appropriate shard for each of the entries to be removed and asynchronously issues a librados call to omap_rm_keys. Signed-off-by: J. Eric Ivancich <ivancich@redhat.com> (cherry picked from commit 0521c2a) Conflicts: src/rgw/driver/rados/rgw_rados.cc - remove bitx debugging
|
Needed to include one commit from #46030 . |
|
jenkins test api |
|
jenkins test make check |
|
Making this a draft and removing the Quincy milestone, re-add if appropriate. |
|
This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days. |
|
This pull request has been automatically closed because there has been no activity for 90 days. Please feel free to reopen this pull request (or open a new one) if the proposed change is still appropriate. Thank you for your contribution! |
Contribution Guidelines
To sign and title your commits, please refer to Submitting Patches to Ceph.
If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.
When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an
xbetween the brackets:[x]. Spaces and capitalization matter when checking off items this way.Checklist
Show available Jenkins commands
jenkins retest this pleasejenkins test classic perfjenkins test crimson perfjenkins test signedjenkins test make checkjenkins test make check arm64jenkins test submodulesjenkins test dashboardjenkins test dashboard cephadmjenkins test apijenkins test docsjenkins render docsjenkins test ceph-volume alljenkins test ceph-volume toxjenkins test windowsjenkins test rook e2e