Skip to content

rgw: Generational BILog Trim#41051

Closed
adamemerson wants to merge 9 commits intoceph:wip-rgw-multisite-reshardfrom
adamemerson:wip-multisite-reshard-trim
Closed

rgw: Generational BILog Trim#41051
adamemerson wants to merge 9 commits intoceph:wip-rgw-multisite-reshardfrom
adamemerson:wip-multisite-reshard-trim

Conversation

@adamemerson
Copy link
Contributor

@adamemerson adamemerson commented Apr 28, 2021

Generational BILog trim.

@adamemerson adamemerson requested a review from cbodley April 28, 2021 01:05
@mattbenjamin mattbenjamin mentioned this pull request Apr 28, 2021
Merged
31 tasks
mattbenjamin
mattbenjamin reviewed Apr 28, 2021
View reviewed changes
@cbodley
Copy link
Contributor

cbodley commented Apr 29, 2021

thanks @adamemerson, but this logic needs to be integrated with the background trim logic in rgw_trim_bilog.cc rather than the admin api/command

when the background trimming thread identifies a bucket to trim, it spawns BucketTrimInstanceCR to query the peer zone's markers, then uses RGWRadosBILogTrimCR to issue the rados cls calls

we need to teach BucketTrimInstanceCR about generations. if, for example, we have logs for gens 0,1,2 but all of the peers are on gen=2, we can delete the logs for 0 and 1 and update the bucket instance's layout accordingly

however, this trimming logic is careful to throttle how much work it does on each bucket, because too many omap deletes can melt osds. so we probably want to limit BucketTrimInstanceCR to a single gen deletion per call?

@adamemerson adamemerson force-pushed the wip-rgw-multisite-reshard branch from 36c2959 to 95355ab Compare May 12, 2021 16:05
@github-actions
Copy link

github-actions bot commented May 12, 2021

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@adamemerson adamemerson force-pushed the wip-multisite-reshard-trim branch from c0d6675 to 2ff7e2d Compare May 12, 2021 20:17
@adamemerson adamemerson force-pushed the wip-multisite-reshard-trim branch 2 times, most recently from 6cbcd02 to cd4c8c7 Compare May 14, 2021 23:05
@adamemerson
Copy link
Contributor Author

adamemerson commented May 14, 2021

@cbodley re²view if you would, please.

@cbodley cbodley changed the title Generational BILog Trim rgw: Generational BILog Trim May 17, 2021
cbodley
cbodley reviewed May 17, 2021
View reviewed changes
@smanjara smanjara self-requested a review May 18, 2021 05:58
@adamemerson adamemerson force-pushed the wip-multisite-reshard-trim branch from cd4c8c7 to 0e08a8f Compare May 18, 2021 22:40
@adamemerson
Copy link
Contributor Author

adamemerson commented May 18, 2021

@cbodley Please re³view.

@ceph ceph deleted a comment from github-actions bot May 18, 2021
@adamemerson
Copy link
Contributor Author

adamemerson commented May 19, 2021

There's one test failure, but it looks to be an issue with BILog_List in radosgw-admin

cbodley
cbodley reviewed May 24, 2021
View reviewed changes
@cbodley cbodley mentioned this pull request May 25, 2021
Closed
yuvalif
yuvalif reviewed May 26, 2021
View reviewed changes
@adamemerson
rgw: add sync_info to BILog_Status output
5c668f3 
Needed so we can get the incremental generation.

Guard this behind a version check and return the original output if
less than 2.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
rgw: Add cast from bucket_index_log_layout
dee330e 
To bucket_index_layout_generation

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
rgw: BucketInstanceTrimCR knows about generations
a545b9f 
Fetch the current generation from remote peers and trim the minimum
marker on the minimum generation.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson adamemerson mentioned this pull request May 26, 2021
Closed
@adamemerson
common: get_str_vec takes std::string_view
e731450 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
rgw: Bilog trim takes markers as string view
08082f0 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
rgw: Trim bilog with generation
b25e694 
From the REST interface and radosgw-admin. Assume Generation 0 if none
provided and error if it doesn't exist.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
rgw: Add RGWAsyncPutBucketInstanceInfoCR
7552d1c 
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson
rgw: Add RGWRadosRemoveOidCR
4a07dac 
A more generally applicable way of removing objects in coroutines.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson adamemerson force-pushed the wip-multisite-reshard-trim branch from 0e08a8f to ff84465 Compare May 26, 2021 22:03
@adamemerson
Copy link
Contributor Author

adamemerson commented May 26, 2021

@cbodley Please re⁴view.

@adamemerson adamemerson force-pushed the wip-multisite-reshard-trim branch from ff84465 to c96c51c Compare May 27, 2021 15:59
cbodley
cbodley reviewed Jun 1, 2021
View reviewed changes
@adamemerson
rgw: Trim old generations in BucketTrimInstanceCR
ea950a9 
Only one generation per call.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
@adamemerson adamemerson force-pushed the wip-multisite-reshard-trim branch from c96c51c to ea950a9 Compare June 2, 2021 18:03
@adamemerson
Copy link
Contributor Author

adamemerson commented Jun 2, 2021

@cbodley Okay, let's see if we're good.

cbodley
cbodley approved these changes Jun 2, 2021
View reviewed changes
Copy link
Contributor

@cbodley cbodley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good! let's start thinking about how to get some test coverage, especially for the deletion of old generations and updates to the bucket info

@cbodley
Copy link
Contributor

cbodley commented Jun 3, 2021

rebased and merged into #39002, thanks @adamemerson!

@cbodley cbodley closed this Jun 3, 2021
@adamemerson adamemerson deleted the wip-multisite-reshard-trim branch September 15, 2022 21:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mattbenjamin mattbenjamin mattbenjamin left review comments

@yuvalif yuvalif yuvalif left review comments

@cbodley cbodley cbodley approved these changes

@smanjara smanjara Awaiting requested review from smanjara

Assignees

No one assigned

Labels

common rgw

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@adamemerson @cbodley @mattbenjamin @yuvalif