Skip to content

rgw/multisite: track shard sync status objects per generation#41538

Closed
yuvalif wants to merge 1 commit intoceph:wip-rgw-multisite-reshardfrom
yuvalif:wip-yuval-per-genration-status2
Closed

rgw/multisite: track shard sync status objects per generation#41538
yuvalif wants to merge 1 commit intoceph:wip-rgw-multisite-reshardfrom
yuvalif:wip-yuval-per-genration-status2

Conversation

@yuvalif
Copy link
Contributor

@yuvalif yuvalif commented May 25, 2021

Signed-off-by: Yuval Lifshitz ylifshit@redhat.com


Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox

@yuvalif yuvalif added the rgw label May 25, 2021
@yuvalif yuvalif requested a review from cbodley May 25, 2021 18:36
}

return new RGWReadBucketPipeSyncStatusCoroutine(&sc, sync_pairs[num], sync_status, nullptr);
return new RGWReadBucketPipeSyncStatusCoroutine(&sc, sync_pairs[num], sync_status, nullptr, full_status.incremental_gen);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i don't think anything is initializing this full_status. the only other place i see it used is for RGWRemoteBucketManager::init_sync_status_cr() which writes a new one with InitBucketFullSyncStatusCR

i think RGWBucketPipeSyncStatusManager::read_sync_status() needs to read this full_status before trying to read any of the shards

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this part of our main code path? is this fix needed?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

its called by radosgw-admin bucket sync markers, which is part of the workflow for manual bilog trimming

if bilogs aren't being trimmed automatically (or fast enough), users may have to trim them manually with radosgw-admin bilog trim - and doing that safely (i.e. not trimming entries that haven't yet been processed by sync) requires the markers from the peer zone

speaking of, we should probably document this stuff, especially now that we've made it more complicated

@github-actions
Copy link

github-actions bot commented Jun 3, 2021

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@cbodley
Copy link
Contributor

cbodley commented Jun 3, 2021

Adam's commits merged in #41051, can you please rebase?

@adamemerson
Copy link
Contributor

This needs a trip to the rebasement.

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
@yuvalif yuvalif force-pushed the wip-yuval-per-genration-status2 branch from 6f74dfd to 74c10f1 Compare June 10, 2021 06:02
@yuvalif
Copy link
Contributor Author

yuvalif commented Jun 10, 2021

@cbodley and @adamemerson,
rebased with latest code. both reshard tests are now passing.

@smanjara
Copy link
Contributor

@cbodley @yuvalif is this ready to be merged? Want to use this pr to fix the radosgw-admin bucket sync status crash.

@yuvalif
Copy link
Contributor Author

yuvalif commented Jun 14, 2021

@cbodley @yuvalif is this ready to be merged? Want to use this pr to fix the radosgw-admin bucket sync status crash.

ready to merge. make check fails on some python linting issue, unrelated to this PR:

./workunits/rgw/test_rgw_reshard.py:10:1: F401 're' imported but unused
./workunits/rgw/test_rgw_reshard.py:12:1: F401 'pprint.pprint' imported but unused

@cbodley
Copy link
Contributor

cbodley commented Jun 14, 2021

merged into #39002

@cbodley cbodley closed this Jun 14, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants