Skip to content

src/test: Addition of reshared bucket disable-enable test#44838

Closed
TRYTOBE8TME wants to merge 159 commits intoceph:wip-rgw-multisite-reshardfrom
TRYTOBE8TME:wip-rgw-enable-disable-reshard-test-add
Closed

src/test: Addition of reshared bucket disable-enable test#44838
TRYTOBE8TME wants to merge 159 commits intoceph:wip-rgw-multisite-reshardfrom
TRYTOBE8TME:wip-rgw-enable-disable-reshard-test-add

Conversation

@TRYTOBE8TME
Copy link

Signed-off-by: Kalpesh Pandya kapandya@redhat.com

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox

Copy link
Contributor

@cbodley cbodley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks great, is it passing for you?

@TRYTOBE8TME
Copy link
Author

looks great, is it passing for you?

Yeah, it's passing locally but I need to check it on teuthology

@TRYTOBE8TME TRYTOBE8TME force-pushed the wip-rgw-enable-disable-reshard-test-add branch from ed8ae85 to 74b6ac6 Compare February 1, 2022 05:49
@adamemerson adamemerson force-pushed the wip-rgw-multisite-reshard branch from b6342d0 to 8800bf7 Compare February 2, 2022 00:10
@github-actions
Copy link

github-actions bot commented Feb 2, 2022

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@TRYTOBE8TME TRYTOBE8TME force-pushed the wip-rgw-enable-disable-reshard-test-add branch from 74b6ac6 to b51c123 Compare February 2, 2022 05:02
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
allows other code to spawn this coroutine without having the class
definition

Signed-off-by: Casey Bodley <cbodley@redhat.com>
RGWShardCollectCR was hard-coded to ignore ENOENT errors and print a
'failed to fetch log status' error message. this moves that logic into a
handle_result() virtual function. it also exposes the member variables
'status' and 'max_concurrent' as protected, so they can be consulted or
modified by overrides of handle_result() and spawn_next()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
a coroutine to initialize a bucket for full sync using a new bucket-wide
sync status object

Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
full sync happens as the bucket level, so the shards will always start
in StateIncrementalSync

Signed-off-by: Casey Bodley <cbodley@redhat.com>
yuvalif and others added 8 commits February 9, 2022 06:50
use an API that does not check for cache inconsistency
hence, "WARNING: The bucket info cache is inconsistent" warnings is removed from reshard

Signed-off-by: Yuval Lifshitz <ylifshit@redhat.com>
Writing bucket instance info is surprising, as if you pass a null
pointer for the attributes, it just erases all the attributes.

To avoid disturbing users and other 'system objects', make a special
case that we can pass in explicitly.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
And refuse to remove the only log.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
several places were getting the current index layout indirectly
with layout.logs.back() and rgw::log_to_index_layout(). use
get_current_index() instead so we don't rely on layout.logs, which may
be empty for indexless buckets

Signed-off-by: Casey Bodley <cbodley@redhat.com>
always verify that logs is not empty before calling logs.back() or
logs.front()

Signed-off-by: Casey Bodley <cbodley@redhat.com>
RGWSI_BucketIndex_RADOS::handle_overwrite() is already writing the
datalog/bilog entries related to BUCKET_DATASYNC_DISABLED

RGWBucket::sync() calls handle_overwrite() indirectly from
bucket->put_info() when it writes the bucket instance with this new
BUCKET_DATASYNC_DISABLED flag, so RGWBucket::sync() shouldn't
duplicate those writes here

Signed-off-by: Casey Bodley <cbodley@redhat.com>
@adamemerson adamemerson force-pushed the wip-rgw-multisite-reshard branch from d0f01cf to fe5ea5e Compare February 9, 2022 16:35
@github-actions
Copy link

github-actions bot commented Feb 9, 2022

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

Shilpa Jagannath and others added 5 commits February 11, 2022 12:15
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Fetch the current generation from remote peers and trim the minimum
marker on the minimum generation.

Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
The test includes disable-enable bucket sync along with resharding
the bucket.

Signed-off-by: Kalpesh Pandya <kapandya@redhat.com>
@TRYTOBE8TME TRYTOBE8TME force-pushed the wip-rgw-enable-disable-reshard-test-add branch from b51c123 to 4e53747 Compare February 11, 2022 07:01
@TRYTOBE8TME TRYTOBE8TME marked this pull request as ready for review February 14, 2022 09:25
@github-actions
Copy link

This pull request has been automatically marked as stale because it has not had any activity for 60 days. It will be closed if no further activity occurs for another 30 days.
If you are a maintainer or core committer, please follow-up on this pull request to identify what steps should be taken by the author to move this proposed change forward.
If you are the author of this pull request, thank you for your proposed contribution. If you believe this change is still appropriate, please ensure that any feedback has been addressed and ask for a code review.

@github-actions github-actions bot added the stale label Apr 16, 2022
@adamemerson adamemerson force-pushed the wip-rgw-multisite-reshard branch from 69074e1 to ef7373f Compare April 28, 2022 21:07
@TRYTOBE8TME TRYTOBE8TME requested a review from a team as a code owner April 28, 2022 21:07
@github-actions
Copy link

This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved

@adamemerson adamemerson force-pushed the wip-rgw-multisite-reshard branch from ef7373f to 8a70544 Compare May 5, 2022 03:26
@cbodley cbodley force-pushed the wip-rgw-multisite-reshard branch from 230c448 to ede6a85 Compare May 27, 2022 19:48
@cbodley cbodley deleted the branch ceph:wip-rgw-multisite-reshard June 2, 2022 20:04
@cbodley cbodley closed this Jun 2, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants