rgw: more multisite dynamic resharding pieces.#34352
Conversation
|
there might be a better place from where generation number can be passed into RGWSI_BucketIndex_RADOS::get_bucket_index_object(). |
|
looks good! i think it's probably time to think about how to test these reshard changes. our teuthology suite mostly tests functionality using really small buckets, so we aren't covering resharding much at all some work was started here in #22939. its gone very stale by now, but i think the approach was good - we need something that can issue s3 requests with boto to create buckets/objects, run radosgw-admin reshard commands, then verify the results. are you willing to resurrect that pr and get it working so we can build on it? for this pr, can you at least do some smoke testing around reshards to make sure we didn't break anything obvious? |
@cbodley sure. Will take a look at the pr. Thanks. |
|
Managed to run some workunit tests. @cbodley Deleting objects are still failing. Looking into that. |
7636e2e to
f584d47
Compare
|
Full rgw suite run: |
|
@cbodley I have rebased on top of your fixes before running the full suite. Can we merge this one? |
rados object names of bi shard to take a unique id called generation number. Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
for better flexibility. Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
qa/workunits/rgw/test_rgw_reshard.py
Outdated
| def install_reqs(): | ||
| out = exec_cmd('cat /etc/*release') | ||
| result = {} | ||
| for row in out.decode('utf8').split('\n'): | ||
| if '=' in row: | ||
| k, v = row.split('=') | ||
| result[k.strip()] = v.strip('""') | ||
| if result['NAME'] == 'Ubuntu': | ||
| exec_cmd('sudo apt install -y python3-pip') | ||
| else: | ||
| exec_cmd('sudo yum install -y python3-pip') | ||
| exec_cmd('sudo pip3 install boto3') |
There was a problem hiding this comment.
this shouldn't be installing stuff with sudo. if we need to set up some dependencies on the teuthology worker, i think we're better off creating a wrapper shell script that sets up a virtualenv and runs this script in it
There was a problem hiding this comment.
Signed-off-by: Shilpa Jagannath <smanjara@redhat.com>
f584d47 to
71b657b
Compare
|
@smanjara in future, could you prefix the title of your commit message with the subcomponent your are changing ? see https://github.com/ceph/ceph/blob/master/SubmittingPatches.rst#3-describe-your-changes |
| BUCKET_NAME1 = 'myfoo' | ||
| BUCKET_NAME2 = 'mybar' | ||
| VER_BUCKET_NAME = 'myver' | ||
| ENDPOINT = 'http://localhost:80' |
There was a problem hiding this comment.
this is failing when the rgw/verify suite runs with proto/https.yaml, because radosgw is running on port 443
sure |
rados object names of bi shard to take a unique id called
generation number.
Signed-off-by: Shilpa Jagannath smanjara@redhat.com
Checklist
Show available Jenkins commands
jenkins retest this pleasejenkins test crimson perfjenkins test signedjenkins test make checkjenkins test make check arm64jenkins test submodulesjenkins test dashboardjenkins test dashboard backendjenkins test docsjenkins render docsjenkins test ceph-volume alljenkins test ceph-volume tox