rgw multisite: add bucket log generation to datalog entries#38561
rgw multisite: add bucket log generation to datalog entries#38561adamemerson wants to merge 4 commits intoceph:wip-rgw-multisite-reshardfrom
Conversation
c94870b to
0ef90d5
Compare
0ef90d5 to
381769f
Compare
|
@adamemerson I feel like the first half of this PR should merge to master and get backported for pacific, and the second half that touches the datalog entries and data sync should instead target the https://github.com/ceph/ceph/commits/wip-rgw-multisite-reshard working branch. could you please split this up so we can get the |
|
This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved |
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Drop entries from past generations. Send entries of future generations to the error repo for retry. Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
381769f to
ca66b68
Compare
|
@cbodley Do we want to merge this in and pull in the error repo and urgent later, or fold this, error repo, and urgent in together? |
| DataLogEntityType entity_type; | ||
| std::string key; | ||
| ceph::real_time timestamp; | ||
| uint64_t gen_id; |
There was a problem hiding this comment.
can we please name this gen to be consistent with the fields in rgw_bucket_layout.h?
|
|
||
| void encode(ceph::buffer::list& bl) const { | ||
| ENCODE_START(1, 1, bl); | ||
| ENCODE_START(2, 2, bl); |
There was a problem hiding this comment.
this raises the compat version to 2, which means that DECODE_START(2, bl) below would throw if it gets a struct_v < 2, and we wouldn't reach this backward-compatible decode logic:
if (struct_v < 2)
gen_id = 0;
else
decode(gen_id, bl);this would cause issues with rolling upgrades in a zone with multiple radosgws
|
i retargeted this to ceph:wip-rgw-multisite-reshard. could you please rebase to resolve the conflicts there? |
|
This pull request can no longer be automatically merged: a rebase is needed and changes have to be manually resolved |
8632248 to
f329d99
Compare
|
@adamemerson i rebased, tested, and got this working in #39396. could you please review the changes? |
|
thanks @adamemerson! this merged into #39002 |
Add the generation id to the datalog.
In Data Sync send entries with an earlier data log to the error repo for retry.
(Moved the gen_id from the bucket shard to the data change as part of debugging, partly because Yehuda suggested it in an email.)