rgw multisite: add bucket log generation to datalog entries#39396
rgw multisite: add bucket log generation to datalog entries#39396cbodley merged 8 commits intoceph:wip-rgw-multisite-reshardfrom
Conversation
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
src/rgw/rgw_datalog.h
Outdated
|
|
||
| void encode(ceph::buffer::list& bl) const { | ||
| ENCODE_START(1, 1, bl); | ||
| ENCODE_START(2, 2, bl); |
There was a problem hiding this comment.
For rolling upgrades, we should change this so it sends the 1,1 version (that doesn't know about gen ids and can be decoded by old versions) when we're in generation zero. That way we only break compatibility when we reshard.
There was a problem hiding this comment.
if an upgraded radosgw encodes this with v2, a non-upgraded radosgw will still be able to decode it as v1 because the compat version allows it. the DECODE_FINISH() macro will just skip past the unrecognized bytes at the end
conversely, it could be helpful to raise an error in the case where a datalog entry with gen>0 would be misinterpreted by older clients. how about this?
const uint8_t compat = (gen == 0) ? 1 : 2; // require decoders to recognize v2 when gen>0
ENCODE_START(2, compat, bl);There was a problem hiding this comment.
That sounds fine to me!
There was a problem hiding this comment.
we only try to decode these entries in RGWDataChangesBE::list(), so an error there would effectively prevent old radosgws from serving those entries to other zones in RGWOp_DATALog_List
but if gen>0, require decoders to understand the v2 format. this way, old clients can't decode entries with gen>0, so they won't be able to serve them to other zones Signed-off-by: Casey Bodley <cbodley@redhat.com>
Drop entries from past generations. Send entries of future generations to the error repo for retry. Signed-off-by: Adam C. Emerson <aemerson@redhat.com>
wait until we've read the bucket sync status and found that we're in incremental sync before we start using incremental_gen for comparison Signed-off-by: Casey Bodley <cbodley@redhat.com>
Signed-off-by: Casey Bodley <cbodley@redhat.com>
ce9933d to
efb9705
Compare
rebased version of #38561 with additional fixes
Checklist
Show available Jenkins commands
jenkins retest this pleasejenkins test classic perfjenkins test crimson perfjenkins test signedjenkins test make checkjenkins test make check arm64jenkins test submodulesjenkins test dashboardjenkins test apijenkins test docsjenkins render docsjenkins test ceph-volume alljenkins test ceph-volume tox