Ceph » rgw

another documentation based solution is to allow appending on the pool:

ceph osd pool set ec_pool allow_ec_overwrites true

the data_extra_pool is defined in struct RGWZonePlacementInfo here: https://github.com/ceph/ceph/blob/ef153371/src/rgw/rgw_zone_types.h#L218-L220
its default value is set by init_zone_pool_names() here: https://github.com/ceph/ceph/blob/ef153371/src/rgw/rgw_zone.cc#L685

if the bucke't pool is an EC pool, we should use a namespace (e.g. "bucket-logging") inside "data_extra_pool" instead of ther bucket's pool.
we should also add an optional pg conf for a separate "bucket logging pool" in case that the "data_extra_pool" has charectaristics which are not suiteable for long term data

this could be a new bucket_logging_pool variable in struct RGWZonePlacementInfo that defaults to the same value of data_extra_pool but with :logging appended. RGWZonePlacementInfo::decode() would need to handle the case where we're decoding an earlier version without bucket_logging_pool, and should initialize it to that same default

1. create a dedicated bucket logging pool (replicated) that the "tail" objects of the log bucket will use. note that because of the way we are flushing the log object to the bucket, we create an empty "head" object (that could be in whatever pool the bucket was set on).
2. similar to (1), but do that only if the log bucket is on EC pool
3. use the dedicated pool only temporarily, and copy the log object into the log bucket when flushed
4. same as (3), but do that only if the log bucket is on EC pool

the cons in (1) and (2) is that we don't honor the users request as for which pool should hold the logs - they might want (for efficiency) to actually use an EC pool in this case
the cons in (3) and (4) is that we pay an extra cost of copy when flushing. this is not too bad from overall perf perspective, since this is infrequent. the main issue here would be if flushing happens explicitly due to time limit, and this add extra latency to the op triggering it. there are some ways around it, by making the flushing async, but this would complicate the implementation