please note this is not a backport from master. it only applies to hammer as a workaround.

we added 3 more fields to pg_pool_t between v0.94.6 and v0.94.7, the last field is use_gmt_hitset, and a new feature bit is used to track this change. OSDMap has a field of map<int64_t,pg_pool_t>. the CRC check is added since v0.91, where the OSDMap encoding version was bumped from 8. in this change, we disable CRC check by downgrading the OSDMap encoding version to v7 if the connection.features does not have the GMT feature bit. but this could also remove information that matters that gets added after OSDMap v8. so this change can only stay in hammer.

• hammer (<= v0.94.6) => hammer (with this fix): osdmap messages are sent without CRC enabled if peer does not support GMT feature.
• hammer (<= v0.94.6) => jewel (without mon/OSDMonitor: encode full osdmaps with features all OSDs can understand #11284): should upgrade to hammer (with this fix) first.
• hammer (with this fix) => jewel (with mon/OSDMonitor: encode full osdmaps with features all OSDs can understand #11284): monitor will encode full map with features - CEPH_FEATURE_SERVER_JEWEL. so we need to make sure that in the next jewel release:
  1. all (sub)fields of OSDMap respect the features parameter when encode()
  2. be pessimistic if CEPH_FEATURE_SERVER_JEWEL is missing. say, to encode using the first encoding version when the v0.94.4[1] was released. for example, suppose foo and bar are added into pg_pool_t after v0.94.4 is released, and the encoding version of pg_pool_t is then v23. we should stick with v17 if CEPH_FEATURE_SERVER_JEWEL is missing when encoding pg_pool_t.
• jewel (without mon/OSDMonitor: encode full osdmaps with features all OSDs can understand #11284) => jewel (with mon/OSDMonitor: encode full osdmaps with features all OSDs can understand #11284): if new fields are added to osdmap in between these two releases. we should ignore CRC mismatches (feature implemented in fixes for CRC mismatch failures on OSD when upgrade #11258).

[1] v0.94.4 is the first hammer release allowed to upgrade to jewel. see http://ceph.com/releases/v10-2-0-jewel-released/. and the encoding version of pg_pool_t was v17 then.

tested at tested at http://pulpito.front.sepia.ceph.com:80/kchai-2016-10-08_08:59:22-upgrade-wip-17386-hammer---basic-mira/ and http://pulpito.ceph.com/kchai-2016-10-08_09:07:08-upgrade-wip-17386-hammer---basic-mira/

To clarify that http://tracker.ceph.com/issues/17534 was created and made to be a backport of http://tracker.ceph.com/issues/17386. It's fine that it's a workaround instead of cherry-picks. The Fixes: should be changed to http://tracker.ceph.com/issues/17534 instead.

Does that sound sensible ?

sounds very sensible!

@athanatos could you take a look?

I thought we weren't going to use this approach for hammer, and instead modify jewel and newer to not include the new bits in the OSDMap until the user flips the switch on the monitor banning old nodes.

got it. i will repurpose this PR for updating the document to note down the upgrade process of hammer: osd first then mon.

@athanatos could you take a look again?

LGTM!

@dachary could you take a look and merge it if appropriate? thanks