Skip to content

mgr/cephadm: re-enable osdspec affinity passing#35374

Merged
sebastian-philipp merged 2 commits intoceph:masterfrom
jschmid1:activate_osdspec_affinity
Jul 3, 2020
Merged

mgr/cephadm: re-enable osdspec affinity passing#35374
sebastian-philipp merged 2 commits intoceph:masterfrom
jschmid1:activate_osdspec_affinity

Conversation

@jschmid1
Copy link
Contributor

@jschmid1 jschmid1 commented Jun 4, 2020

Signed-off-by: Joshua Schmid jschmid@suse.de

#34835 is merged now.

There is a big pitfall to this patch:

When the mgr is updated before the osd daemons, the mgr will crash. Currently there is no safeguard against this scenario besides the 'update order' itself.

Signed-off-by: Joshua Schmid <jschmid@suse.de>
@jschmid1 jschmid1 requested a review from a team as a code owner June 4, 2020 06:55
@jschmid1
Copy link
Contributor Author

jschmid1 commented Jun 4, 2020

fixing TestCephadm.test_apply_osd_save

Signed-off-by: Joshua Schmid <jschmid@suse.de>
@sebastian-philipp
Copy link
Contributor

Signed-off-by: Joshua Schmid jschmid@suse.de

#34835 is merged now.

There is a big pitfall to this patch:

When the mgr is updated before the osd daemons, the mgr will crash. Currently there is no safeguard against this scenario besides the 'update order' itself.

well, the upgrade order demands that MGRs are upgraded before OSDs.

@jschmid1
Copy link
Contributor Author

jschmid1 commented Jun 4, 2020

well, the upgrade order demands that MGRs are upgraded before OSDs.

err.. right

@jschmid1
Copy link
Contributor Author

I guess the best option we have is to wait a bit more until we get this in

@sebastian-philipp
Copy link
Contributor

hm. Why do we have a problem here at all?

  • In case MGR is new and existing OSDs are old and we pause the upgrade, OSDs will simply don't have a spec affinity.

  • In case MGR is new and existing OSDs are old and we deploy a new OSD, then we're going to download a new container image for the OSD.

  • In case the MGR is new and we want to deploy an old OSD, (which I think might be a valid use case), waiting a bit longer to merge this PR won't help, as we would need to wait post pacific to merge this.

@jschmid1
Copy link
Contributor Author

  • In case MGR is new and existing OSDs are old and we pause the upgrade, OSDs will simply don't have a spec affinity.

I think this is the most problematic case. In this case the OSDs don't have a version of ceph-osd that supports the --osdspec-affinity option. cephadm however wants to assign assign new OSDs to the osdspec name and passes the --osdspec-affinity flag to ceph-osd, which results in a crash.

Problems like this will probably occur more often in the future when we have cross-dependencies (that are not easily resolvable with version constraints)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants