Skip to content

mgr/cephadm: fix 'auth get-or-create' call#40071

Merged
liewegas merged 2 commits intoceph:masterfrom
liewegas:cephadm-fix-auth-caps-on-upgrade
Mar 15, 2021
Merged

mgr/cephadm: fix 'auth get-or-create' call#40071
liewegas merged 2 commits intoceph:masterfrom
liewegas:cephadm-fix-auth-caps-on-upgrade

Conversation

@liewegas
Copy link
Member

If get-or-create caps doesn't work, we do 'auth caps' to update caps,
but this can also fail if the mons are running 15.2.0 (the 'mgr' profile
was updated to allow 'auth caps' in 15.2.1, see 467a27a).

Signed-off-by: Sage Weil sage@newdream.net

@liewegas liewegas requested a review from a team as a code owner March 12, 2021 16:15
@liewegas liewegas force-pushed the cephadm-fix-auth-caps-on-upgrade branch from c3d3ef1 to 7c6d722 Compare March 12, 2021 16:37
@adk3798
Copy link
Contributor

adk3798 commented Mar 12, 2021

Is this specifically to fix upgrades from a version < 15.2.1 to a version > 15.2.1 where this is a temporary issue while the mgr daemons have already been upgraded but the mon daemons have not been upgraded yet? I'm guessing we can't just leave it in a state like this where we call 'auth get-or-create' without caps then add them in after with 'auth caps' if 'auth caps' doesn't work because then any newly created daemons wouldn't have their caps set.

@liewegas liewegas requested a review from adk3798 March 12, 2021 17:35
@liewegas
Copy link
Member Author

Is this specifically to fix upgrades from a version < 15.2.1 to a version > 15.2.1 where this is a temporary issue while the mgr daemons have already been upgraded but the mon daemons have not been upgraded yet? I'm guessing we can't just leave it in a state like this where we call 'auth get-or-create' without caps then add them in after with 'auth caps' if 'auth caps' doesn't work because then any newly created daemons wouldn't have their caps set.

Right. And I think the first question is how and why did the mgr caps change to cause this problem. Reproducing again to get a closer look...

1- Pass caps to 'auth get-or-create'
2- Only try 'auth caps' if the get-or-create failed

Note that the 'auth caps' step can fail if upgrading from 15.2.0 since
'profile mgr' didn't include 'auth caps' until 15.2.1.  We're not
addressing that for now...

Fixes: 7c0d532

Signed-off-by: Sage Weil <sage@newdream.net>
@liewegas liewegas changed the title mgr/cephadm: tolerate 'auth caps' failure mgr/cephadm: fix 'auth get-or-create' call Mar 12, 2021
@liewegas
Copy link
Member Author

jenkins test make check

Copy link
Contributor

@adk3798 adk3798 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, do we still want a log message when the 'auth caps' call fails like you had originally?

If we're upgrading from 15.2.0, we may fail to update caps.  Instead of
failing the upgrade hard, warn to the log and continue.  This is less
than ideal, but the caps will get corrected the next time the daemon is
redeployed on the next upgrade, and most likely the previous caps will
continue to work (given they were presumably working before the upgrade).

Signed-off-by: Sage Weil <sage@newdream.net>
@liewegas
Copy link
Member Author

@adk3798 added another patch that adds the warning and allows the upgrade to proceed (instead of failing hard, which breaks our rados/cephadm/upgrade tests that start from 15.2.0)

@liewegas liewegas merged commit 52c016a into ceph:master Mar 15, 2021
@liewegas liewegas deleted the cephadm-fix-auth-caps-on-upgrade branch March 15, 2021 20:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants