The method, OSD::mon_cmd_set_config() currently sets a config option
related to mClock during OSD boot-up. The method used to wait on a
condition variable until the mon ack'ed the command. This was generally
not a problem.

But there could be scenarios where monitor could be slow to respond, or
due to a flaky network, response could be delayed. The OSD could therefore
be blocked from booting-up. To avoid this, the conditional wait can be
replaced with an async Context completion.

Moreover, persisting this in the monitor store is not very critical. An
existing fallback mechanism stores this value in the in-memory "values"
map of the config subsystem. This can be read by the OSD at any point
during its operation.

The issue of the OSDs being blocked from booting-up properly was
observed when running tests with failure injections during OSD boot-up.

Following are the changes:

The changes to mon_cmd_set_config() are generic and any osd specific
option may be set in the config monitor store using this method.

• Implement Context based completion tracking of the mon command using
  MonCmdSetConfigOnFinish which is derived from the base Context class.
  In case of failures, the finish() method is overriden to save the config
  option in the config subsystem's "values" map at CONF_DEFAULT level
  using set_val_default() method. This allows users to modify this at a
  later point using "ceph config set" cli command.

• Additionally, if requested, finish() also checks if the config option
  needs to be applied on each op shard and calls update_scheduler_config()
  on each shard. This is required for some config options related to the
  mClock scheduler.

Fixes: https://tracker.ceph.com/issues/57040
Signed-off-by: Sridhar Seshasayee sseshasa@redhat.com

Contribution Guidelines

• To sign and title your commits, please refer to Submitting Patches to Ceph.

• If you are submitting a fix for a stable branch (e.g. "pacific"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

Checklist

• Tracker (select at least one)
  • References tracker ticket
  • Very recent bug; references commit where it was introduced
  • New feature (ticket optional)
  • Doc update (no ticket needed)
  • Code cleanup (no ticket needed)
• Component impact
  • Affects Dashboard, opened tracker ticket
  • Affects Orchestrator, opened tracker ticket
  • No impact that needs to be tracked
• Documentation (select at least one)
  • Updates relevant documentation
  • No doc update is appropriate
• Tests (select at least one)
  • Includes unit test(s)
  • Includes integration test(s)
  • Includes bug reproducer
  • No tests