Skip to content

reef:mon/Monitor.cc: exit function if !osdmon()->is_writeable()#51409

Merged
yuriw merged 2 commits intoceph:reeffrom
kamoltat:wip-ksirivad-reef-backport-50857
May 22, 2023
Merged

reef:mon/Monitor.cc: exit function if !osdmon()->is_writeable()#51409
yuriw merged 2 commits intoceph:reeffrom
kamoltat:wip-ksirivad-reef-backport-50857

Conversation

@kamoltat
Copy link
Member

@kamoltat kamoltat commented May 9, 2023

NOTE: no need to backport #47340 because it already exists in reef.

Problem:

  1. In the function maybe_go_degraded_stretch_mode()
    when osdmon is not writeable we shouldn't go into
    trigger_degraded_stretch_mode because we will
    crash at ceph_assert(osdmon()->is_writeable()).
    The current code does not exit maybe_go_degraded_stretch_mode()
    when we are waiting for osdmon to be writeable, therefore,
    we crash.

Solution:

  1. Exit the function by returning nothing after going into
    wait_for_writeable_ctx, since at that point we would have
    queued the context and all we have to do is wait for finish
    context to execute maybe_go_degraded_stretch_mode again.
    Also, added a bit of logging so that user is aware
    when osdmon and monmon are not writeable.
    We fix other parts of the monitor code that are missing
    the return after wait_for_writeable_ctx and wait_for_readable_ctx
    as well.

Fixes:
https://tracker.ceph.com/issues/59271

Backporting relevant commits from main PRs:

#50857

Contribution Guidelines

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows

kamoltat added 2 commits May 9, 2023 14:10
Problem:

In the function `maybe_go_degraded_stretch_mode()`
when `osdmon` is not writeable we shouldn't go into
`trigger_degraded_stretch_mode` because we will
crash at `ceph_assert(osdmon()->is_writeable())`.
The current code does not exit `maybe_go_degraded_stretch_mode()`
when we are waiting for `osdmon` to be writeable, therefore,
we crash.

Solution:

Exit the function by returning nothing after going into
`wait_for_writeable_ctx`, since at that point we would have
queued the context and all we have to do is wait for finish
context to execute `maybe_go_degraded_stretch_mode` again.

Also, added a bit of logging so that user is aware
when `osdmon` and `monmon` are not writeable.

We fix other parts of the monitor code that are missing
the return after `wait_for_writeable_ctx` and `wait_for_readable_ctx`
as well.

Fixes: https://tracker.ceph.com/issues/59271

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit 7f7c2d5)
Separate `mon-stretch` from `mon`.

Renamed `mon-stretched-cluster.sh` to
`mon-stretch-fail-recovery.sh`.

This isolation of stretch cluster test will enable
developers to get results faster for stretch-cluster
related stuff.

Signed-off-by: Kamoltat <ksirivad@redhat.com>
(cherry picked from commit 431c455)
@kamoltat kamoltat added backport: no-conflicts Backport without conflicts backport reef labels May 9, 2023
@kamoltat kamoltat self-assigned this May 9, 2023
@kamoltat kamoltat requested a review from a team as a code owner May 9, 2023 15:24
@kamoltat kamoltat requested a review from gregsfortytwo May 15, 2023 14:50
@ljflores ljflores added this to the reef milestone May 16, 2023
@ljflores
Copy link
Member

@yuriw yuriw merged commit f8edcf4 into ceph:reef May 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants