Skip to content

mgr: process map before notifying clients#56047

Merged
batrick merged 1 commit intoceph:mainfrom
batrick:i64799
Apr 22, 2024
Merged

mgr: process map before notifying clients#56047
batrick merged 1 commit intoceph:mainfrom
batrick:i64799

Conversation

@batrick
Copy link
Member

@batrick batrick commented Mar 7, 2024 •
edited
Loading

Oddly the maps are processed after notifying the modules. I'm not sure if this actually can cause modules to read from the old maps (since finisher contexts are queued) but it seemed odd.

Fixes: https://tracker.ceph.com/issues/64799

Contribution Guidelines

  • To sign and title your commits, please refer to Submitting Patches to Ceph.

  • If you are submitting a fix for a stable branch (e.g. "quincy"), please refer to Submitting Patches to Ceph - Backports for the proper workflow.

  • When filling out the below checklist, you may click boxes directly in the GitHub web UI. When entering or editing the entire PR message in the GitHub web UI editor, you may also select a checklist item by adding an x between the brackets: [x]. Spaces and capitalization matter when checking off items this way.

Checklist

  • Tracker (select at least one)
    • References tracker ticket
    • Very recent bug; references commit where it was introduced
    • New feature (ticket optional)
    • Doc update (no ticket needed)
    • Code cleanup (no ticket needed)
  • Component impact
    • Affects Dashboard, opened tracker ticket
    • Affects Orchestrator, opened tracker ticket
    • No impact that needs to be tracked
  • Documentation (select at least one)
    • Updates relevant documentation
    • No doc update is appropriate
  • Tests (select at least one)
Show available Jenkins commands
  • jenkins retest this please
  • jenkins test classic perf
  • jenkins test crimson perf
  • jenkins test signed
  • jenkins test make check
  • jenkins test make check arm64
  • jenkins test submodules
  • jenkins test dashboard
  • jenkins test dashboard cephadm
  • jenkins test api
  • jenkins test docs
  • jenkins render docs
  • jenkins test ceph-volume all
  • jenkins test ceph-volume tox
  • jenkins test windows
  • jenkins test rook e2e

@batrick batrick requested a review from a team as a code owner March 7, 2024 18:37
@batrick
mgr: process map before notifying clients
190f379 
Oddly the maps are processed after notifying the modules. I'm not sure if this
actually can cause modules to read from the old maps (since finisher contexts
are queued) but it seemed odd.

Fixes: https://tracker.ceph.com/issues/64799
Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
batrick added a commit to batrick/ceph that referenced this pull request Mar 28, 2024
@batrick
Merge PR ceph#56047 into wip-batrick-testing-20240328.192822
822b609 
* refs/pull/56047/head:
	mgr: process map before notifying clients
batrick added a commit to batrick/ceph that referenced this pull request Mar 29, 2024
@batrick
Merge PR ceph#56047 into wip-batrick-testing-20240329.130222
9eb93a2 
* refs/pull/56047/head:
	mgr: process map before notifying clients
batrick added a commit to batrick/ceph that referenced this pull request Mar 31, 2024
@batrick
Merge PR ceph#56047 into wip-batrick-testing-20240331.000745
60dab94 
* refs/pull/56047/head:
	mgr: process map before notifying clients
batrick added a commit to batrick/ceph that referenced this pull request Apr 2, 2024
@batrick
Merge PR ceph#56047 into wip-batrick-testing-20240402.004512
c15dfb2 
* refs/pull/56047/head:
	mgr: process map before notifying clients
@batrick
Copy link
Member Author

batrick commented Apr 2, 2024 •
edited
Loading

https://tracker.ceph.com/issues/65215#note-8

Good to merge.

@batrick batrick requested a review from vshankar April 4, 2024 01:07
@gregsfortytwo
Copy link
Member

gregsfortytwo commented Apr 10, 2024

Hmm are you sure there's not some weird reason for this? I could definitely see one of the modules needing to do something before the mgr process triggers everything else, though it would be gross. Did you do some git archaeology to see where it came from?

@batrick
Copy link
Member Author

batrick commented Apr 10, 2024

Hmm are you sure there's not some weird reason for this? I could definitely see one of the modules needing to do something before the mgr process triggers everything else, though it would be gross. Did you do some git archaeology to see where it came from?

The call to handle_fs_map has always been that way. John actually mentioned a race possibility in the comment in that method:

ac30e6c#diff-cf12a9b0314445319a3f842090e100e08a1181f469acc37c09ee4387ce4005b6R488-R491

In any case, the reversed update was also present for handle_osd_map in that same commit. This just makes map updates consistent in the mgr.

@batrick
Copy link
Member Author

batrick commented Apr 17, 2024

This PR is under test in https://tracker.ceph.com/issues/65530.

@batrick batrick requested review from ljflores and rzarzynski April 17, 2024 12:44
rzarzynski
rzarzynski approved these changes Apr 17, 2024
View reviewed changes
Copy link
Contributor

@rzarzynski rzarzynski left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. However, let's not hurry up with backporting this chnage. IMHO it deserves some baking in main.

src/mgr/Mgr.cc
case CEPH_MSG_MON_MAP:
/* MonClient passthrough of MonMap to us */
handle_mon_map(); /* use monc's monmap */
py_module_registry->notify_all("mon_map", "");
Copy link
Contributor

@rzarzynski rzarzynski Apr 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now this looks far more intuitive / less weird.

The call to handle_fs_map has always been that way. John actually mentioned a race possibility in the comment in that method:

OK, so this peculiar order has not been brought by any fix commit.

@batrick
Copy link
Member Author

batrick commented Apr 18, 2024

This PR is under test in https://tracker.ceph.com/issues/65562.

@batrick
Copy link
Member Author

batrick commented Apr 20, 2024

This PR is under test in https://tracker.ceph.com/issues/65596.

@batrick
Copy link
Member Author

batrick commented Apr 22, 2024

https://tracker.ceph.com/issues/65596#note-1

@batrick batrick merged commit f34b439 into ceph:main Apr 22, 2024
@batrick batrick deleted the i64799 branch April 22, 2024 19:07
This was referenced Apr 23, 2024
Merged
Merged
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@rzarzynski rzarzynski rzarzynski approved these changes

@vshankar vshankar Awaiting requested review from vshankar

@ljflores ljflores Awaiting requested review from ljflores

Assignees

No one assigned

Labels

cleanup core mgr wip-pdonnell-testing

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@batrick @gregsfortytwo @rzarzynski