Skip to content

tentacle: Beacon diff#66958

Merged
leonidc merged 4 commits intoceph:tentaclefrom
Hezko:wip-74413-tentacle
Feb 2, 2026
Merged

tentacle: Beacon diff#66958
leonidc merged 4 commits intoceph:tentaclefrom
Hezko:wip-74413-tentacle

Conversation

@Hezko
Copy link
Contributor

@Hezko Hezko commented Jan 18, 2026

backport tracker: https://tracker.ceph.com/issues/74413

backport of #65563
parent tracker: https://tracker.ceph.com/issues/72394

this backport was staged using ceph-backport.sh version 16.0.0.6848
find the latest version at https://github.com/ceph/ceph/blob/main/src/script/ceph-backport.sh

@Hezko Hezko requested a review from a team as a code owner January 18, 2026 09:26
@Hezko Hezko added this to the tentacle milestone Jan 18, 2026
@Hezko Hezko assigned Hezko and leonidc and unassigned Hezko Jan 18, 2026
leonidc and others added 4 commits January 27, 2026 11:27
@leonidc
nvmeofgw: beacon diff implementation in the monitor and in the MonCli…
3b5d28b 
…ent.

     -monclient encodes subsystems by beacon-diff rules if BEACON_DIFF
      bit is enabled by quorum
     -monitor processes beacons by beacon-diff new schema
     -monitor detects sequence out of order(ooo) condition and handles it
     -in case ooo detected monitor send ack to the gw with the expected correct sequence
     -monitor skips failovers for some interval when ooo detected
     -monitor ignores all becons with incorrect sequences until gw sends expected one
     -coding upgrade rules

  Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>

Fixes: https://tracker.ceph.com/issues/72394
(cherry picked from commit 3555a28)
@athanatos
mon/MonClient: add get_monmap_required_features() helper
d87ff95 
Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit 9580809)
@athanatos
mon: add NVMEOF_BEACON_DIFF to mon_feature_t and mon CompatSet
5467ad9 
NOPE NOPE
In order for the client to safely send BEACON_DIFF messages, it
needs to be the case that the leader at the time of receipt will
support BEACON_DIFF.

Simply using the connection features for the MonClient's target mon is
insufficient, because it might be a peon.  If the peon supports
BEACON_DIFF and the leader does not the leader will either crash or
interpret it as a full BEACON.  Neither outcome is acceptable.

Instead, we need to wire up a feature bit to the MonMap mon_feature_t
members and the CompatSet.

Adding FEATURE_BEACON_DIFF to ceph::features::mon get_supported()
and get_persistent() ensures that once all monitors in the quorum
support it, MonMap::get_required_features() will include it.
See Elector::propose_to_peers, Monitor::(win|lose)_election,
MonmapMonitor::apply_mon_features.

Once FEATURE_BEACON_DIFF is present in MonMap::get_required_features():
- Monitor::apply_monmap_to_compatset_features() will prevent
  downgrades of the monitors by updating the CompatSet to include
  CEPH_MON_FEATURE_INCOMPAT_NVMEOF_BEACON_DIFF
- Monitor::calc_quorum_requirements() will set
  Monitor::required_features to require the NVMEOF_BEACON_DIFF
  for any monitor peers.
- MonClient::get_monmap_required_features() will eventually include
  ceph::features::mon::FEATURE_NVMEOF_BEACON_DIFF.

Signed-off-by: Samuel Just <sjust@redhat.com>
(cherry picked from commit e55ad7b)
@leonidc
nvmeofgw: prevent map corruption while processing beacons from delete…
5059a1b 
…d gws

Fix race issue of map corruption when deleted gw sends beacons
but this gw data was removed from pending map and still exists in map.
Process beacons only if GW's data exists in both maps:
main-map and pending-map, otherwise just ignore beacons.

fixes: https://tracker.ceph.com/issues/74160

Signed-off-by: Leonid Chernin <leonidc@il.ibm.com>
(cherry picked from commit 3636dc3)
@Hezko Hezko force-pushed the wip-74413-tentacle branch from 2453b1e to 5059a1b Compare January 27, 2026 09:28
@leonidc leonidc merged commit 6dddf54 into ceph:tentacle Feb 2, 2026
12 of 13 checks passed
batrick added a commit to batrick/ceph that referenced this pull request Mar 12, 2026
@batrick
Revert "Merge pull request ceph#66958 from Hezko/wip-74413-tentacle"
35d19d6 
This reverts commit 6dddf54, reversing
changes made to 07ec509.

Conflicts:
	src/mon/NVMeofGwMon.h: conflicts with header change from 19c9be2
batrick added a commit to batrick/ceph that referenced this pull request Mar 12, 2026
@batrick
Revert "Merge pull request ceph#66958 from Hezko/wip-74413-tentacle"
5b219a7 
This reverts commit 6dddf54, reversing
changes made to 07ec509.

Conflicts:
	src/mon/NVMeofGwMon.h: conflicts with header change from 19c9be2

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
batrick added a commit to batrick/ceph that referenced this pull request Mar 12, 2026
@batrick
Revert "Merge pull request ceph#66958 from Hezko/wip-74413-tentacle"
7327bd4 
This reverts commit 6dddf54, reversing
changes made to 07ec509.

Backport 6dddf54 introduced a new connection feature bit
NVMEOF_BEACON_DIFF but there are plans (ceph#66624) to make further
enhancements on that feature bit. This would cause the mons to crash
during upgrades.

However, this connection feature bit should not have been added to
begin with. The correct way to do this is extend
e55ad7b by @athanatos to require
`CEPH_MON_FEATURE_INCOMPAT_NVMEOF_BEACON_DIFF` if all mons support it.
This should be done by having mons add/update their supported features
the MonMap via an update from `MMonJoin` (see for instance `crush_loc`
which was recently added to `mon_info_t`). Once the supported features
indicated for each mon in the `MonMap` show they understand the new
NVMEOF_BEACON_DIFF, then it should be turned on globally in the
`MonMap` as a required feature (added to the incompat set).

Conflicts:
	src/mon/NVMeofGwMon.h: conflicts with header change from 19c9be2

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
batrick added a commit to batrick/ceph that referenced this pull request Mar 15, 2026
@batrick
Revert "Merge pull request ceph#66958 from Hezko/wip-74413-tentacle"
ddef0d1 
This reverts commit 6dddf54, reversing
changes made to 07ec509.

Backport 6dddf54 introduced a new connection feature bit
NVMEOF_BEACON_DIFF but there are plans (ceph#66624) to make further
enhancements on that feature bit. This would cause the mons to crash
during upgrades.

However, this connection feature bit should not have been added to
begin with. The correct way to do this is extend
e55ad7b by @athanatos to require
`CEPH_MON_FEATURE_INCOMPAT_NVMEOF_BEACON_DIFF` if all mons support it.
This should be done by having mons add/update their supported features
the MonMap via an update from `MMonJoin` (see for instance `crush_loc`
which was recently added to `mon_info_t`). Once the supported features
indicated for each mon in the `MonMap` show they understand the new
NVMEOF_BEACON_DIFF, then it should be turned on globally in the
`MonMap` as a required feature (added to the incompat set).

Conflicts:
	src/mon/NVMeofGwMon.h: conflicts with header change from 19c9be2
                               fix missing header change in ceph#66584

Signed-off-by: Patrick Donnelly <pdonnell@redhat.com>
@VallariAg VallariAg mentioned this pull request Mar 15, 2026
Merged
batrick added a commit that referenced this pull request Mar 18, 2026
@batrick
Merge PR #67750 into tentacle
147f7c6 
* refs/pull/67750/head:
	Revert "Merge pull request #66958 from Hezko/wip-74413-tentacle"

Reviewed-by: Yuri Weinstein <yweins@redhat.com>
Tested-by: Patrick Donnelly <pdonnell@redhat.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@athanatos athanatos athanatos approved these changes

Assignees

@leonidc leonidc

Labels

build/ops core mon nvmeof tests

Projects

None yet

Milestone

tentacle

Development

Successfully merging this pull request may close these issues.

3 participants

@Hezko @athanatos @leonidc