libnetwork: fix flaky Swarm service DNS#50229
Merged
thaJeztah merged 1 commit intomoby:masterfrom Jun 19, 2025
Merged
Conversation
When libnetwork receives a watch event for a driver table entry from NetworkDB it passes the event along to the interested driver. This code contains a subtle bug: update events from NetworkDB are passed along to the driver as Delete events! This bug was lying dormant as driver-table entries can only be added by the driver, not updated. Now that NetworkDB broadcasts an UpdateEvent to watchers if the entry is already known to the local NetworkDB, irrespective of whether the event received from the remote peer was a CREATE or UPDATE event, the bug is causing problems. Whenever a remote node replaces an entry in the overlay_peer_table but the intermediate delete state was not received by the local node, the new CREATE event would be translated to an UpdateEvent by NetworkDB and subsequently handled by the overlay driver as if the entry was deleted! Bubble table UPDATE events up to the network driver as Update events. Signed-off-by: Cory Snider <csnider@mirantis.com>
aepifanov
approved these changes
Jun 18, 2025
robmry
approved these changes
Jun 19, 2025
akerouanton
approved these changes
Jun 19, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
- What I did
- How I did it
When libnetwork receives a watch event for a driver table entry from NetworkDB it passes the event along to the interested driver. This code contains a subtle bug: update events from NetworkDB are passed along to the driver as Delete events! This bug was lying dormant as driver-table entries can only be added by the driver, not updated. Now that NetworkDB broadcasts an UpdateEvent to watchers if the entry is already known to the local NetworkDB, irrespective of whether the event received from the remote peer was a CREATE or UPDATE event, the bug is causing problems. Whenever a remote node replaces an entry in the overlay_peer_table but the intermediate delete state was not received by the local node, the new CREATE event would be translated to an UpdateEvent by NetworkDB and subsequently handled by the overlay driver as if the entry was deleted!
Bubble table UPDATE events up to the network driver as Update events.
- How to verify it
By inspection? I can't find a way to reproducibly trigger the buggy behaviour.
- Human readable description for the release notes
- A picture of a cute animal (not mandatory but encouraged)