JAVA-3005 Refresh entire node list when new node added#1604
Merged
Conversation
akhaku
commented
Jul 4, 2022
| logPrefix, | ||
| event.broadcastRpcAddress); | ||
| metadataManager.addNode(event.broadcastRpcAddress); | ||
| metadataManager.refreshNodes(); |
Contributor
Author
There was a problem hiding this comment.
Note that refreshNodes is an async operation and we aren't waiting for the result. That's probably fine though, we're in an async code path anyway (debounced handling of topology events).
Contributor
There was a problem hiding this comment.
Indeed, but addNode is also async. That's fine.
akhaku
commented
Jul 4, 2022
|
|
||
| ### 4.14.2 (in progress) | ||
|
|
||
| - [bug] JAVA-3002 JAVA-3005: Refresh entire node list when a new node is added |
Contributor
Author
There was a problem hiding this comment.
we could also just reference one of these tickets and close the other one as a duplicate, let me know
Empirically, Cassandra does not appear to send TopologyChangeType.REMOVED_NODE when a node in the ring crashes hard and is eventually replaced, even though the bad node is removed from the system tables. This results in the driver holding on to that node until a restart. The 3.x driver refreshes the entire node list when a node is added, so when a new node comes up to replace the crashed node, the crashed node is removed from the driver's state. With this change we port that behavior to the 4.x driver and refresh the entire node list when a new node is added. This also fixes JAVA-3002.
Contributor
Author
|
cc @absurdfarce |
Contributor
|
No worries @akhaku , it is in the process of being reviewed right now :) |
absurdfarce
approved these changes
Jul 13, 2022
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Empirically, Cassandra does not appear to send
TopologyChangeType.REMOVED_NODE when a node in the ring
crashes hard and is eventually replaced, even though the
bad node is removed from the system tables. This results in
the driver holding on to that node until a restart.
The 3.x driver refreshes the entire node list when a node is
added, so when a new node comes up to replace the crashed
node, the crashed node is removed from the driver's state.
With this change we port that behavior to the 4.x driver and
refresh the entire node list when a new node is added.
This also fixes JAVA-3002.