Handle deletion of swapped out node in a background periodic task#786
Merged
mattisonchao merged 6 commits intooxia-db:mainfrom Oct 25, 2025
Merged
Handle deletion of swapped out node in a background periodic task#786mattisonchao merged 6 commits intooxia-db:mainfrom
mattisonchao merged 6 commits intooxia-db:mainfrom
Conversation
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
|
|
||
| chanBufferSize = 100 | ||
|
|
||
| DefaultPeriodicTasksInterval = 1 * time.Minute |
Member
There was a problem hiding this comment.
Maybe we could use backoff to avoid being delayed too much?
Collaborator
Author
There was a problem hiding this comment.
I think delaying the shard deletion by 1 min is ok and should never be a problem. If any backoff, I'd do a backoff from 1 min -> 1h -> 1day for this kind of operations.
Member
There was a problem hiding this comment.
NO, we shouldn't do a 1-day backoff. The shard deletion will be delayed for a very long time. :)
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
Signed-off-by: Matteo Merli <mmerli@apache.org>
779fb53 to
aeb509a
Compare
mattisonchao
approved these changes
Oct 25, 2025
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
#845
After a
SwapNode()initiated leader election, let's not wait until the removed node has deleted the shard since this is delaying the leader election time.Instead, move the logic out to a periodic task with retries.
Note: the node is immediately moved to a different category in the status and marked in the
PendingDeleteShardNodeslist. This is because it will not participate anymore in the newTermQuorum.