-
Notifications
You must be signed in to change notification settings - Fork 0
background trim #61
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
background trim #61
Conversation
ShooterIT
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
| resetManualFailover(); | ||
|
|
||
| /* Cancel all ASM tasks when switching into slave */ | ||
| if (was_master) clusterAsmCancel(NULL, "switching to replica"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
do we need to cancel the trimming task in some cases?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
such as cluster reset
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think we should cancel on emptyData() and this is mostly a concern for active trimming. e.g. after emptyData(), we don't try to unnecessarily continue trimming. I added a commit to cancel pending tasks, in the follow-up PR, it will also cancel active trim tasks. Do you see some other cases that we need to cancel?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
here, the master switches to replica, we should cancel, right?
and clusterDelNode?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not sure if we need to cancel it if we become a replica. e.g. trimming on master was in progress, replica finished it earlier. We trigger the failover. The new master will not propagate TRIMSLOTS as there is no unowned data. Though, the new replica will still have keys to be trimmed. So, I feel like replica should continue trimming. Let me know if you have a better solution in mind.
for clusterDelNode(), do we call this for our own node? Is this the scenario that we are removed from the cluster?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Is there a scenario that the node sees itself in the clusterDelNode() ?
hmm, seems I mixed up canceling the trim and canceling the task?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
yeah, we have to cancel the task if other party is deleted. So, looks like a node will never figure out it is deleted from the cluster unless you call CLUSTER RESET.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
after second thought, we should cancel trimming task if becoming replica, for active trimming, don't cancel if the trimming is in progress, cancel pending trimming tasks
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is different between pending task and active trimming? Consider this scenario:
e.g. trimming on master was in progress, replica finished it earlier. We trigger the failover. The new master will not propagate TRIMSLOTS as there is no unowned data. Though, the new replica will still have keys to be trimmed.
If we cancel the pending trimming task, replica will never do the clean up.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
hmm, maybe after active trimming, something will be clear to me.
please merge
73e998b to
1f70cfb
Compare
Adds background trim.