Tribe nodes should apply cluster state updates in batches#14993
Tribe nodes should apply cluster state updates in batches#14993jasontedor merged 1 commit intoelastic:masterfrom jasontedor:tribe-node-cluster-state-batch
Conversation
|
@bleskes I'll rebase this pull request on master when #14899 is reintegrated there. The salient commit for this review is thus 476ab3c91b038f5a328bf9360c87e4ee792643d0 pending #14899 (all the changes for that commit are in TribeService.java). |
|
I think we can go further with this one. Now it just saves on cluster state update task, but the work done is the same. I think each executor can pick the latest task (double check that batching maintains order, it should :)) and only apply that. We could in theory also have a global executor (i.e., for all clients) which takes the tasks, groups them by source cluster and applies them to a shared builder but then we run the risk of exceptions in one client blocking updates from all client. I don't think it's worth it. |
@bleskes Addressed in 74adaca4cebd9c0fde32a5080d7cfbbf2dbe9746. |
|
@bleskes In addition to 74adaca4cebd9c0fde32a5080d7cfbbf2dbe9746, I also pushed 479e50adb26b2f5453841273c05cdb17c530065e to only return a new cluster state instance in the tribe service if there were changes to the cluster state. |
|
LGTM. Double checking - do we have a test that makes sure tasks are passed in order? |
I don't think so; should we address that in a separate issue? |
++ on another issue.. It's an important semantics. We should test it under heavy concurrency. |
I opened #15483 to track this. |
Tribe nodes should apply cluster state updates in batches
This commit applies the general mechanism for applying cluster state updates in batches to tribe nodes.
Relates #14899, relates #14725