Batch cluster state tasks for index opening, closing, and blocks#83432
Batch cluster state tasks for index opening, closing, and blocks#83432joegallo wants to merge 8 commits intoelastic:masterfrom
Conversation
and use it for the three index-block tasks
|
Pinging @elastic/es-data-management (Team:Data Management) |
original-brownbear
left a comment
There was a problem hiding this comment.
Thanks Joe! I'll take a detailed look tomorrow. One small point only after a quick read. I think this might be fine as is but I'll have to think through potential side effects for a bit.
| return allocationService.reroute(updatedState, "indices opened [" + indicesAsString + "]"); | ||
| } | ||
| }, | ||
| ClusterStateTaskExecutor.unbatched() |
There was a problem hiding this comment.
Not sure we want to batch this in with the other operations? It seems like this is not something that ILM will do in a hot loop right?
I could see some strange side effects of batching open and close index updates into a single batch in some edge cases.
|
Hi @joegallo, I've created a changelog YAML for you. |
|
With the callback-accepting simple batching executor I added in ee83da0, we could defer the |
|
w.r.t. batching of opens and closes, I'm hesitant to only batch the blocks, because it seems like the blocks are part of the close, and then the close is always followed by an open (at least in the case of ILM). OTOH if the closes and opens are cheaper/faster by an order of magnitude or something like that, then maybe it doesn't matter. |
Closes #81627
Pretty straightforward -- instead of using
unbatched()five times, we use a simple batching executor that just loops through the tasks and executes them all.A few thoughts:
open-indicesuses anAckedClusterStateUpdateTaskrather than just aClusterStateUpdateTask-- is it still okay to batch those together with the rest of the tasks?