Alloc. awareness should ignore nodes without attr#69374
Conversation
Today we count `null` (i.e. missing) as a valid attribute value in allocation awareness, even though allocation awareness forbids the allocation of shards to such a node. Prior to elastic#69334 this didn't matter, a data node without allocation attributes was pointless. However, elastic#69334 means we now can allocate shards to such a node: for instance, there is no need for nodes holding only enrich indices to have allocation attributes. Therefore we should stop counting `null` as one of the attribute values.
|
Pinging @elastic/es-distributed (Team:Distributed) |
ywelsch
left a comment
There was a problem hiding this comment.
I find it a bit difficult to understand the implications of this PR. will it break clusters that have cluster.routing.allocation.awareness.attributes defined but where the nodes are missing the attributes?
| .add(newNode("X-0", emptyMap())) | ||
| ).build(), strategy); | ||
|
|
||
| assertThat(clusterState.getRoutingNodes().shardsWithState(UNASSIGNED), empty()); |
There was a problem hiding this comment.
where do we assert that X-0 is ignored, i.e. does not hold a shard copy?
|
Today if you have a cluster containing a data node without all the awareness attributes defined then those nodes get no shards at all. It's hard to imagine that there are any clusters in this state today, they would have these empty data nodes and would almost always have unassigned shards too. The effect of this change is to reduce the effective number of attribute values in such a cluster, which relaxes the upper bound on how many shards can be allocated to each value, which shouldn't break anything. |
ywelsch
left a comment
There was a problem hiding this comment.
ok, I see, thank you for the explanation. LGTM
Today we count `null` (i.e. missing) as a valid attribute value in allocation awareness, even though allocation awareness forbids the allocation of shards to such a node. Prior to #69334 this didn't matter, a data node without allocation attributes was pointless. However, #69334 means we now can allocate shards to such a node: for instance, there is no need for nodes holding only enrich indices to have allocation attributes. Therefore we should stop counting `null` as one of the attribute values.
Today we count
null(i.e. missing) as a valid attribute value inallocation awareness, even though allocation awareness forbids the
allocation of shards to such a node. Prior to #69334 this didn't matter,
a data node without allocation attributes was pointless.
However, #69334 means we now can allocate shards to such a node: for
instance, there is no need for nodes holding only enrich indices to have
allocation attributes. Therefore we should stop counting
nullas oneof the attribute values.