Skip to content

[8.5] [ML] Do not make autoscaling decision when memory is undetermined (#90259)#90335

Merged
elasticsearchmachine merged 1 commit intoelastic:8.5from
dimitris-athanasiou:backport/8.5/pr-90259
Sep 26, 2022
Merged

[8.5] [ML] Do not make autoscaling decision when memory is undetermined (#90259)#90335
elasticsearchmachine merged 1 commit intoelastic:8.5from
dimitris-athanasiou:backport/8.5/pr-90259

Conversation

@dimitris-athanasiou
Copy link
Copy Markdown
Contributor

Backports the following commits to 8.5:

…astic#90259)

When a processor autoscaling decider was added in elastic#89645, an unwanted
change of behaviour sneaked in. In particular, if we cannot determine
required memory capacity, we previously returned `new AutoscalingDeciderResult(null)`
where as we now return an autoscaling result with no memory capacity
and whatever the result of the processor decider is.

Previously, if we returned a result with null capacity, the cluster
would remain as-is. Now, it is possible to cause unwanted scaling.

This commit fixes this by checking if the memory decider result
was undetermined and returns an empty result if so.

Also, some logging warnings have been added to pop up scenarios
that shouldn't happen like when the memory tracker is not called
by the master node or it has no memory estimate for anomaly detection
or analytics jobs.
@dimitris-athanasiou dimitris-athanasiou added :ml Machine learning >non-issue auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport Team:ML Meta label for the ML team labels Sep 26, 2022
@elasticsearchmachine elasticsearchmachine merged commit 101bfd8 into elastic:8.5 Sep 26, 2022
@dimitris-athanasiou dimitris-athanasiou deleted the backport/8.5/pr-90259 branch September 26, 2022 09:03
@csoulios csoulios added v8.5.0 and removed v8.5.1 labels Nov 1, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-merge-without-approval Automatically merge pull request when CI checks pass (NB doesn't wait for reviews!) backport :ml Machine learning >non-issue Team:ML Meta label for the ML team v8.5.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants