Skip to content

[7.x] [ML] Start gathering and storing inference stats (#53429)#54738

Merged
benwtrent merged 3 commits intoelastic:7.xfrom
benwtrent:backport/7.x/pr-53429
Apr 13, 2020
Merged

[7.x] [ML] Start gathering and storing inference stats (#53429)#54738
benwtrent merged 3 commits intoelastic:7.xfrom
benwtrent:backport/7.x/pr-53429

Conversation

@benwtrent
Copy link
Copy Markdown
Member

Backports the following commits to 7.x:

@benwtrent benwtrent added :ml Machine learning backport labels Apr 3, 2020
@elasticmachine
Copy link
Copy Markdown
Collaborator

Pinging @elastic/ml-core (:ml)

benwtrent and others added 2 commits April 13, 2020 07:18
This PR enables stats on inference to be gathered and stored in the `.ml-stats-*` indices.

Each node + model_id will have its own running stats document and these will later be summed together when returning _stats to the user.

`.ml-stats-*` is ILM managed (when possible). So, at any point the underlying index could change. This means that a stats document that is read in and then later updated will actually be a new doc in a new index. This complicates matters as this means that having a running knowledge of seq_no and primary_term is complicated and almost impossible. This is because we don't know the latest index name.

We should also strive for throughput, as this code sits in the middle of an ingest pipeline (or even a query).
@benwtrent benwtrent force-pushed the backport/7.x/pr-53429 branch from 67e735e to 8016271 Compare April 13, 2020 11:21
@benwtrent benwtrent merged commit c5c7ee9 into elastic:7.x Apr 13, 2020
@benwtrent benwtrent deleted the backport/7.x/pr-53429 branch April 13, 2020 12:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport :ml Machine learning

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants