[ML] Testing trained models in UI by jgowdyelastic · Pull Request #128359 · elastic/kibana

jgowdyelastic · 2022-03-23T11:37:20Z

Adds the ability to test some trained models using user specified input.
Currently supported models:

Lang ident - tested using a simulated ingest pipeline.
pytorch - tested by calling the infer trained model deployment API

This PR adds a wrapper for the es ingest pipeline simulate endpoint.
This has been added to ML because there is no simulate function shared from the ingest pipeline plugin. Also the endpoint supplied in the ingest pipeline plugin renames the docs parameter to documents making it incorrect if the user wished to copy the payload.

Action to open the testing flyout

Pytorch testing flyout

Lang ident testing flyout

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Unit or functional tests were updated or added to match the most common scenarios
This was checked for breaking API changes and was labeled appropriately

jgowdyelastic · 2022-03-24T09:43:47Z

@elasticmachine merge upstream

jgowdyelastic · 2022-03-28T10:12:54Z

@elasticmachine merge upstream

elasticmachine · 2022-03-28T10:14:21Z

Pinging @elastic/ml-ui (:ml)

x-pack/plugins/ml/public/application/trained_models/models_management/models_list.tsx

...ic/application/trained_models/models_management/test_models/models/lang_ident/lang_ident.tsx

darnautov · 2022-03-28T13:53:23Z

...lugins/ml/public/application/trained_models/models_management/test_models/models/ner/ner.tsx

+  RAW,
+}
+
+export const NerModel: FC<Props> = ({ model }) => {


This component looks very similar to the LandIdent one. I presume we should create a reusable component instead.

Updated in 09fbbd1

darnautov · 2022-03-28T13:53:57Z

.../public/application/trained_models/models_management/test_models/models/ner/ner_inference.ts

+  entity: estypes.MlTrainedModelEntities | null;
+}>;
+
+export class NerInference {


similar to the Lang Ident, I think it should be a Kibana endpoint

...ml/public/application/trained_models/models_management/test_models/models/ner/ner_output.tsx

peteharverson

Tested locally, and overall works really well! Left a few comments.

...ic/application/trained_models/models_management/test_models/models/lang_ident/lang_ident.tsx

...lugins/ml/public/application/trained_models/models_management/test_models/models/ner/ner.tsx

...ication/trained_models/models_management/test_models/models/lang_ident/lang_ident_output.tsx

peteharverson · 2022-03-28T16:52:22Z

...ml/public/application/trained_models/models_management/test_models/models/ner/ner_output.tsx

+                    id="xpack.ml.trainedModels.testModelsFlyout.ner.output.probabilityTitle"
+                    defaultMessage="Probability"
+                  />
+                  : {entity.class_probability}


Again with the exact value shown in the 'Raw output' tab. I'd consider formatting this to say 3 sigfigs.

Updated in c843fbf

...ml/public/application/trained_models/models_management/test_models/models/ner/ner_output.tsx

peteharverson · 2022-03-28T17:01:48Z

...ml/public/application/trained_models/models_management/test_models/models/ner/ner_output.tsx

+      return 'Location';
+
+    default:
+      return 'cross';


Is there another term that can be displayed for the default case, like other?

Updated in a012a58

MISC entities are now show like this:

Unknown entries will have the type Unknown and the same ? icon

peteharverson · 2022-03-28T17:04:18Z

...lugins/ml/public/application/trained_models/models_management/test_models/models/ner/ner.tsx

+    } catch (e) {
+      setIsRunning(false);
+      setOutput(null);
+      setErrorText(extractErrorMessage(e));


Is it worth checking the length of text entered, as with even a few lines of text I am getting a timeout [10s] waiting for inference result error.

I've increased the timeout to 30s dd2795f

Increasing to 30s has helped on my setup, although with long passages of text I am finding the network request stays in a pending state and never seems to time out.

darnautov

LGTM

jgowdyelastic · 2022-03-29T14:17:22Z

@elasticmachine merge upstream

kibana-ci · 2022-03-29T15:46:48Z

💚 Build Succeeded

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`ml`	1559	1574	+15

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`ml`	3.3MB	3.3MB	+10.8KB

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
`ml`	102	103	+1

Total ESLint disabled count

id	before	after	diff
`ml`	105	106	+1

History

💔 Build #34477 failed 3779977
💔 Build #34454 failed 27550ae
💔 Build #34441 failed dd2795f
💔 Build #34429 failed b3b311f
💔 Build #34413 failed 899e085
💔 Build #34404 failed c843fbf

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

peteharverson

Tested latest changes and LGTM.

As discussed, looks like the Elasticsearch side is not timing out properly when I try and pass a long string text when testing the NER model.

jgowdyelastic added 3 commits March 23, 2022 11:36

[ML] Testing trained models in UI

36a187b

folder rename

120b890

code clean up

95d523a

kibanamachine and others added 4 commits March 24, 2022 05:43

Merge branch 'main' into testing-trained-models-in-ui

87424df

translations

945343d

adding comments

865e980

endpoint comments

d307b58

Merge branch 'main' into testing-trained-models-in-ui

337ae34

jgowdyelastic requested review from darnautov and peteharverson March 28, 2022 10:13

jgowdyelastic added :ml backport:skip This PR does not require backporting release_note:feature Makes this part of the condensed release notes Feature:3rd Party Models ML 3rd party models ci:deploy-cloud v8.2.0 labels Mar 28, 2022

jgowdyelastic marked this pull request as ready for review March 28, 2022 10:14

jgowdyelastic requested a review from a team as a code owner March 28, 2022 10:14

darnautov reviewed Mar 28, 2022

View reviewed changes

jgowdyelastic added 2 commits March 28, 2022 17:07

small changes based on review

2a9f478

removing testing text

53125e5

peteharverson reviewed Mar 28, 2022

View reviewed changes

jgowdyelastic added 5 commits March 29, 2022 11:43

refactoring to remove duplicate code

09fbbd1

changing misc entities

a012a58

probably is now 3 sig figs

c843fbf

class refactor

0a733e5

another refactor

899e085

darnautov approved these changes Mar 29, 2022

View reviewed changes

jgowdyelastic added 7 commits March 29, 2022 13:25

fixing enitiy highlighting

b3b311f

adding infer timeout

dd2795f

show class name for known types

27550ae

refactoring highlighting

d26c011

moving unknown entity type

d27112c

removing default badge tooltips

3779977

fixing linting error

828b177

kibanamachine and others added 2 commits March 29, 2022 10:17

Merge branch 'main' into testing-trained-models-in-ui

78d54f9

small import changes

a64077f

peteharverson approved these changes Mar 29, 2022

View reviewed changes

jgowdyelastic merged commit f4ed8e1 into elastic:main Mar 29, 2022

jgowdyelastic deleted the testing-trained-models-in-ui branch March 29, 2022 16:06

jgowdyelastic mentioned this pull request Apr 27, 2022

[ML] Adds trained model testing for additional pytorch models #129209

Merged

4 tasks

lcawl mentioned this pull request Apr 27, 2022

[DOCS] Testing trained models in UI elastic/stack-docs#2119

Merged

tylersmalley added ci:cloud-deploy Create or update a Cloud deployment and removed ci:deploy-cloud labels Aug 17, 2022

Conversation

jgowdyelastic commented Mar 23, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

jgowdyelastic commented Mar 24, 2022

Uh oh!

jgowdyelastic commented Mar 28, 2022

Uh oh!

elasticmachine commented Mar 28, 2022

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

peteharverson left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jgowdyelastic Mar 29, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

jgowdyelastic Mar 29, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

darnautov left a comment

Choose a reason for hiding this comment

Uh oh!

jgowdyelastic commented Mar 29, 2022

Uh oh!

kibana-ci commented Mar 29, 2022

💚 Build Succeeded

Metrics [docs]

Module Count

Async chunks

ESLint disabled line counts

Total ESLint disabled count

History

Uh oh!

peteharverson left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

jgowdyelastic commented Mar 23, 2022 •

edited

Loading

jgowdyelastic Mar 29, 2022 •

edited

Loading

jgowdyelastic Mar 29, 2022 •

edited

Loading