[Entity Analytics] [Entity Store] Refactor entity store enablement (server side)#199638
Merged
tiansivive merged 14 commits intoelastic:mainfrom Nov 19, 2024
Merged
Conversation
tiansivive
commented
Nov 12, 2024
| return { engines, succeeded: true }; | ||
| } | ||
|
|
||
| public async status(): Promise<GetEntityStoreStatusResponse> { |
Contributor
Author
There was a problem hiding this comment.
Is this the correct way to compute the status?
What should we do in those "in between" states, where, for example, not all engines are stopped?
Maybe worry about it later if we ever decide allowing individual engine setup?
Contributor
There was a problem hiding this comment.
What should we do in those "in between" states, where, for example, not all engines are stopped?
is the current response "running"? I think thats fine
Contributor
|
Pinging @elastic/security-entity-analytics (Team:Entity Analytics) |
hop-dev
reviewed
Nov 14, 2024
Contributor
There was a problem hiding this comment.
I think this should be handled inside createRiskScoreLatestIndex what do you think?
Contributor
Author
There was a problem hiding this comment.
Not sure...
I thought about it for a while, and it does fit with our overall approach but I'm worried it might be too strict of an abstraction. For example, if we ever get some feature where it is important to handle these errors (maybe we want to show something in the UI) we would be stuck 🤔
In general I think it's probably best to leave each client/consumer handle the errors. Since this is such a common way for us to handle this particular error, maybe we can have some utility fn to reduce the boilerplate?
Contributor
|
Tested locally, working great 🚀 we just need to add an integration test for the status endpoint |
10d402c to
b8dae7a
Compare
Contributor
💛 Build succeeded, but was flaky
Failed CI StepsTest Failures
Metrics [docs]Async chunks
History
|
hop-dev
approved these changes
Nov 19, 2024
Contributor
Flaky Test Runner Stats🟠 Some tests failed. - kibana-flaky-test-suite-runner#7420[❌] x-pack/test/security_solution_api_integration/test_suites/entity_analytics/entity_store/trial_license_complete_tier/configs/ess.config.ts: 60/100 tests passed. |
Contributor
|
Starting backport for target branches: 8.x |
Contributor
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
Contributor
Author
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
tiansivive
added a commit
to tiansivive/kibana
that referenced
this pull request
Nov 20, 2024
…erver side) (elastic#199638) ## Summary This PR adds 2 new endpoints regarding enablement of the Entity Store: * `api/entity_store/enable`, which initializes entity engines for both `user` and `host` entities * `api/entity_store/status`, which computes a global store status based on the individual engine status In addition, running initialization of multiple engines in parallel is now allowed. ### How to test 1. Use dev tools to call `POST kbn:/api/entity_store/enable` 2. Check that two engines were created and that the status is `installing` by calling `GET kbn:/api/entity_store/status` 3. Wait a few seconds and keep calling the `status` endpoint. Once initialization finishes, the status should switch to `running` --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 3757e64) # Conflicts: # oas_docs/output/kibana.serverless.yaml # oas_docs/output/kibana.yaml
Contributor
Author
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
tiansivive
added a commit
to tiansivive/kibana
that referenced
this pull request
Nov 20, 2024
…erver side) (elastic#199638) ## Summary This PR adds 2 new endpoints regarding enablement of the Entity Store: * `api/entity_store/enable`, which initializes entity engines for both `user` and `host` entities * `api/entity_store/status`, which computes a global store status based on the individual engine status In addition, running initialization of multiple engines in parallel is now allowed. ### How to test 1. Use dev tools to call `POST kbn:/api/entity_store/enable` 2. Check that two engines were created and that the status is `installing` by calling `GET kbn:/api/entity_store/status` 3. Wait a few seconds and keep calling the `status` endpoint. Once initialization finishes, the status should switch to `running` --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> (cherry picked from commit 3757e64) # Conflicts: # oas_docs/output/kibana.serverless.yaml # oas_docs/output/kibana.yaml
tiansivive
added a commit
that referenced
this pull request
Nov 21, 2024
…ent (server side) (#199638) (#200875) # Backport This will backport the following commits from `main` to `8.x`: - [[Entity Analytics] [Entity Store] Refactor entity store enablement (server side) (#199638)](#199638) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Tiago Vila Verde","email":"tiago.vilaverde@elastic.co"},"sourceCommit":{"committedDate":"2024-11-19T13:11:24Z","message":"[Entity Analytics] [Entity Store] Refactor entity store enablement (server side) (#199638)\n\n## Summary\r\n\r\nThis PR adds 2 new endpoints regarding enablement of the Entity Store:\r\n* `api/entity_store/enable`, which initializes entity engines for both\r\n`user` and `host` entities\r\n* `api/entity_store/status`, which computes a global store status based\r\non the individual engine status\r\n\r\nIn addition, running initialization of multiple engines in parallel is\r\nnow allowed.\r\n\r\n\r\n### How to test\r\n\r\n1. Use dev tools to call `POST kbn:/api/entity_store/enable`\r\n2. Check that two engines were created and that the status is\r\n`installing` by calling `GET kbn:/api/entity_store/status`\r\n3. Wait a few seconds and keep calling the `status` endpoint. Once\r\ninitialization finishes, the status should switch to `running`\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"3757e641278a5186919e35a0f980ac3cda7e8ccd","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["technical debt","release_note:skip","v9.0.0","Team:Entity Analytics","backport:version","v8.17.0"],"number":199638,"url":"https://github.com/elastic/kibana/pull/199638","mergeCommit":{"message":"[Entity Analytics] [Entity Store] Refactor entity store enablement (server side) (#199638)\n\n## Summary\r\n\r\nThis PR adds 2 new endpoints regarding enablement of the Entity Store:\r\n* `api/entity_store/enable`, which initializes entity engines for both\r\n`user` and `host` entities\r\n* `api/entity_store/status`, which computes a global store status based\r\non the individual engine status\r\n\r\nIn addition, running initialization of multiple engines in parallel is\r\nnow allowed.\r\n\r\n\r\n### How to test\r\n\r\n1. Use dev tools to call `POST kbn:/api/entity_store/enable`\r\n2. Check that two engines were created and that the status is\r\n`installing` by calling `GET kbn:/api/entity_store/status`\r\n3. Wait a few seconds and keep calling the `status` endpoint. Once\r\ninitialization finishes, the status should switch to `running`\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"3757e641278a5186919e35a0f980ac3cda7e8ccd"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/199638","number":199638,"mergeCommit":{"message":"[Entity Analytics] [Entity Store] Refactor entity store enablement (server side) (#199638)\n\n## Summary\r\n\r\nThis PR adds 2 new endpoints regarding enablement of the Entity Store:\r\n* `api/entity_store/enable`, which initializes entity engines for both\r\n`user` and `host` entities\r\n* `api/entity_store/status`, which computes a global store status based\r\non the individual engine status\r\n\r\nIn addition, running initialization of multiple engines in parallel is\r\nnow allowed.\r\n\r\n\r\n### How to test\r\n\r\n1. Use dev tools to call `POST kbn:/api/entity_store/enable`\r\n2. Check that two engines were created and that the status is\r\n`installing` by calling `GET kbn:/api/entity_store/status`\r\n3. Wait a few seconds and keep calling the `status` endpoint. Once\r\ninitialization finishes, the status should switch to `running`\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"3757e641278a5186919e35a0f980ac3cda7e8ccd"}},{"branch":"8.x","label":"v8.17.0","labelRegex":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
paulinashakirova
pushed a commit
to paulinashakirova/kibana
that referenced
this pull request
Nov 26, 2024
…erver side) (elastic#199638) ## Summary This PR adds 2 new endpoints regarding enablement of the Entity Store: * `api/entity_store/enable`, which initializes entity engines for both `user` and `host` entities * `api/entity_store/status`, which computes a global store status based on the individual engine status In addition, running initialization of multiple engines in parallel is now allowed. ### How to test 1. Use dev tools to call `POST kbn:/api/entity_store/enable` 2. Check that two engines were created and that the status is `installing` by calling `GET kbn:/api/entity_store/status` 3. Wait a few seconds and keep calling the `status` endpoint. Once initialization finishes, the status should switch to `running` --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Dec 12, 2024
…erver side) (elastic#199638) ## Summary This PR adds 2 new endpoints regarding enablement of the Entity Store: * `api/entity_store/enable`, which initializes entity engines for both `user` and `host` entities * `api/entity_store/status`, which computes a global store status based on the individual engine status In addition, running initialization of multiple engines in parallel is now allowed. ### How to test 1. Use dev tools to call `POST kbn:/api/entity_store/enable` 2. Check that two engines were created and that the status is `installing` by calling `GET kbn:/api/entity_store/status` 3. Wait a few seconds and keep calling the `status` endpoint. Once initialization finishes, the status should switch to `running` --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
5 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR adds 2 new endpoints regarding enablement of the Entity Store:
api/entity_store/enable, which initializes entity engines for bothuserandhostentitiesapi/entity_store/status, which computes a global store status based on the individual engine statusIn addition, running initialization of multiple engines in parallel is now allowed.
How to test
POST kbn:/api/entity_store/enableinstallingby callingGET kbn:/api/entity_store/statusstatusendpoint. Once initialization finishes, the status should switch torunning