[SecuritySolutions] Create Entity Store 'entities/list' API#192806
[SecuritySolutions] Create Entity Store 'entities/list' API#192806machadoum merged 13 commits intoelastic:mainfrom
Conversation
ea4953c to
7e39998
Compare
add API test Add data client test
b5266c3 to
fbb7479
Compare
|
Pinging @elastic/security-entity-analytics (Team:Entity Analytics) |
| paths: {} | ||
| components: | ||
| schemas: | ||
| UserEntityRecord: |
There was a problem hiding this comment.
We'll have to keep these schemas in sync with the entity definitions metadata fields, right?
I wonder if there's a way to represent that, maybe deduce the names of the fields from the generated zod records?
Just pointing it out for the future, I don't necessarily think we need to include that in this PR
There was a problem hiding this comment.
Yes, unfortunately, we need to keep the API schema and Entity Store schema in sync.
I couldn't figure out a good way to do that automatically. 😞
tiansivive
left a comment
There was a problem hiding this comment.
LGTM! Good stuff! 🎉
I have a couple minor comments I think are worth some discussion, even if I don't necessarily think they're blocking the PR
jbudz
left a comment
There was a problem hiding this comment.
.buildkite/ftr_security_configs.yml
|
@elasticmachine merge upstream |
|
@elasticmachine merge upstream |
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]Module Count
Async chunks
History
To update your PR or re-run it, just comment with: cc @machadoum |
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
…192806) This PR introduces the following API routes for listing Entity Store "entities": <meta charset="utf-8"><b style="font-weight:normal;" id="docs-internal-guid-9410c5d7-7fff-e873-6830-887939a306fb"><div dir="ltr" style="margin-left:-0.75pt;" align="left"> List Entities | GET /api/entity_store/entities/list -- | -- </div></b> The PR includes the following: - The OpenAPI schemas for the route - The actual Kibana side endpoint - Add searchEntities function to the `EntityStoreDataClient` ### How to test 1. Add some host/user data * Easiest is to use [elastic/security-data-generator](https://github.com/elastic/security-documents-generator) 2. Make sure to add `entityStoreEnabled` under `xpack.securitySolution.enableExperimental` in your `kibana.dev.yml` 3. In kibana dev tools or your terminal, call the `INIT` route for either `user` or `host`. 4. You should now see 2 transforms in kibana. Make sure to re-trigger them if needed so they process the documents. 5. Call the new API, and it should return entities Implements elastic/security-team#10517 ### Checklist - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
…92806) (#193562) # Backport This will backport the following commits from `main` to `8.x`: - [[SecuritySolutions] Create Entity Store 'entities/list' API (#192806)](#192806) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Pablo Machado","email":"pablo.nevesmachado@elastic.co"},"sourceCommit":{"committedDate":"2024-09-19T12:54:53Z","message":"[SecuritySolutions] Create Entity Store 'entities/list' API (#192806)\n\nThis PR introduces the following API routes for listing Entity Store\r\n\"entities\":\r\n\r\n<meta charset=\"utf-8\"><b style=\"font-weight:normal;\"\r\nid=\"docs-internal-guid-9410c5d7-7fff-e873-6830-887939a306fb\"><div\r\ndir=\"ltr\" style=\"margin-left:-0.75pt;\" align=\"left\">\r\nList Entities | GET /api/entity_store/entities/list\r\n-- | --\r\n</div></b>\r\n\r\nThe PR includes the following:\r\n - The OpenAPI schemas for the route\r\n - The actual Kibana side endpoint\r\n - Add searchEntities function to the `EntityStoreDataClient`\r\n \r\n\r\n### How to test\r\n\r\n1. Add some host/user data\r\n* Easiest is to use\r\n[elastic/security-data-generator](https://github.com/elastic/security-documents-generator)\r\n2. Make sure to add `entityStoreEnabled` under\r\n`xpack.securitySolution.enableExperimental` in your `kibana.dev.yml`\r\n3. In kibana dev tools or your terminal, call the `INIT` route for\r\neither `user` or `host`.\r\n4. You should now see 2 transforms in kibana. Make sure to re-trigger\r\nthem if needed so they process the documents.\r\n5. Call the new API, and it should return entities \r\n\r\n\r\n\r\nImplements https://github.com/elastic/security-team/issues/10517\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"27f5da436b70da1a3743ee99c54d8159918b40de","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["v9.0.0","release_note:feature","backport:prev-minor","Theme: entity_analytics","Feature:Entity Analytics","Team:Entity Analytics","v8.16.0"],"number":192806,"url":"https://github.com/elastic/kibana/pull/192806","mergeCommit":{"message":"[SecuritySolutions] Create Entity Store 'entities/list' API (#192806)\n\nThis PR introduces the following API routes for listing Entity Store\r\n\"entities\":\r\n\r\n<meta charset=\"utf-8\"><b style=\"font-weight:normal;\"\r\nid=\"docs-internal-guid-9410c5d7-7fff-e873-6830-887939a306fb\"><div\r\ndir=\"ltr\" style=\"margin-left:-0.75pt;\" align=\"left\">\r\nList Entities | GET /api/entity_store/entities/list\r\n-- | --\r\n</div></b>\r\n\r\nThe PR includes the following:\r\n - The OpenAPI schemas for the route\r\n - The actual Kibana side endpoint\r\n - Add searchEntities function to the `EntityStoreDataClient`\r\n \r\n\r\n### How to test\r\n\r\n1. Add some host/user data\r\n* Easiest is to use\r\n[elastic/security-data-generator](https://github.com/elastic/security-documents-generator)\r\n2. Make sure to add `entityStoreEnabled` under\r\n`xpack.securitySolution.enableExperimental` in your `kibana.dev.yml`\r\n3. In kibana dev tools or your terminal, call the `INIT` route for\r\neither `user` or `host`.\r\n4. You should now see 2 transforms in kibana. Make sure to re-trigger\r\nthem if needed so they process the documents.\r\n5. Call the new API, and it should return entities \r\n\r\n\r\n\r\nImplements https://github.com/elastic/security-team/issues/10517\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"27f5da436b70da1a3743ee99c54d8159918b40de"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/192806","number":192806,"mergeCommit":{"message":"[SecuritySolutions] Create Entity Store 'entities/list' API (#192806)\n\nThis PR introduces the following API routes for listing Entity Store\r\n\"entities\":\r\n\r\n<meta charset=\"utf-8\"><b style=\"font-weight:normal;\"\r\nid=\"docs-internal-guid-9410c5d7-7fff-e873-6830-887939a306fb\"><div\r\ndir=\"ltr\" style=\"margin-left:-0.75pt;\" align=\"left\">\r\nList Entities | GET /api/entity_store/entities/list\r\n-- | --\r\n</div></b>\r\n\r\nThe PR includes the following:\r\n - The OpenAPI schemas for the route\r\n - The actual Kibana side endpoint\r\n - Add searchEntities function to the `EntityStoreDataClient`\r\n \r\n\r\n### How to test\r\n\r\n1. Add some host/user data\r\n* Easiest is to use\r\n[elastic/security-data-generator](https://github.com/elastic/security-documents-generator)\r\n2. Make sure to add `entityStoreEnabled` under\r\n`xpack.securitySolution.enableExperimental` in your `kibana.dev.yml`\r\n3. In kibana dev tools or your terminal, call the `INIT` route for\r\neither `user` or `host`.\r\n4. You should now see 2 transforms in kibana. Make sure to re-trigger\r\nthem if needed so they process the documents.\r\n5. Call the new API, and it should return entities \r\n\r\n\r\n\r\nImplements https://github.com/elastic/security-team/issues/10517\r\n\r\n### Checklist\r\n\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>","sha":"27f5da436b70da1a3743ee99c54d8159918b40de"}},{"branch":"8.x","label":"v8.16.0","labelRegex":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
This PR introduces the following API routes for listing Entity Store "entities":
The PR includes the following:
EntityStoreDataClientHow to test
entityStoreEnabledunderxpack.securitySolution.enableExperimentalin yourkibana.dev.ymlINITroute for eitheruserorhost.Implements https://github.com/elastic/security-team/issues/10517
Checklist