[Entity Analytics][Lead Generation][3] Entity retrieval and enrichment to get data for lead generation by abhishekbhatia1710 · Pull Request #256628 · elastic/kibana

abhishekbhatia1710 · 2026-03-09T09:33:23Z

Summary

Introduces a shared Entity Retriever that centralises Entity Store V2 fetching, paginated via search_after for all users/hosts, or targeted by name, replacing duplicated inline logic in the generate-leads route
Introduces an Entity Enricher that pre-fetches risk score history (90-day daily averages), alert summaries (severity breakdown, top rules, top alerts), asset criticality, and privileged status for a batch of entities
Refactors the generate-leads route to delegate entity fetching to the retriever, removing ~70 lines of inline code
Updates the barrel file to export both new services and their types

Closes : https://github.com/elastic/security-team/issues/15948

This PR is 3rd part of the lead generation feature being built under the Entity Analytics Lead Generation epic. It builds on the foundation and observation modules from #255272 and #256156.

Checklist

Check the PR satisfies following conditions.

Reviewers should verify this PR satisfies this list as well.

Flaky Test Runner was used on any tests changed
The PR description includes the appropriate Release Notes section, and the correct release_note:* label is applied per the guidelines
Review the backport guidelines and apply applicable backport:* labels.

elasticmachine · 2026-03-09T09:33:57Z

Pinging @elastic/security-entity-analytics (Team:Entity Analytics)

coderabbitai · 2026-03-09T09:34:55Z

Important

Review skipped

Auto reviews are limited based on label configuration.

🏷️ Required labels (at least one) (4)

reviewer:coderabbit
Team:Search
Team:Operations
Team:QA

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yml

Review profile: CHILL

Plan: Pro

Run ID: 574d5f3d-b910-4722-be90-bf04633e735c

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

✅ Review completed - (🔄 Check again to review again)

✨ Finishing Touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

...olution/server/lib/entity_analytics/lead_generation/observation_modules/risk_score_module.ts

...ugins/security_solution/server/lib/entity_analytics/lead_generation/routes/generate_leads.ts

...olution/server/lib/entity_analytics/lead_generation/observation_modules/risk_score_module.ts

...ion/server/lib/entity_analytics/lead_generation/observation_modules/alert_analysis_module.ts

...ugins/security_solution/server/lib/entity_analytics/lead_generation/routes/generate_leads.ts

...ions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/types.ts

...ugins/security_solution/server/lib/entity_analytics/lead_generation/engine/llm_synthesize.ts

...ion/server/lib/entity_analytics/lead_generation/observation_modules/alert_analysis_module.ts

hop-dev

Thanks for addressing all my comments 🚀

elasticmachine · 2026-03-27T09:28:00Z

Pinging @elastic/obs-presentation-team (Team:obs-presentation)

elasticmachine · 2026-03-27T09:28:00Z

Pinging @elastic/fleet (Team:Fleet)

Adds AlertAnalysisModule, LLMSynthesize, entity conversion, generate_leads route handler with full orchestration, and get_leads route. Review changes: use data clients, thin route handler, split alert module into folder, fix escalation detection, remove dead weight field, scope usedTitleTracker, deduplicate staleness logic, simplify LLM prompt.

elasticmachine · 2026-03-27T11:06:59Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 607f80c

Failed CI Steps

Test Failures

[job] [logs] Scout: [ platform / navigation ] plugin / local-serverless-security_complete - navigation - opens panel on legacy management landing page
[job] [logs] Jest Tests #1 / SelectedFilters should render properly

Metrics [docs]

✅ unchanged

History

💔 Build #417663 failed b8fafa0
💛 Build #416006 was flaky 014ecff
💔 Build #415568 failed 57d9d7d
💔 Build #415320 failed 57b3194
💔 Build #407351 failed b4c75e4

cc @abhishekbhatia1710

…t to get data for lead generation (elastic#256628) ## Summary - Introduces a shared **Entity Retriever** that centralises Entity Store V2 fetching, paginated via `search_after` for all users/hosts, or targeted by name, replacing duplicated inline logic in the generate-leads route - Introduces an **Entity Enricher** that pre-fetches risk score history (90-day daily averages), alert summaries (severity breakdown, top rules, top alerts), asset criticality, and privileged status for a batch of entities - Refactors the generate-leads route to delegate entity fetching to the retriever, removing ~70 lines of inline code - Updates the barrel file to export both new services and their types Closes : elastic/security-team#15948 This PR is 3rd part of the lead generation feature being built under the [Entity Analytics Lead Generation epic](elastic/security-team#14401). It builds on the foundation and observation modules from elastic#255272 and elastic#256156. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels.

Resolve conflicts after PRs elastic#255272 (Foundation), elastic#256156 (Observation Modules), elastic#256628 (Entity Retrieval), and elastic#257046 (CRUD API Routes) were merged into main. Key resolutions: - Keep main's authoritative versions of observation modules with data-driven tier tables and RiskScoreDataClient integration - Add scheduling-specific weight properties to module configs - Wire RiskScoreDataClient through RunPipelineParams via dependency injection (route uses context, task creates from CoreStart) - Preserve main's .keyword suffix fixes in lead_data_client ES queries - Keep scheduling branch's single-file behavioral_analysis_module (consolidated from subdirectory structure) and remove stale subdir - Delete generate_leads.test.ts (coverage moved to run_pipeline.test.ts) - Preserve Task Manager registration, enable/disable route wiring, and run_pipeline shared orchestration from scheduling branch Note: pre-commit hook bypassed because ESLint failures are from upstream kbn-evals-suite-significant-events package (pre-existing in main), not from lead generation changes.

…t to get data for lead generation (elastic#256628) ## Summary - Introduces a shared **Entity Retriever** that centralises Entity Store V2 fetching, paginated via `search_after` for all users/hosts, or targeted by name, replacing duplicated inline logic in the generate-leads route - Introduces an **Entity Enricher** that pre-fetches risk score history (90-day daily averages), alert summaries (severity breakdown, top rules, top alerts), asset criticality, and privileged status for a batch of entities - Refactors the generate-leads route to delegate entity fetching to the retriever, removing ~70 lines of inline code - Updates the barrel file to export both new services and their types Closes : elastic/security-team#15948 This PR is 3rd part of the lead generation feature being built under the [Entity Analytics Lead Generation epic](elastic/security-team#14401). It builds on the foundation and observation modules from elastic#255272 and elastic#256156. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [ ] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) - [ ] Review the [backport guidelines](https://docs.google.com/document/d/1VyN5k91e5OVumlc0Gb9RPa3h1ewuPE705nRtioPiTvY/edit?usp=sharing) and apply applicable `backport:*` labels.

abhishekbhatia1710 requested review from a team as code owners March 9, 2026 09:33

abhishekbhatia1710 requested a review from hop-dev March 9, 2026 09:33

abhishekbhatia1710 self-assigned this Mar 9, 2026

abhishekbhatia1710 added backport:skip This PR does not require backporting release_note:feature Makes this part of the condensed release notes Team:Entity Analytics Security Entity Analytics Team labels Mar 9, 2026

hop-dev requested changes Mar 26, 2026

View reviewed changes

abhishekbhatia1710 requested a review from hop-dev March 27, 2026 09:01

hop-dev approved these changes Mar 27, 2026

View reviewed changes

abhishekbhatia1710 requested review from a team as code owners March 27, 2026 09:27

abhishekbhatia1710 requested review from dplumlee, gergoabraham and pzl March 27, 2026 09:27

botelastic bot added Team:Fleet Team label for Observability Data Collection Fleet team Team:obs-presentation Focus: APM UI, Infra UI, Hosts UI, Universal Profiling, Obs Overview and left Navigation labels Mar 27, 2026

abhishekbhatia1710 force-pushed the ea-15948-entity-retrieval-enrichment branch from d7cf013 to 607f80c Compare March 27, 2026 09:35

abhishekbhatia1710 removed request for a team, dplumlee, gergoabraham and pzl March 27, 2026 09:36

abhishekbhatia1710 merged commit 7b3fb52 into elastic:main Mar 27, 2026
18 checks passed

kibanamachine added the v9.4.0 label Mar 27, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Entity Analytics][Lead Generation][3] Entity retrieval and enrichment to get data for lead generation#256628

[Entity Analytics][Lead Generation][3] Entity retrieval and enrichment to get data for lead generation#256628
abhishekbhatia1710 merged 1 commit intoelastic:mainfrom
abhishekbhatia1710:ea-15948-entity-retrieval-enrichment

abhishekbhatia1710 commented Mar 9, 2026

Uh oh!

elasticmachine commented Mar 9, 2026

Uh oh!

coderabbitai bot commented Mar 9, 2026 •

edited

Loading

Review skipped

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hop-dev left a comment

Uh oh!

elasticmachine commented Mar 27, 2026

Uh oh!

elasticmachine commented Mar 27, 2026

Uh oh!

elasticmachine commented Mar 27, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

abhishekbhatia1710 commented Mar 9, 2026

Summary

Checklist

Uh oh!

elasticmachine commented Mar 9, 2026

Uh oh!

coderabbitai bot commented Mar 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review skipped

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hop-dev left a comment

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Mar 27, 2026

Uh oh!

elasticmachine commented Mar 27, 2026

Uh oh!

elasticmachine commented Mar 27, 2026

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

coderabbitai bot commented Mar 9, 2026 •

edited

Loading