[Entity Analytics][Leads generation][4] Add API routes, LeadDataClient, and async generation#257046
Conversation
|
Cloud deployments require a Github label, please add |
📝 WalkthroughWalkthroughThis pull request introduces comprehensive lead generation functionality for entity analytics in the Security Solution. It adds a complete pipeline: entity retrieval and enrichment from the Entity Store, a configurable lead generation engine that orchestrates observation modules (risk scoring, temporal state analysis, behavioral/alert analysis), weighted priority scoring with bonuses, and LLM-based content synthesis. Supporting infrastructure includes Elasticsearch-backed persistence via a lead data client, REST API endpoints for CRUD and status operations, and constant definitions for URL routes. The engine collects observations from registered modules, computes entity priorities, enriches leads with alert/risk context, and synthesizes human-readable narratives. All components are integrated into the entity analytics route registry behind an experimental feature flag. ✨ Finishing Touches🧪 Generate unit tests (beta)
Warning Review ran into problems🔥 ProblemsGit: Failed to clone repository. Please run the Comment |
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.
Inline comments:
In
`@x-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/lead_data_client.ts`:
- Around line 203-224: The bulk response from esClient.bulk is not being
inspected, so partial failures (response.body.errors === true or item-level
errors) can occur and then deleteByQuery runs and removes existing docs; modify
the block where esClient.bulk(...) is awaited (the code that builds bulkBody
from leads and calls leadToEsDoc) to capture the bulk response, check
response.body?.errors and iterate response.body.items to collect/ log any
item-level errors (include lead id, status, error), and if any errors exist
throw or return early so the subsequent esClient.deleteByQuery(...) does not
execute; ensure logs reference executionId and indexName for context.
- Around line 301-320: The updateByQuery call in lead_data_client.ts builds a
Painless script by interpolating values from updates into the script source,
which risks script injection; change esClient.updateByQuery to use a
parameterized script instead: construct the script source to assign from params
(e.g., for each entry in updates generate a fixed snippet like
ctx._source['<field>'] = params['p<i>']) and build a corresponding params object
mapping p<i> to the runtime value, then pass that params object to the
updateByQuery call; reference the existing variables esClient.updateByQuery,
updates, id and the surrounding try/catch to locate and replace the unsafe
string interpolation with a params-based script approach similar to
bulkUpdateLeads.
In
`@x-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/observation_modules/temporal_state_module.ts`:
- Around line 95-133: The current aggregation groups by `${entityType}.name` and
treats missing `entity.attributes.privileged` as "previously non-privileged",
which can produce false escalations; change the ES query and downstream checks
to aggregate on a stable identifier (e.g., `${entityType}.id`) or a composite
key (id+name) in `esClient.search`'s `aggs.by_entity.terms.field`, include
`entity.id` and `entity.attributes.privileged` in the `top_hits` `_source`
(oldest_snapshot), and then only mark escalation when the hit's
`entity.attributes` explicitly exists and `privileged === false` (not
undefined); also use the stable `entity.id` (from `hit._source.entity.id`) when
adding to `escalated.add` instead of using `bucket.key` if it is a name to avoid
name reuse skew.
In
`@x-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/enable_lead_generation.ts`:
- Around line 41-46: The endpoint currently returns response.ok({ success: true
}) inside the try block in the enable_lead_generation handler (file
enable_lead_generation.ts), which falsely signals success even though Task
Manager wiring is TODO; change the response to return an HTTP 501 Not
Implemented (use the route response helper, e.g., response.customError or
response.notImplemented with a clear message like "Lead generation enable not
implemented — Task Manager wiring pending (`#15955`)") instead of success; keep
the logger.info line but ensure the handler returns 501 and apply the same
change to the corresponding disable route handler so both endpoints accurately
report not-implemented status.
In
`@x-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/generate_leads.ts`:
- Around line 60-135: The background pipeline captures a request-scoped
Elasticsearch client using asCurrentUser (esClient) which may become invalid
after response returns; replace the request-scoped client usage in this async
fire-and-forget block by obtaining core.elasticsearch.client.asInternalUser (use
asInternalUser for system-level operations) before launching the background task
and pass that internal client to createCRUDClient, createLeadGenerationEngine
modules (createRiskScoreModule, createTemporalStateModule,
createBehavioralAnalysisModule), and createLeadDataClient; alternatively,
refactor to schedule the work via Task Manager instead of running an in-request
background async IIFE so no request-scoped client (asCurrentUser) is used.
In
`@x-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/get_lead_generation_status.ts`:
- Around line 33-50: The status endpoint is returning global space-level status
and ignores the execution that started the job; update the route added in
get_lead_generation_status.ts so it accepts an execution identifier (e.g.,
executionUuid) in the request (query or body) and forward that identifier into
the lead data lookup instead of the current global call: extend the route
validation to require executionUuid, extract it from the incoming request, and
call createLeadDataClient(...).getStatus(executionUuid) (or equivalent method
overload) so getStatus is scoped to that executionUuid; alternatively, if you
intend single-flight only, enforce and document single-generation semantics in
createLeadDataClient/getStatus and reject additional POST /generate requests
when a run is active.
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Run ID: c55374f8-3576-40bb-8502-26c9d6111de9
📒 Files selected for processing (39)
x-pack/solutions/security/plugins/security_solution/common/entity_analytics/constants.tsx-pack/solutions/security/plugins/security_solution/common/entity_analytics/lead_generation/constants.tsx-pack/solutions/security/plugins/security_solution/common/entity_analytics/lead_generation/types.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/engine/index.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/engine/lead_generation_engine.test.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/engine/lead_generation_engine.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/engine/llm_synthesize.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/entity_conversion.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/entity_enricher.test.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/entity_enricher.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/entity_retriever.test.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/entity_retriever.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/index.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/lead_data_client.test.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/lead_data_client.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/observation_modules/alert_analysis_module.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/observation_modules/index.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/observation_modules/risk_score_module.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/observation_modules/temporal_state_module.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/observation_modules/utils.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/bulk_update_leads.test.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/bulk_update_leads.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/disable_lead_generation.test.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/disable_lead_generation.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/dismiss_lead.test.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/dismiss_lead.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/enable_lead_generation.test.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/enable_lead_generation.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/generate_leads.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/get_lead_by_id.test.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/get_lead_by_id.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/get_lead_generation_status.test.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/get_lead_generation_status.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/get_leads.test.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/get_leads.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/routes/register_lead_generation_routes.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/lead_generation/types.tsx-pack/solutions/security/plugins/security_solution/server/lib/entity_analytics/register_entity_analytics_routes.tsx-pack/solutions/security/plugins/security_solution/server/plugin_contract.ts
...ty/plugins/security_solution/server/lib/entity_analytics/lead_generation/lead_data_client.ts
Show resolved
Hide resolved
...ty/plugins/security_solution/server/lib/entity_analytics/lead_generation/lead_data_client.ts
Show resolved
Hide resolved
...ion/server/lib/entity_analytics/lead_generation/observation_modules/temporal_state_module.ts
Show resolved
Hide resolved
...curity_solution/server/lib/entity_analytics/lead_generation/routes/enable_lead_generation.ts
Show resolved
Hide resolved
...ugins/security_solution/server/lib/entity_analytics/lead_generation/routes/generate_leads.ts
Show resolved
Hide resolved
...ty_solution/server/lib/entity_analytics/lead_generation/routes/get_lead_generation_status.ts
Show resolved
Hide resolved
…and async generation Adds LeadDataClient abstraction (CRUD on leads index), async generate endpoint, six new routes (get by id, status, dismiss, bulk update, enable, disable), entity conversion/retrieval/enricher services, LLM synthesis, weighted scoring engine, and observation module refinements. Review fixes: bulk error check before stale cleanup, parameterized Painless scripts, privilege guard in temporal state module, enterprise license gating on all routes, EntityStoreEntity type derivation.
f437e90 to
73cd65e
Compare
- Remove 5 unused Zod response schemas and 7 dead inferred type exports from common/lead_generation/types.ts (never imported anywhere) - Remove 3 dead PATTERN_CATALOG entries (investigation_status, watchlist_inclusion, risk_escalation) for observation types no module produces - Remove unused function parameters (_config, _group, _pattern) and inline _observationTypes into buildTags - Remove deprecated createAlertAnalysisModule alias and its exports
...curity_solution/server/lib/entity_analytics/lead_generation/engine/lead_generation_engine.ts
Outdated
Show resolved
Hide resolved
...ugins/security_solution/server/lib/entity_analytics/lead_generation/engine/llm_synthesize.ts
Outdated
Show resolved
Hide resolved
...ion/server/lib/entity_analytics/lead_generation/observation_modules/alert_analysis_module.ts
Outdated
Show resolved
Hide resolved
.../lib/entity_analytics/lead_generation/observation_modules/behavioral_analysis_module.test.ts
Outdated
Show resolved
Hide resolved
...on/server/lib/entity_analytics/lead_generation/observation_modules/risk_score_module.test.ts
Outdated
Show resolved
Hide resolved
…n LLM prompt - Replace `computeStaleness(now, now)` with literal `'fresh'` since staleness is recalculated at read time in LeadDataClient - Add anti-hallucination instruction for rule names in LLM synthesis prompt - Rename alert_analysis_module/ to behavioral_analysis_module/ to match MODULE_ID = 'behavioral_analysis' (restoring consistency from PR elastic#256156) - Remove deprecated createAlertAnalysisModule alias - Update all import paths to reference behavioral_analysis_module
ymao1
left a comment
There was a problem hiding this comment.
Initial code review looks good! Will desk test the API next
...ugins/security_solution/server/lib/entity_analytics/lead_generation/routes/generate_leads.ts
Show resolved
Hide resolved
...ty/plugins/security_solution/server/lib/entity_analytics/lead_generation/lead_data_client.ts
Outdated
Show resolved
Hide resolved
...ugins/security_solution/server/lib/entity_analytics/lead_generation/routes/generate_leads.ts
Outdated
Show resolved
Hide resolved
...ty/plugins/security_solution/server/lib/entity_analytics/lead_generation/lead_data_client.ts
Outdated
Show resolved
Hide resolved
...ty/plugins/security_solution/server/lib/entity_analytics/lead_generation/entity_retriever.ts
Outdated
Show resolved
Hide resolved
...ity/plugins/security_solution/server/lib/entity_analytics/lead_generation/entity_enricher.ts
Outdated
Show resolved
Hide resolved
|
I see there is a Should we update to use the Also the entity conversion in |
- Wrap generate_leads route handler with withMinimumLicense (was the
only route missing enterprise license gating)
- Pass executionUuid from route into service.generate() so the UUID
returned to the client matches the one stored on persisted leads
- Improve findLeads error logging: distinguish index_not_found (debug)
from real errors (error level), extract error message to avoid
[object Object] in logs
- Let bulkUpdateLeads throw on ES errors so the route handler returns
a proper error response instead of silently returning { updated: 0 }
- Remove dead entity_retriever.ts and entity_enricher.ts (plus tests);
entity retrieval uses entityStoreDataClient.fetchAllUnifiedLatestEntities
and observation modules handle their own data fetching
- Use entity.EngineMetadata.Type (V2 canonical field) over entity.type
in entityRecordToLeadEntity, falling back to entity.type for compat
|
Addressed in the latest push: CRUDClient.listEntities() : Switched from The service no longer takes |
…ent.listEntities() Replace EntityStoreDataClient.fetchAllUnifiedLatestEntities() with the entity-store plugin's CRUDClient.listEntities() API for V2 unified index access. This uses the canonical, properly-owned API from @kbn/entity-store with search_after pagination. - Add fetchAllLeadEntities() paginated helper in entity_conversion.ts - Refactor createLeadGenerationService to accept a fetchEntities function instead of entityStoreDataClient (easier to test, decoupled) - Wire getStartServices -> entityStore.createCRUDClient in generate route - Remove ENTITY_SOURCE_FIELDS (CRUDClient returns full docs) - Add tests for fetchAllLeadEntities pagination logic
# Conflicts: # x-pack/solutions/security/plugins/security_solution/server/plugin_contract.ts
…ibana into ea-15953-api-routes
|
I self desk-tested this PR, with manually adding the changes of entity store crud client sorting done as part of the PR : #259386. Waiting for it to get merged, but this PR can be reviewed. |
...y/plugins/security_solution/server/lib/entity_analytics/lead_generation/entity_conversion.ts
Show resolved
Hide resolved
jaredburgettelastic
left a comment
There was a problem hiding this comment.
👍 Great job, the code is very straightforward!
persistLeads wrote camelCase fields (executionId, chatRecommendations)
but findLeads/esDocToLead expected snake_case (execution_uuid,
chat_recommendations), causing silent data loss. Added toSnakeCaseDoc
transform so the service persists docs in the same format as the data
client.
The deleteByQuery used { term: { executionId } } but ES dynamically
maps string fields as text — term queries on text fields don't match
UUIDs (tokenized at hyphens), so must_not matched every document and
deleted all newly-inserted leads.
All term/terms queries on id, status, and execution_uuid now target the
.keyword sub-field so they work correctly against the default dynamic
text+keyword mapping.
💛 Build succeeded, but was flaky
Failed CI StepsMetrics [docs]Unknown metric groupsReferences to deprecated APIs
Unreferenced deprecated APIs
History
|
…e_for_children6 * commit '3402744f63ca1196e97b11ffac4e7f7efab240df': (80 commits) [PerUserAuth] Add EARS auth type for Connectors V2 (elastic#253695) Fix `@elastic/eui/require-aria-label-for-modals` lint violations across `@elastic/kibana-core` files (elastic#259757) [Entity Analytics][Leads generation][4] Add API routes, LeadDataClient, and async generation (elastic#257046) [Agent Builder] Agent-centric UX redesign (elastic#258005) fix query streams failing test (elastic#260277) [Lens as code] Add list layout to the new API (elastic#259967) [FTR] Add warning comments to deployment-agnostic FTR base configs (elastic#260018) [Discover][Logs profile] Fix missing search highlights (elastic#260056) Plugin system: safe deletion (elastic#259038) [Infra] Fix Hosts filter options to match selected schema (elastic#259825) Manual Entity Resolution and flyout representation (elastic#260162) [Cascade] Handle grouping on fields with unset values (elastic#260033) [Fleet] generate OTel config for integration packages with otelcol inputs (elastic#259968) [Search] Switch over to V2 index management details (elastic#259866) [inference] increase timeout for ES inference calls (elastic#260382) [ES|QL] Enable subqueries (elastic#257455) [ES|QL] Change Point order free options (elastic#260282) [Auth] Added authentication strategy for UIAM OAuth (elastic#256182) [Security Solution] Add "alerts_candidate_count" rule execution metric (elastic#259917) [api-docs] 2026-03-31 Daily api_docs build (elastic#260380) ...
Resolve conflicts after PRs elastic#255272 (Foundation), elastic#256156 (Observation Modules), elastic#256628 (Entity Retrieval), and elastic#257046 (CRUD API Routes) were merged into main. Key resolutions: - Keep main's authoritative versions of observation modules with data-driven tier tables and RiskScoreDataClient integration - Add scheduling-specific weight properties to module configs - Wire RiskScoreDataClient through RunPipelineParams via dependency injection (route uses context, task creates from CoreStart) - Preserve main's .keyword suffix fixes in lead_data_client ES queries - Keep scheduling branch's single-file behavioral_analysis_module (consolidated from subdirectory structure) and remove stale subdir - Delete generate_leads.test.ts (coverage moved to run_pipeline.test.ts) - Preserve Task Manager registration, enable/disable route wiring, and run_pipeline shared orchestration from scheduling branch Note: pre-commit hook bypassed because ESLint failures are from upstream kbn-evals-suite-significant-events package (pre-existing in main), not from lead generation changes.
…t, and async generation (elastic#257046) ## Summary - **LeadDataClient abstraction** : Extracted all Elasticsearch CRUD operations (create, find, get-by-id, dismiss, bulk-update, status, delete) into a dedicated `LeadDataClient` with snake_case ↔ camelCase transform layer between index mappings and API types - **Async generate endpoint** — Refactored `POST /generate` to return an `executionUuid` immediately and run the pipeline (entity retrieval -> observation modules -> scoring -> synthesis -> persistence) in the background - **Six new API routes** : Added `GET /{id}`, `GET /status`, `POST /{id}/_dismiss`, `POST /bulk_update`, `POST /enable`, and `POST /disable` with Zod-validated schemas, versioned routing, and proper authorization - **Shared Zod schemas** : Moved all request/response schemas to common types (generateLeadsRequestSchema, findLeadsRequestSchema, getLeadByIdRequestSchema, dismissLeadRequestSchema, bulkUpdateLeadsRequestSchema, generateLeadsResponseSchema) - **Refactored GET /leads** : Now delegates to `LeadDataClient.findLeads` with page/perPage/sortField/sortOrder/status query params - **Unit tests** : Added tests for LeadDataClient (createLeads, findLeads, getLeadById, dismissLead, bulkUpdateLeads, getStatus, deleteAllLeads) and all route handlers (get_leads, get_lead_by_id, get_lead_generation_status, dismiss_lead, bulk_update_leads, enable, disable) Enable and disable routes are placeholder stubs pending Task Manager wiring (elastic#15955). Relates to: elastic/security-team#15953 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Flaky Test Runner was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the guidelines - [ ] Review the backport guidelines and apply applicable `backport:*` labels. ## Testing Steps **Manual verification** Start Kibana locally and test the API endpoints below. ## API Examples ### Generate leads (async) ```bash curl -X POST "http://localhost:5601/internal/entity_analytics/leads/generate" \ -H "kbn-xsrf: true" \ -H "Content-Type: application/json" \ -H "elastic-api-version: 1" \ -u <....> \ -d '{"connectorId": "optional-connector-id"}' ``` Returns: `{ "executionUuid": "abc-123-..." }` ### Find leads ```bash curl -X GET "http://localhost:5601/internal/entity_analytics/leads?page=1&perPage=10&sortField=priority&sortOrder=desc" \ -H "kbn-xsrf: true" \ -H "elastic-api-version: 1" \ -u <....> ``` Returns: `{ "leads": [...], "total": 42, "page": 1, "perPage": 10 }` ### Get lead by ID ```bash curl -X GET "http://localhost:5601/internal/entity_analytics/leads/{id}" \ -H "kbn-xsrf: true" \ -H "elastic-api-version: 1" \ -u <....> ``` Returns: full `leadSchema` object ### Get status ```bash curl -X GET "http://localhost:5601/internal/entity_analytics/leads/status" \ -H "kbn-xsrf: true" \ -H "elastic-api-version: 1" \ -u <....> ``` Returns: `{ "isEnabled": false, "indexExists": true, "totalLeads": 42, "lastRun": "..." }` ### Dismiss a lead ```bash curl -X POST "http://localhost:5601/internal/entity_analytics/leads/{id}/_dismiss" \ -H "kbn-xsrf: true" \ -H "elastic-api-version: 1" \ -u <....> ``` Returns: `{ "success": true }` ### Bulk update ```bash curl -X POST "http://localhost:5601/internal/entity_analytics/leads/bulk_update" \ -H "kbn-xsrf: true" \ -H "Content-Type: application/json" \ -H "elastic-api-version: 1" \ -u <....> \ -d '{"ids": ["id-1", "id-2"], "status": "dismissed"}' ``` Returns: `{ "updated": 2 }` ### Enable scheduled generation ```bash curl -X POST "http://localhost:5601/internal/entity_analytics/leads/enable" \ -H "kbn-xsrf: true" \ -H "elastic-api-version: 1" \ -u <....> ``` Returns: `{ "success": true }` ### Disable scheduled generation ```bash curl -X POST "http://localhost:5601/internal/entity_analytics/leads/disable" \ -H "kbn-xsrf: true" \ -H "elastic-api-version: 1" \ -u <....> ``` Returns: `{ "success": true }` --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Jared Burgett <147995946+jaredburgettelastic@users.noreply.github.com> Co-authored-by: jaredburgettelastic <jared.burgett@elastic.co>
…t, and async generation (elastic#257046) ## Summary - **LeadDataClient abstraction** : Extracted all Elasticsearch CRUD operations (create, find, get-by-id, dismiss, bulk-update, status, delete) into a dedicated `LeadDataClient` with snake_case ↔ camelCase transform layer between index mappings and API types - **Async generate endpoint** — Refactored `POST /generate` to return an `executionUuid` immediately and run the pipeline (entity retrieval -> observation modules -> scoring -> synthesis -> persistence) in the background - **Six new API routes** : Added `GET /{id}`, `GET /status`, `POST /{id}/_dismiss`, `POST /bulk_update`, `POST /enable`, and `POST /disable` with Zod-validated schemas, versioned routing, and proper authorization - **Shared Zod schemas** : Moved all request/response schemas to common types (generateLeadsRequestSchema, findLeadsRequestSchema, getLeadByIdRequestSchema, dismissLeadRequestSchema, bulkUpdateLeadsRequestSchema, generateLeadsResponseSchema) - **Refactored GET /leads** : Now delegates to `LeadDataClient.findLeads` with page/perPage/sortField/sortOrder/status query params - **Unit tests** : Added tests for LeadDataClient (createLeads, findLeads, getLeadById, dismissLead, bulkUpdateLeads, getStatus, deleteAllLeads) and all route handlers (get_leads, get_lead_by_id, get_lead_generation_status, dismiss_lead, bulk_update_leads, enable, disable) Enable and disable routes are placeholder stubs pending Task Manager wiring (elastic#15955). Relates to: elastic/security-team#15953 ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [ ] Flaky Test Runner was used on any tests changed - [x] The PR description includes the appropriate Release Notes section, and the correct `release_note:*` label is applied per the guidelines - [ ] Review the backport guidelines and apply applicable `backport:*` labels. ## Testing Steps **Manual verification** Start Kibana locally and test the API endpoints below. ## API Examples ### Generate leads (async) ```bash curl -X POST "http://localhost:5601/internal/entity_analytics/leads/generate" \ -H "kbn-xsrf: true" \ -H "Content-Type: application/json" \ -H "elastic-api-version: 1" \ -u <....> \ -d '{"connectorId": "optional-connector-id"}' ``` Returns: `{ "executionUuid": "abc-123-..." }` ### Find leads ```bash curl -X GET "http://localhost:5601/internal/entity_analytics/leads?page=1&perPage=10&sortField=priority&sortOrder=desc" \ -H "kbn-xsrf: true" \ -H "elastic-api-version: 1" \ -u <....> ``` Returns: `{ "leads": [...], "total": 42, "page": 1, "perPage": 10 }` ### Get lead by ID ```bash curl -X GET "http://localhost:5601/internal/entity_analytics/leads/{id}" \ -H "kbn-xsrf: true" \ -H "elastic-api-version: 1" \ -u <....> ``` Returns: full `leadSchema` object ### Get status ```bash curl -X GET "http://localhost:5601/internal/entity_analytics/leads/status" \ -H "kbn-xsrf: true" \ -H "elastic-api-version: 1" \ -u <....> ``` Returns: `{ "isEnabled": false, "indexExists": true, "totalLeads": 42, "lastRun": "..." }` ### Dismiss a lead ```bash curl -X POST "http://localhost:5601/internal/entity_analytics/leads/{id}/_dismiss" \ -H "kbn-xsrf: true" \ -H "elastic-api-version: 1" \ -u <....> ``` Returns: `{ "success": true }` ### Bulk update ```bash curl -X POST "http://localhost:5601/internal/entity_analytics/leads/bulk_update" \ -H "kbn-xsrf: true" \ -H "Content-Type: application/json" \ -H "elastic-api-version: 1" \ -u <....> \ -d '{"ids": ["id-1", "id-2"], "status": "dismissed"}' ``` Returns: `{ "updated": 2 }` ### Enable scheduled generation ```bash curl -X POST "http://localhost:5601/internal/entity_analytics/leads/enable" \ -H "kbn-xsrf: true" \ -H "elastic-api-version: 1" \ -u <....> ``` Returns: `{ "success": true }` ### Disable scheduled generation ```bash curl -X POST "http://localhost:5601/internal/entity_analytics/leads/disable" \ -H "kbn-xsrf: true" \ -H "elastic-api-version: 1" \ -u <....> ``` Returns: `{ "success": true }` --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Jared Burgett <147995946+jaredburgettelastic@users.noreply.github.com> Co-authored-by: jaredburgettelastic <jared.burgett@elastic.co>
Summary
LeadDataClientwith snake_case ↔ camelCase transform layer between index mappings and API typesPOST /generateto return anexecutionUuidimmediately and run the pipeline (entity retrieval -> observation modules -> scoring -> synthesis -> persistence) in the backgroundGET /{id},GET /status,POST /{id}/_dismiss,POST /bulk_update,POST /enable, andPOST /disablewith Zod-validated schemas, versioned routing, and proper authorizationLeadDataClient.findLeadswith page/perPage/sortField/sortOrder/status query paramsEnable and disable routes are placeholder stubs pending Task Manager wiring (#15955).
Relates to: https://github.com/elastic/security-team/issues/15953
Checklist
Check the PR satisfies following conditions.
Reviewers should verify this PR satisfies this list as well.
release_note:*label is applied per the guidelinesbackport:*labels.Testing Steps
Manual verification Start Kibana locally and test the API endpoints below.
API Examples
Generate leads (async)
Returns:
{ "executionUuid": "abc-123-..." }Find leads
Returns:
{ "leads": [...], "total": 42, "page": 1, "perPage": 10 }Get lead by ID
Returns: full
leadSchemaobjectGet status
Returns:
{ "isEnabled": false, "indexExists": true, "totalLeads": 42, "lastRun": "..." }Dismiss a lead
Returns:
{ "success": true }Bulk update
Returns:
{ "updated": 2 }Enable scheduled generation
Returns:
{ "success": true }Disable scheduled generation
Returns:
{ "success": true }