Description

Wanted to create this epic to cover all the new work around the Security Assistant Knowledge Base for 8.15. The main effort can be summarized as adding support for 'custom Knowledge Base content' (both adding raw content and linking indices/data streams), but also includes improvements to how the KB is setup, managed in settings, and some new content we're planning on shipping, like exports of the Elastic Security Labs content.

Background & resources

• Open internal issues for tracking:
  [UX] Knowledge Base design: Stack management and AI Assistant UIs #9392
  [Epic] AI Assistant - Incorporate Latest Elastic Security Labs' Research into KB #8043
  [Epic] AI Assistant and Insights - Allow users to import custom KB articles #8737

• PR's thus far:
  [Security Assistant] Automatically Install Knowledge Base #182763
  [Security Assistant] Migrates to LangGraph and adds KB Tools #184554
  [Security Assistant] Adds Security Labs Knowledge Base content #184885
  [Security Assistant] Enables automatic setup of Knowledge Base and LangGraph code paths for 8.15 #188168

• Point of contact: @spong @jamesspi @YulNaumenko

• Test environments: TBD, but available by enabling the assistantKnowledgeBaseByDefault feature flag

Which documentation set does this change impact?

ESS and serverless

ESS release

8.15

Serverless release

Soon after 8.15 FF, so near mid-July

Feature differences

Should have fully parity, but if anything ELSER/KB setup might end up being automatic in Serverless (so no 'Install Knowledge Base' button within the assistant).

API docs impact

We'll be introducing a whole new API for managing KB documents. This API is intended to be public at some point, but may ship as experimental or internal initially so we have flexibility to make modifications.

Initial OpenAPI Specs were added in [Security Assistant] Automatically Install Knowledge Base #182763, but they are still subject to change.

Prerequisites, privileges, feature flags

• ESS: Enterprise License w/ 4GB ML Node so ELSER can be deployed
• Serverless: Security Complete Product Tier
• Feature Flag: Currently all functionality is behind the assistantKnowledgeBaseByDefault xpack.securitySolution.enableExperimental feature flag