Skip to content

[8.x] [RCA] AI-assisted root cause analysis (#197200)#203767

Merged
dgieselaar merged 2 commits intoelastic:8.xfrom
dgieselaar:backport/8.x/pr-197200
Dec 12, 2024
Merged

[8.x] [RCA] AI-assisted root cause analysis (#197200)#203767
dgieselaar merged 2 commits intoelastic:8.xfrom
dgieselaar:backport/8.x/pr-197200

Conversation

@dgieselaar
Copy link
Copy Markdown
Contributor

Backport

This will backport the following commits from main to 8.x:

Questions ?

Please refer to the Backport tool documentation

Implements an LLM-based root cause analysis process. At a high level, it
works by investigating entities - which means pulling in alerts, SLOs,
and log patterns. From there, it can inspect related entities to get to
the root cause.

The backend implementation lives in
`x-pack/packages/observability_utils-*` (`service_rca`). It can be
imported into any server-side plugin and executed from there.

The UI changes are mostly contained to
`x-pack/plugins/observability_solution/observabillity_ai_assistant_app`.
This plugin now exports a `RootCauseAnalysisContainer` which takes a
stream of data that is returned by the root cause analysis process.

The current implementation lives in the Investigate app. There, it calls
its own endpoint that kicks off the RCA process, and feeds it into the
`RootCauseAnalysisContainer` exposed by the Observability AI Assistant
app plugin. I've left it in a route there so the investigation itself
can be updated as the process runs - this would allow the user to close
the browser and come back later, and see a full investigation.

> [!NOTE]
> Notes for reviewing teams
>
> @kbn/es-types:
> - support both types and typesWithBodyKey
> - simplify KeysOfSources type
>
> @kbn/server-route-repository:
> - abortable streamed responses
>
> @kbn/sse-utils*:
> - abortable streamed responses
> - serialize errors in specific format for more reliable re-hydration
of errors
> - keep connection open with SSE comments
>
> @kbn/inference-*:
> - export *Of variants of types, for easier manual inference
> - add automated retries for `output` API
> - add `name` to tool responses for type inference (get type of tool
response via tool name)
> - add `data` to tool responses for transporting internal data (not
sent to the LLM)
> - simplify `chunksIntoMessage`
> - allow consumers of nlToEsql task to add to `system` prompt
> - add toolCallId to validation error message
>
> @kbn/aiops*:
> - export `categorizationAnalyzer` for use in observability-ai*
>
> @kbn/observability-ai-assistant*
> - configurable limit (tokens or doc count) for knowledge base recall
>
> @kbn/slo*:
> - export client that returns summary indices

---------

Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>
Co-authored-by: Maryam Saeidi <maryam.saeidi@elastic.co>
Co-authored-by: Bena Kansara <bena.kansara@elastic.co>
(cherry picked from commit fa1998c)

# Conflicts:
#	.github/CODEOWNERS
@dgieselaar dgieselaar added the backport This PR is a backport of another PR label Dec 11, 2024
@dgieselaar dgieselaar enabled auto-merge (squash) December 11, 2024 11:49
@botelastic botelastic bot added ci:project-deploy-observability Create an Observability project Team:Obs AI Assistant Observability AI Assistant Team:actionable-obs Formerly "obs-ux-management", responsible for SLO, o11y alerting, significant events, & synthetics. labels Dec 11, 2024
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/obs-ai-assistant (Team:Obs AI Assistant)

@github-actions
Copy link
Copy Markdown
Contributor

🤖 GitHub comments

Expand to view the GitHub comments

Just comment with:

  • /oblt-deploy : Deploy a Kibana instance using the Observability test environments.
  • run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

@dgieselaar dgieselaar force-pushed the backport/8.x/pr-197200 branch from 3e55a9b to efdd6f5 Compare December 11, 2024 11:59
@elasticmachine
Copy link
Copy Markdown
Contributor

⏳ Build in-progress

  • Buildkite Build
  • Commit: efdd6f5
  • Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-203767-efdd6f50852f

@dgieselaar dgieselaar merged commit b3ba62a into elastic:8.x Dec 12, 2024
@dgieselaar dgieselaar deleted the backport/8.x/pr-197200 branch December 12, 2024 14:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport This PR is a backport of another PR ci:project-deploy-observability Create an Observability project Team:actionable-obs Formerly "obs-ux-management", responsible for SLO, o11y alerting, significant events, & synthetics. Team:Obs AI Assistant Observability AI Assistant

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants