[Obs AI] Add o11y data-generators (OpenRCA and RCAEval) for producing logs, metrics, traces by sorenlouv · Pull Request #256591 · elastic/kibana

sorenlouv · 2026-03-07T22:51:19Z

This PR adds support for alternative datasets to OpenTelemetry-demo. The scripts for ingesting the datasets OpenRCA and RCAEval make it easy to populate Elasticsearch with realistic logs, traces, and metrics for evaluating Observability Agent Builder tools against known root-cause scenarios.

Try it

1. Start EDOT Collector (only necessary when ingesting traces)

node scripts/edot_collector.js

Run one of the following commands from x-pack/solutions/observability/plugins/observability_agent_builder/.

2. RCAEval

# List all cases
npx tsx scripts/ingest_rcaeval.ts

# Ingest a single failure case (adservice returning incorrect values)
npx tsx scripts/ingest_rcaeval.ts --case adservice_f4/1

# Or skip traces if you don't want to run the EDOT Collector
npx tsx scripts/ingest_rcaeval.ts --case adservice_f4/1 --skip-traces --skip-metrics

See the full README for all options, ground-truth expected root causes for each case, and verification commands.

Cursor:

Using the datasets to evaluate the Obs Agent: https://cursor.com/dashboard?tab=shared-chats&shareId=observability-agent-validation-plan-shared-25MTZpoD4xzN

…trics and traces

sorenlouv · 2026-03-09T13:34:24Z

src/platform/packages/shared/kbn-edot-collector/src/get_edot_collector_configuration.ts

    },
    processors: {
-      elastictrace: {},
+      elasticapm: {},


elastictrace processor has been deprecated and succeeded by elasticapm processor elastic/elastic-agent#10588

This reverts commit 36f2d39.

…into add-rca-datasets

elasticmachine · 2026-03-18T11:52:41Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 592fc5e

Failed CI Steps

Test Failures

[job] [logs] Scout: [ security / entity_store ] plugin / local-serverless-security_complete - Entity Store Main logs extraction - Should extract properly extract host
[job] [logs] Scout: [ platform / workflows_management ] plugin / local-serverless-security_complete - Workflow execution concurrency control - drop strategy drops new executions until there is an already running execution

Metrics [docs]

Unknown metric groups

ESLint disabled in files

id	before	after	diff
`observabilityAgentBuilder`	0	3	+3

Total ESLint disabled count

id	before	after	diff
`observabilityAgentBuilder`	2	5	+3

History

💚 Build #408535 succeeded 0298358
💔 Build #407354 failed d873eb4
💔 Build #407112 failed a7f4c20
💔 Build #406715 failed 36f2d39
💔 Build #406626 failed 0bcc2b6

… logs, metrics, traces (elastic#256591) This PR adds support for alternative datasets to OpenTelemetry-demo. The scripts for ingesting the datasets [OpenRCA](https://github.com/microsoft/OpenRCA) and [RCAEval](https://github.com/phamquiluan/RCAEval) make it easy to populate Elasticsearch with realistic logs, traces, and metrics for evaluating Observability Agent Builder tools against known root-cause scenarios. ### Try it **1. Start EDOT Collector (only necessary when ingesting traces)** ```sh node scripts/edot_collector.js ``` Run one of the following commands from `x-pack/solutions/observability/plugins/observability_agent_builder/`. **2. RCAEval** ```bash # List all cases npx tsx scripts/ingest_rcaeval.ts # Ingest a single failure case (adservice returning incorrect values) npx tsx scripts/ingest_rcaeval.ts --case adservice_f4/1 # Or skip traces if you don't want to run the EDOT Collector npx tsx scripts/ingest_rcaeval.ts --case adservice_f4/1 --skip-traces --skip-metrics ``` See the full README for all options, ground-truth expected root causes for each case, and verification commands. **Cursor:** - Using the datasets to evaluate the Obs Agent: https://cursor.com/dashboard?tab=shared-chats&shareId=observability-agent-validation-plan-shared-25MTZpoD4xzN --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

…d_agent_navigation2 * commit 'b511b784a89644463497411bc8cfac03522d43a9': (40 commits) skip failing test suite (elastic#252959) skip failing test suite (elastic#255548) skip failing test suite (elastic#256140) skip failing test suite (elastic#257103) skip failing test suite (elastic#258148) [SharedUX] Add solution view switch callout to spaces plugin (elastic#258093) skip tests consistently failing on ECH (elastic#258157) [EDR Workflows][Osquery] Disable tags for scheduled queries (elastic#258222) [Security solution][Attacks] Add navigation E2E test (elastic#255373) [canvas] fix unable to load embeddable when no references are provided (elastic#257779) docs(streams): update Discovery settings labels and help text (elastic#258328) [ResponseOps] Fixes x-pack/platform/test/alerting_api_integration/spaces_only/tests/alerting/group4/alert_severity.ts flaky test (elastic#258226) [Lens as Code] Fix legend truncation issues (elastic#258216) Upgraded flatted (elastic#258252) [One Discover][Logs UX] Update OpenTelemetry Semantic Conventions (elastic#256613) add Agent Builder compatibility to connectors (elastic#257491) [Obs AI] Add o11y data-generators (OpenRCA and RCAEval) for producing logs, metrics, traces (elastic#256591) [One Workflow] Update execution history UI: show nested workflows steps (elastic#257352) [One Workflow] bulkUpdateSchedules should be called with request to follow auth (elastic#258150) [Agent Builder] Semantic Meta Layer (elastic#254849) ...

… logs, metrics, traces (elastic#256591) This PR adds support for alternative datasets to OpenTelemetry-demo. The scripts for ingesting the datasets [OpenRCA](https://github.com/microsoft/OpenRCA) and [RCAEval](https://github.com/phamquiluan/RCAEval) make it easy to populate Elasticsearch with realistic logs, traces, and metrics for evaluating Observability Agent Builder tools against known root-cause scenarios. ### Try it **1. Start EDOT Collector (only necessary when ingesting traces)** ```sh node scripts/edot_collector.js ``` Run one of the following commands from `x-pack/solutions/observability/plugins/observability_agent_builder/`. **2. RCAEval** ```bash # List all cases npx tsx scripts/ingest_rcaeval.ts # Ingest a single failure case (adservice returning incorrect values) npx tsx scripts/ingest_rcaeval.ts --case adservice_f4/1 # Or skip traces if you don't want to run the EDOT Collector npx tsx scripts/ingest_rcaeval.ts --case adservice_f4/1 --skip-traces --skip-metrics ``` See the full README for all options, ground-truth expected root causes for each case, and verification commands. **Cursor:** - Using the datasets to evaluate the Obs Agent: https://cursor.com/dashboard?tab=shared-chats&shareId=observability-agent-validation-plan-shared-25MTZpoD4xzN --------- Co-authored-by: kibanamachine <42973632+kibanamachine@users.noreply.github.com>

sorenlouv added 11 commits March 7, 2026 23:42

Add o11y data-generators (OpenRCA and RCAEval) for producing logs, me…

505238c

…trics and traces

Fix —clean command

035dc02

Add all root causes (ground truth) to readme

ef16a17

Simplify readme

92fb37a

Add —clean shorthand

2d6453e

Remove ground truth labels

601014d

RCAEval fixes

f31ab5b

Add support for service dest metrics

f301ade

Improve scripts

05c6208

Fix time window

6f107d4

Improve cleanup

0bcc2b6

sorenlouv marked this pull request as ready for review March 9, 2026 10:53

sorenlouv requested a review from a team as a code owner March 9, 2026 10:53

sorenlouv added release_note:skip Skip the PR/issue when compiling release notes backport:skip This PR does not require backporting labels Mar 9, 2026

Remove OpenRCA

36f2d39

sorenlouv commented Mar 9, 2026

View reviewed changes

sorenlouv and others added 5 commits March 9, 2026 16:22

Revert "Remove OpenRCA"

42b1996

This reverts commit 36f2d39.

Merge branch 'main' into add-rca-datasets

53e05ce

Merge branch 'add-rca-datasets' of https://github.com/sorenlouv/kibana …

a7f4c20

…into add-rca-datasets

Add to tsconfig

d873eb4

Changes from node scripts/regenerate_moon_projects.js --update

cb2d2ad

elastic-vault-github-plugin-prod bot requested a review from a team as a code owner March 10, 2026 10:54

sorenlouv added 2 commits March 10, 2026 23:52

Merge branch 'main' into add-rca-datasets

e0b79ee

Merge branch 'main' into add-rca-datasets

0298358

jbudz removed the request for review from a team March 13, 2026 21:55

sorenlouv added 2 commits March 16, 2026 12:57

Merge branch 'main' into add-rca-datasets

00e879e

Merge branch 'main' into add-rca-datasets

592fc5e

Improve AGENTS.md instructions

e5f482d

arturoliduena approved these changes Mar 18, 2026

View reviewed changes

sorenlouv merged commit cdc836c into elastic:main Mar 18, 2026
19 checks passed

sorenlouv deleted the add-rca-datasets branch March 18, 2026 14:11

kibanamachine added the v9.4.0 label Mar 18, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Obs AI] Add o11y data-generators (OpenRCA and RCAEval) for producing logs, metrics, traces#256591

[Obs AI] Add o11y data-generators (OpenRCA and RCAEval) for producing logs, metrics, traces#256591
sorenlouv merged 22 commits intoelastic:mainfrom
sorenlouv:add-rca-datasets

sorenlouv commented Mar 7, 2026 •

edited

Loading

Uh oh!

sorenlouv Mar 9, 2026

Uh oh!

elasticmachine commented Mar 18, 2026

ESLint disabled in files

Total ESLint disabled count

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

sorenlouv commented Mar 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Try it

Uh oh!

sorenlouv Mar 9, 2026

Choose a reason for hiding this comment

Uh oh!

elasticmachine commented Mar 18, 2026

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

ESLint disabled in files

Total ESLint disabled count

History

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

sorenlouv commented Mar 7, 2026 •

edited

Loading