Skip to content

[Rule Migration] Add PackageClient to fetch integrations#207197

Merged
P1llus merged 4 commits intoelastic:mainfrom
P1llus:siem_migration_add_epm_client_rag
Jan 21, 2025
Merged

[Rule Migration] Add PackageClient to fetch integrations#207197
P1llus merged 4 commits intoelastic:mainfrom
P1llus:siem_migration_add_epm_client_rag

Conversation

@P1llus
Copy link
Copy Markdown
Member

@P1llus P1llus commented Jan 20, 2025

Summary

Awaiting the changes to the EPR API to include datasets we temporarily included a JSON file to simulate the expected outcome so we could move forward while waiting.

Since the changes is now done, the calls to https://epr.elastic.co/search? now returns data_streams in their responses so this PR replaces the temp JSON with the proper PackageClient implementation.

Tested a few migrations with both prebuilt rule matches and without, and seems to be working as intended. The integration count also seems to be consistent with what to be expected.

@P1llus P1llus requested a review from a team as a code owner January 20, 2025 13:27
@P1llus P1llus changed the title Siem migration add epm client rag [Rule Migration] Add PackageClient to fetch integrations Jan 20, 2025
@P1llus P1llus added release_note:skip Skip the PR/issue when compiling release notes v9.0.0 Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. backport:version Backport to applied version labels v8.18.0 labels Jan 20, 2025
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@P1llus P1llus added Team:Threat Hunting Security Solution Threat Hunting Team and removed Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. labels Jan 20, 2025
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-threat-hunting (Team:Threat Hunting)

@P1llus P1llus added the Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. label Jan 20, 2025
Copy link
Copy Markdown
Contributor

@semd semd left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@P1llus P1llus enabled auto-merge (squash) January 21, 2025 12:28
@elasticmachine
Copy link
Copy Markdown
Contributor

elasticmachine commented Jan 21, 2025

💔 Build Failed

Failed CI Steps

Test Failures

  • [job] [logs] Jest Tests #3 / Configure updates field options correctly when not required
  • [job] [logs] Jest Tests #3 / usePersistConfiguration calls postCaseConfigure when the version is empty

Metrics [docs]

✅ unchanged

History

cc @P1llus

@P1llus P1llus disabled auto-merge January 21, 2025 13:06
@P1llus P1llus merged commit 1e5abbe into elastic:main Jan 21, 2025
@kibanamachine
Copy link
Copy Markdown
Contributor

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12890599898

@P1llus
Copy link
Copy Markdown
Member Author

P1llus commented Jan 21, 2025

Unsure why elasticmachine did not report it, but reran the flaky tests (not related to this PR) and it was successful in buildkite, github also reported all tests passing.

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Jan 21, 2025
)

## Summary

Awaiting the changes to the EPR API to include datasets we temporarily
included a JSON file to simulate the expected outcome so we could move
forward while waiting.

Since the changes is now done, the calls to
https://epr.elastic.co/search? now returns data_streams in their
responses so this PR replaces the temp JSON with the proper
PackageClient implementation.

Tested a few migrations with both prebuilt rule matches and without, and
seems to be working as intended. The integration count also seems to be
consistent with what to be expected.

(cherry picked from commit 1e5abbe)
@kibanamachine
Copy link
Copy Markdown
Contributor

💚 All backports created successfully

Status Branch Result
8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Jan 21, 2025
…) (#207391)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Rule Migration] Add PackageClient to fetch integrations
(#207197)](#207197)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Marius
Iversen","email":"marius.iversen@elastic.co"},"sourceCommit":{"committedDate":"2025-01-21T15:46:44Z","message":"[Rule
Migration] Add PackageClient to fetch integrations (#207197)\n\n##
Summary\r\n\r\nAwaiting the changes to the EPR API to include datasets
we temporarily\r\nincluded a JSON file to simulate the expected outcome
so we could move\r\nforward while waiting.\r\n\r\nSince the changes is
now done, the calls to\r\nhttps://epr.elastic.co/search? now returns
data_streams in their\r\nresponses so this PR replaces the temp JSON
with the proper\r\nPackageClient implementation.\r\n\r\nTested a few
migrations with both prebuilt rule matches and without, and\r\nseems to
be working as intended. The integration count also seems to
be\r\nconsistent with what to be
expected.","sha":"1e5abbe8eeb198a62b9ef43179db3124fc6c7a1b","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Threat
Hunting","Team:
SecuritySolution","backport:version","v8.18.0"],"title":"[Rule
Migration] Add PackageClient to fetch
integrations","number":207197,"url":"https://github.com/elastic/kibana/pull/207197","mergeCommit":{"message":"[Rule
Migration] Add PackageClient to fetch integrations (#207197)\n\n##
Summary\r\n\r\nAwaiting the changes to the EPR API to include datasets
we temporarily\r\nincluded a JSON file to simulate the expected outcome
so we could move\r\nforward while waiting.\r\n\r\nSince the changes is
now done, the calls to\r\nhttps://epr.elastic.co/search? now returns
data_streams in their\r\nresponses so this PR replaces the temp JSON
with the proper\r\nPackageClient implementation.\r\n\r\nTested a few
migrations with both prebuilt rule matches and without, and\r\nseems to
be working as intended. The integration count also seems to
be\r\nconsistent with what to be
expected.","sha":"1e5abbe8eeb198a62b9ef43179db3124fc6c7a1b"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/207197","number":207197,"mergeCommit":{"message":"[Rule
Migration] Add PackageClient to fetch integrations (#207197)\n\n##
Summary\r\n\r\nAwaiting the changes to the EPR API to include datasets
we temporarily\r\nincluded a JSON file to simulate the expected outcome
so we could move\r\nforward while waiting.\r\n\r\nSince the changes is
now done, the calls to\r\nhttps://epr.elastic.co/search? now returns
data_streams in their\r\nresponses so this PR replaces the temp JSON
with the proper\r\nPackageClient implementation.\r\n\r\nTested a few
migrations with both prebuilt rule matches and without, and\r\nseems to
be working as intended. The integration count also seems to
be\r\nconsistent with what to be
expected.","sha":"1e5abbe8eeb198a62b9ef43179db3124fc6c7a1b"}},{"branch":"8.x","label":"v8.18.0","branchLabelMappingKey":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Marius Iversen <marius.iversen@elastic.co>
viduni94 pushed a commit to viduni94/kibana that referenced this pull request Jan 23, 2025
)

## Summary

Awaiting the changes to the EPR API to include datasets we temporarily
included a JSON file to simulate the expected outcome so we could move
forward while waiting.

Since the changes is now done, the calls to
https://epr.elastic.co/search? now returns data_streams in their
responses so this PR replaces the temp JSON with the proper
PackageClient implementation.

Tested a few migrations with both prebuilt rule matches and without, and
seems to be working as intended. The integration count also seems to be
consistent with what to be expected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:version Backport to applied version labels release_note:skip Skip the PR/issue when compiling release notes Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Threat Hunting Security Solution Threat Hunting Team v8.18.0 v9.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants