Skip to content

[Security Solution] Rework test plan for importing prebuilt rules#213434

Merged
banderror merged 12 commits intoelastic:mainfrom
banderror:rework-prebuilt-rule-import-test-plan
Mar 24, 2025
Merged

[Security Solution] Rework test plan for importing prebuilt rules#213434
banderror merged 12 commits intoelastic:mainfrom
banderror:rework-prebuilt-rule-import-test-plan

Conversation

@banderror
Copy link
Copy Markdown
Contributor

@banderror banderror commented Mar 6, 2025

Epic: #174168
Partially addresses: #202079, #210358

Summary

We started to rework and introduce functional changes to our existing test plans for prebuilt rule customization, upgrade, and export/import workflows.

Specifically, this PR:

  • Restructures the test plan, introduces a more fine-grained list of sections.
  • Rewrites almost all the existing scenarios. In most cases it boils down to splitting a scenario into 2+ more specific scenarios, where each describes exactly what happens in the GIVEN and THEN sections. This is very important, as it:
    • makes these scenarios ready to be implemented right away: 1 scenario = 1 test to write
    • helps with ensuring that we covered edge cases
  • Adds new scenarios for handling missing base versions according to #210358.
  • Adds a placeholder section for licensing scenarios according to #11502.
  • Removes scenarios for the overwrite request parameter - this is common importing logic which is not related to prebuilt rules.
  • Addresses my own comments from this review.

The new test plan should be in line with the changes discussed in #210358.

@banderror banderror added release_note:skip Skip the PR/issue when compiling release notes test-plan v9.0.0 Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. Team:Detection Rule Management Security Detection Rule Management Team Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area backport:version Backport to applied version labels v8.18.0 v9.1.0 v8.19.0 labels Mar 6, 2025
@banderror banderror self-assigned this Mar 6, 2025
@banderror banderror force-pushed the rework-prebuilt-rule-import-test-plan branch 4 times, most recently from 90c067b to 155993a Compare March 7, 2025 15:41
@banderror banderror force-pushed the rework-prebuilt-rule-import-test-plan branch from 155993a to 87353e9 Compare March 7, 2025 18:14
@banderror banderror marked this pull request as ready for review March 7, 2025 18:14
@banderror banderror requested a review from a team as a code owner March 7, 2025 18:14
@banderror banderror requested a review from jkelas March 7, 2025 18:14
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-detections-response (Team:Detections and Resp)

@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-solution (Team: SecuritySolution)

@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management)

@banderror banderror requested review from dplumlee, maximpn, nikitaindik and pborgonovi and removed request for jkelas March 7, 2025 18:15
@banderror
Copy link
Copy Markdown
Contributor Author

banderror commented Mar 24, 2025

I have a little comment though. Test scenarios mention nothing about revision. It should be bumped whenever any modification happens. Since revision is important for prebuilt rule upgrade I believe it should be mentioned in the test scenarios. WDYT?

@maximpn Not sure I understand what exactly are you suggesting.

  • The revision field is only important for the upgrade workflow where it is used for OCC. So I believe we should have a scenario in one of the upgrade test plans and a corresponding API integration test for that. Maybe a few scenarios and a few tests. It would have nothing to do with the import.
  • This is a test plan for the prebuilt rule import workflow. On import, we increment the revision field for those rules that already exist and get updated. This logic is not specific to prebuilt rules, or prebuilt rules vs custom rules; it's something that applies to importing any rules. If we want it to be covered, it should be covered in a separate test plan for the common importing logic, which we don't have at the moment. I'd personally suggest to not focus on covering common importing logic at this point. Feel free to insist!

@banderror banderror force-pushed the rework-prebuilt-rule-import-test-plan branch from 20cd257 to 6ba5f61 Compare March 24, 2025 18:38
@banderror banderror enabled auto-merge (squash) March 24, 2025 18:40
@banderror banderror merged commit a9106c6 into elastic:main Mar 24, 2025
9 checks passed
@kibanamachine
Copy link
Copy Markdown
Contributor

Starting backport for target branches: 8.18, 8.x, 9.0

https://github.com/elastic/kibana/actions/runs/14043658955

kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Mar 24, 2025
…astic#213434)

**Epic:** elastic#174168
**Partially addresses:**
elastic#202079,
elastic#210358

## Summary

We started to rework and introduce functional changes to our existing
test plans for prebuilt rule customization, upgrade, and export/import
workflows.

Specifically, this PR:

- Restructures the test plan, introduces a more fine-grained list of
sections.
- Rewrites almost all the existing scenarios. In most cases it boils
down to splitting a scenario into 2+ more specific scenarios, where each
describes _exactly_ what happens in the GIVEN and THEN sections. This is
very important, as it:
- makes these scenarios ready to be implemented right away: 1 scenario =
1 test to write
  - helps with ensuring that we covered edge cases
- Adds new scenarios for handling missing base versions according to
[elastic#210358](elastic#210358).
- Adds a placeholder section for licensing scenarios according to
[elastic#11502](elastic/security-team#11502).
- Removes scenarios for the `overwrite` request parameter - this is
common importing logic which is not related to prebuilt rules.
- Addresses my own comments from [this
review](elastic#206893 (review)).

The new test plan should be in line with the changes discussed in
elastic#210358.

(cherry picked from commit a9106c6)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Mar 24, 2025
…astic#213434)

**Epic:** elastic#174168
**Partially addresses:**
elastic#202079,
elastic#210358

## Summary

We started to rework and introduce functional changes to our existing
test plans for prebuilt rule customization, upgrade, and export/import
workflows.

Specifically, this PR:

- Restructures the test plan, introduces a more fine-grained list of
sections.
- Rewrites almost all the existing scenarios. In most cases it boils
down to splitting a scenario into 2+ more specific scenarios, where each
describes _exactly_ what happens in the GIVEN and THEN sections. This is
very important, as it:
- makes these scenarios ready to be implemented right away: 1 scenario =
1 test to write
  - helps with ensuring that we covered edge cases
- Adds new scenarios for handling missing base versions according to
[elastic#210358](elastic#210358).
- Adds a placeholder section for licensing scenarios according to
[elastic#11502](elastic/security-team#11502).
- Removes scenarios for the `overwrite` request parameter - this is
common importing logic which is not related to prebuilt rules.
- Addresses my own comments from [this
review](elastic#206893 (review)).

The new test plan should be in line with the changes discussed in
elastic#210358.

(cherry picked from commit a9106c6)
kibanamachine pushed a commit to kibanamachine/kibana that referenced this pull request Mar 24, 2025
…astic#213434)

**Epic:** elastic#174168
**Partially addresses:**
elastic#202079,
elastic#210358

## Summary

We started to rework and introduce functional changes to our existing
test plans for prebuilt rule customization, upgrade, and export/import
workflows.

Specifically, this PR:

- Restructures the test plan, introduces a more fine-grained list of
sections.
- Rewrites almost all the existing scenarios. In most cases it boils
down to splitting a scenario into 2+ more specific scenarios, where each
describes _exactly_ what happens in the GIVEN and THEN sections. This is
very important, as it:
- makes these scenarios ready to be implemented right away: 1 scenario =
1 test to write
  - helps with ensuring that we covered edge cases
- Adds new scenarios for handling missing base versions according to
[elastic#210358](elastic#210358).
- Adds a placeholder section for licensing scenarios according to
[elastic#11502](elastic/security-team#11502).
- Removes scenarios for the `overwrite` request parameter - this is
common importing logic which is not related to prebuilt rules.
- Addresses my own comments from [this
review](elastic#206893 (review)).

The new test plan should be in line with the changes discussed in
elastic#210358.

(cherry picked from commit a9106c6)
@kibanamachine
Copy link
Copy Markdown
Contributor

💚 All backports created successfully

Status Branch Result
8.18
8.x
9.0

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

kibanamachine added a commit that referenced this pull request Mar 24, 2025
…es (#213434) (#215796)

# Backport

This will backport the following commits from `main` to `9.0`:
- [[Security Solution] Rework test plan for importing prebuilt rules
(#213434)](#213434)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Georgii
Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-03-24T19:00:36Z","message":"[Security
Solution] Rework test plan for importing prebuilt rules
(#213434)\n\n**Epic:**
https://github.com/elastic/kibana/issues/174168\n**Partially
addresses:**\nhttps://github.com//issues/202079,\nhttps://github.com//issues/210358\n\n##
Summary\n\nWe started to rework and introduce functional changes to our
existing\ntest plans for prebuilt rule customization, upgrade, and
export/import\nworkflows.\n\nSpecifically, this PR:\n\n- Restructures
the test plan, introduces a more fine-grained list of\nsections.\n-
Rewrites almost all the existing scenarios. In most cases it boils\ndown
to splitting a scenario into 2+ more specific scenarios, where
each\ndescribes _exactly_ what happens in the GIVEN and THEN sections.
This is\nvery important, as it:\n- makes these scenarios ready to be
implemented right away: 1 scenario =\n1 test to write\n - helps with
ensuring that we covered edge cases\n- Adds new scenarios for handling
missing base versions according
to\n[#210358](https://github.com/elastic/kibana/issues/210358).\n- Adds
a placeholder section for licensing scenarios according
to\n[#11502](https://github.com/elastic/security-team/issues/11502).\n-
Removes scenarios for the `overwrite` request parameter - this
is\ncommon importing logic which is not related to prebuilt rules.\n-
Addresses my own comments from
[this\nreview](https://github.com/elastic/kibana/pull/206893#pullrequestreview-2633863218).\n\nThe
new test plan should be in line with the changes discussed
in\nhttps://github.com//issues/210358.","sha":"a9106c69904e259412df9bbc77b4411507e1d011","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","Feature:Prebuilt Detection
Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security
Solution] Rework test plan for importing prebuilt
rules","number":213434,"url":"https://github.com/elastic/kibana/pull/213434","mergeCommit":{"message":"[Security
Solution] Rework test plan for importing prebuilt rules
(#213434)\n\n**Epic:**
https://github.com/elastic/kibana/issues/174168\n**Partially
addresses:**\nhttps://github.com//issues/202079,\nhttps://github.com//issues/210358\n\n##
Summary\n\nWe started to rework and introduce functional changes to our
existing\ntest plans for prebuilt rule customization, upgrade, and
export/import\nworkflows.\n\nSpecifically, this PR:\n\n- Restructures
the test plan, introduces a more fine-grained list of\nsections.\n-
Rewrites almost all the existing scenarios. In most cases it boils\ndown
to splitting a scenario into 2+ more specific scenarios, where
each\ndescribes _exactly_ what happens in the GIVEN and THEN sections.
This is\nvery important, as it:\n- makes these scenarios ready to be
implemented right away: 1 scenario =\n1 test to write\n - helps with
ensuring that we covered edge cases\n- Adds new scenarios for handling
missing base versions according
to\n[#210358](https://github.com/elastic/kibana/issues/210358).\n- Adds
a placeholder section for licensing scenarios according
to\n[#11502](https://github.com/elastic/security-team/issues/11502).\n-
Removes scenarios for the `overwrite` request parameter - this
is\ncommon importing logic which is not related to prebuilt rules.\n-
Addresses my own comments from
[this\nreview](https://github.com/elastic/kibana/pull/206893#pullrequestreview-2633863218).\n\nThe
new test plan should be in line with the changes discussed
in\nhttps://github.com//issues/210358.","sha":"a9106c69904e259412df9bbc77b4411507e1d011"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/213434","number":213434,"mergeCommit":{"message":"[Security
Solution] Rework test plan for importing prebuilt rules
(#213434)\n\n**Epic:**
https://github.com/elastic/kibana/issues/174168\n**Partially
addresses:**\nhttps://github.com//issues/202079,\nhttps://github.com//issues/210358\n\n##
Summary\n\nWe started to rework and introduce functional changes to our
existing\ntest plans for prebuilt rule customization, upgrade, and
export/import\nworkflows.\n\nSpecifically, this PR:\n\n- Restructures
the test plan, introduces a more fine-grained list of\nsections.\n-
Rewrites almost all the existing scenarios. In most cases it boils\ndown
to splitting a scenario into 2+ more specific scenarios, where
each\ndescribes _exactly_ what happens in the GIVEN and THEN sections.
This is\nvery important, as it:\n- makes these scenarios ready to be
implemented right away: 1 scenario =\n1 test to write\n - helps with
ensuring that we covered edge cases\n- Adds new scenarios for handling
missing base versions according
to\n[#210358](https://github.com/elastic/kibana/issues/210358).\n- Adds
a placeholder section for licensing scenarios according
to\n[#11502](https://github.com/elastic/security-team/issues/11502).\n-
Removes scenarios for the `overwrite` request parameter - this
is\ncommon importing logic which is not related to prebuilt rules.\n-
Addresses my own comments from
[this\nreview](https://github.com/elastic/kibana/pull/206893#pullrequestreview-2633863218).\n\nThe
new test plan should be in line with the changes discussed
in\nhttps://github.com//issues/210358.","sha":"a9106c69904e259412df9bbc77b4411507e1d011"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
kibanamachine added a commit that referenced this pull request Mar 24, 2025
…les (#213434) (#215794)

# Backport

This will backport the following commits from `main` to `8.18`:
- [[Security Solution] Rework test plan for importing prebuilt rules
(#213434)](#213434)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Georgii
Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-03-24T19:00:36Z","message":"[Security
Solution] Rework test plan for importing prebuilt rules
(#213434)\n\n**Epic:**
https://github.com/elastic/kibana/issues/174168\n**Partially
addresses:**\nhttps://github.com//issues/202079,\nhttps://github.com//issues/210358\n\n##
Summary\n\nWe started to rework and introduce functional changes to our
existing\ntest plans for prebuilt rule customization, upgrade, and
export/import\nworkflows.\n\nSpecifically, this PR:\n\n- Restructures
the test plan, introduces a more fine-grained list of\nsections.\n-
Rewrites almost all the existing scenarios. In most cases it boils\ndown
to splitting a scenario into 2+ more specific scenarios, where
each\ndescribes _exactly_ what happens in the GIVEN and THEN sections.
This is\nvery important, as it:\n- makes these scenarios ready to be
implemented right away: 1 scenario =\n1 test to write\n - helps with
ensuring that we covered edge cases\n- Adds new scenarios for handling
missing base versions according
to\n[#210358](https://github.com/elastic/kibana/issues/210358).\n- Adds
a placeholder section for licensing scenarios according
to\n[#11502](https://github.com/elastic/security-team/issues/11502).\n-
Removes scenarios for the `overwrite` request parameter - this
is\ncommon importing logic which is not related to prebuilt rules.\n-
Addresses my own comments from
[this\nreview](https://github.com/elastic/kibana/pull/206893#pullrequestreview-2633863218).\n\nThe
new test plan should be in line with the changes discussed
in\nhttps://github.com//issues/210358.","sha":"a9106c69904e259412df9bbc77b4411507e1d011","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","Feature:Prebuilt Detection
Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security
Solution] Rework test plan for importing prebuilt
rules","number":213434,"url":"https://github.com/elastic/kibana/pull/213434","mergeCommit":{"message":"[Security
Solution] Rework test plan for importing prebuilt rules
(#213434)\n\n**Epic:**
https://github.com/elastic/kibana/issues/174168\n**Partially
addresses:**\nhttps://github.com//issues/202079,\nhttps://github.com//issues/210358\n\n##
Summary\n\nWe started to rework and introduce functional changes to our
existing\ntest plans for prebuilt rule customization, upgrade, and
export/import\nworkflows.\n\nSpecifically, this PR:\n\n- Restructures
the test plan, introduces a more fine-grained list of\nsections.\n-
Rewrites almost all the existing scenarios. In most cases it boils\ndown
to splitting a scenario into 2+ more specific scenarios, where
each\ndescribes _exactly_ what happens in the GIVEN and THEN sections.
This is\nvery important, as it:\n- makes these scenarios ready to be
implemented right away: 1 scenario =\n1 test to write\n - helps with
ensuring that we covered edge cases\n- Adds new scenarios for handling
missing base versions according
to\n[#210358](https://github.com/elastic/kibana/issues/210358).\n- Adds
a placeholder section for licensing scenarios according
to\n[#11502](https://github.com/elastic/security-team/issues/11502).\n-
Removes scenarios for the `overwrite` request parameter - this
is\ncommon importing logic which is not related to prebuilt rules.\n-
Addresses my own comments from
[this\nreview](https://github.com/elastic/kibana/pull/206893#pullrequestreview-2633863218).\n\nThe
new test plan should be in line with the changes discussed
in\nhttps://github.com//issues/210358.","sha":"a9106c69904e259412df9bbc77b4411507e1d011"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/213434","number":213434,"mergeCommit":{"message":"[Security
Solution] Rework test plan for importing prebuilt rules
(#213434)\n\n**Epic:**
https://github.com/elastic/kibana/issues/174168\n**Partially
addresses:**\nhttps://github.com//issues/202079,\nhttps://github.com//issues/210358\n\n##
Summary\n\nWe started to rework and introduce functional changes to our
existing\ntest plans for prebuilt rule customization, upgrade, and
export/import\nworkflows.\n\nSpecifically, this PR:\n\n- Restructures
the test plan, introduces a more fine-grained list of\nsections.\n-
Rewrites almost all the existing scenarios. In most cases it boils\ndown
to splitting a scenario into 2+ more specific scenarios, where
each\ndescribes _exactly_ what happens in the GIVEN and THEN sections.
This is\nvery important, as it:\n- makes these scenarios ready to be
implemented right away: 1 scenario =\n1 test to write\n - helps with
ensuring that we covered edge cases\n- Adds new scenarios for handling
missing base versions according
to\n[#210358](https://github.com/elastic/kibana/issues/210358).\n- Adds
a placeholder section for licensing scenarios according
to\n[#11502](https://github.com/elastic/security-team/issues/11502).\n-
Removes scenarios for the `overwrite` request parameter - this
is\ncommon importing logic which is not related to prebuilt rules.\n-
Addresses my own comments from
[this\nreview](https://github.com/elastic/kibana/pull/206893#pullrequestreview-2633863218).\n\nThe
new test plan should be in line with the changes discussed
in\nhttps://github.com//issues/210358.","sha":"a9106c69904e259412df9bbc77b4411507e1d011"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
kibanamachine added a commit that referenced this pull request Mar 24, 2025
…es (#213434) (#215795)

# Backport

This will backport the following commits from `main` to `8.x`:
- [[Security Solution] Rework test plan for importing prebuilt rules
(#213434)](#213434)

<!--- Backport version: 9.6.6 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sorenlouv/backport)

<!--BACKPORT [{"author":{"name":"Georgii
Gorbachev","email":"georgii.gorbachev@elastic.co"},"sourceCommit":{"committedDate":"2025-03-24T19:00:36Z","message":"[Security
Solution] Rework test plan for importing prebuilt rules
(#213434)\n\n**Epic:**
https://github.com/elastic/kibana/issues/174168\n**Partially
addresses:**\nhttps://github.com//issues/202079,\nhttps://github.com//issues/210358\n\n##
Summary\n\nWe started to rework and introduce functional changes to our
existing\ntest plans for prebuilt rule customization, upgrade, and
export/import\nworkflows.\n\nSpecifically, this PR:\n\n- Restructures
the test plan, introduces a more fine-grained list of\nsections.\n-
Rewrites almost all the existing scenarios. In most cases it boils\ndown
to splitting a scenario into 2+ more specific scenarios, where
each\ndescribes _exactly_ what happens in the GIVEN and THEN sections.
This is\nvery important, as it:\n- makes these scenarios ready to be
implemented right away: 1 scenario =\n1 test to write\n - helps with
ensuring that we covered edge cases\n- Adds new scenarios for handling
missing base versions according
to\n[#210358](https://github.com/elastic/kibana/issues/210358).\n- Adds
a placeholder section for licensing scenarios according
to\n[#11502](https://github.com/elastic/security-team/issues/11502).\n-
Removes scenarios for the `overwrite` request parameter - this
is\ncommon importing logic which is not related to prebuilt rules.\n-
Addresses my own comments from
[this\nreview](https://github.com/elastic/kibana/pull/206893#pullrequestreview-2633863218).\n\nThe
new test plan should be in line with the changes discussed
in\nhttps://github.com//issues/210358.","sha":"a9106c69904e259412df9bbc77b4411507e1d011","branchLabelMapping":{"^v9.1.0$":"main","^v8.19.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections
and Resp","Team: SecuritySolution","Team:Detection Rule
Management","Feature:Prebuilt Detection
Rules","backport:version","v8.18.0","v9.1.0","v8.19.0"],"title":"[Security
Solution] Rework test plan for importing prebuilt
rules","number":213434,"url":"https://github.com/elastic/kibana/pull/213434","mergeCommit":{"message":"[Security
Solution] Rework test plan for importing prebuilt rules
(#213434)\n\n**Epic:**
https://github.com/elastic/kibana/issues/174168\n**Partially
addresses:**\nhttps://github.com//issues/202079,\nhttps://github.com//issues/210358\n\n##
Summary\n\nWe started to rework and introduce functional changes to our
existing\ntest plans for prebuilt rule customization, upgrade, and
export/import\nworkflows.\n\nSpecifically, this PR:\n\n- Restructures
the test plan, introduces a more fine-grained list of\nsections.\n-
Rewrites almost all the existing scenarios. In most cases it boils\ndown
to splitting a scenario into 2+ more specific scenarios, where
each\ndescribes _exactly_ what happens in the GIVEN and THEN sections.
This is\nvery important, as it:\n- makes these scenarios ready to be
implemented right away: 1 scenario =\n1 test to write\n - helps with
ensuring that we covered edge cases\n- Adds new scenarios for handling
missing base versions according
to\n[#210358](https://github.com/elastic/kibana/issues/210358).\n- Adds
a placeholder section for licensing scenarios according
to\n[#11502](https://github.com/elastic/security-team/issues/11502).\n-
Removes scenarios for the `overwrite` request parameter - this
is\ncommon importing logic which is not related to prebuilt rules.\n-
Addresses my own comments from
[this\nreview](https://github.com/elastic/kibana/pull/206893#pullrequestreview-2633863218).\n\nThe
new test plan should be in line with the changes discussed
in\nhttps://github.com//issues/210358.","sha":"a9106c69904e259412df9bbc77b4411507e1d011"}},"sourceBranch":"main","suggestedTargetBranches":["9.0","8.18","8.x"],"targetPullRequestStates":[{"branch":"9.0","label":"v9.0.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"8.18","label":"v8.18.0","branchLabelMappingKey":"^v(\\d+).(\\d+).\\d+$","isSourceBranch":false,"state":"NOT_CREATED"},{"branch":"main","label":"v9.1.0","branchLabelMappingKey":"^v9.1.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/213434","number":213434,"mergeCommit":{"message":"[Security
Solution] Rework test plan for importing prebuilt rules
(#213434)\n\n**Epic:**
https://github.com/elastic/kibana/issues/174168\n**Partially
addresses:**\nhttps://github.com//issues/202079,\nhttps://github.com//issues/210358\n\n##
Summary\n\nWe started to rework and introduce functional changes to our
existing\ntest plans for prebuilt rule customization, upgrade, and
export/import\nworkflows.\n\nSpecifically, this PR:\n\n- Restructures
the test plan, introduces a more fine-grained list of\nsections.\n-
Rewrites almost all the existing scenarios. In most cases it boils\ndown
to splitting a scenario into 2+ more specific scenarios, where
each\ndescribes _exactly_ what happens in the GIVEN and THEN sections.
This is\nvery important, as it:\n- makes these scenarios ready to be
implemented right away: 1 scenario =\n1 test to write\n - helps with
ensuring that we covered edge cases\n- Adds new scenarios for handling
missing base versions according
to\n[#210358](https://github.com/elastic/kibana/issues/210358).\n- Adds
a placeholder section for licensing scenarios according
to\n[#11502](https://github.com/elastic/security-team/issues/11502).\n-
Removes scenarios for the `overwrite` request parameter - this
is\ncommon importing logic which is not related to prebuilt rules.\n-
Addresses my own comments from
[this\nreview](https://github.com/elastic/kibana/pull/206893#pullrequestreview-2633863218).\n\nThe
new test plan should be in line with the changes discussed
in\nhttps://github.com//issues/210358.","sha":"a9106c69904e259412df9bbc77b4411507e1d011"}},{"branch":"8.x","label":"v8.19.0","branchLabelMappingKey":"^v8.19.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}]
BACKPORT-->

Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co>
@banderror banderror deleted the rework-prebuilt-rule-import-test-plan branch March 25, 2025 09:45
cqliu1 pushed a commit to cqliu1/kibana that referenced this pull request Mar 31, 2025
…astic#213434)

**Epic:** elastic#174168
**Partially addresses:**
elastic#202079,
elastic#210358

## Summary

We started to rework and introduce functional changes to our existing
test plans for prebuilt rule customization, upgrade, and export/import
workflows.

Specifically, this PR:

- Restructures the test plan, introduces a more fine-grained list of
sections.
- Rewrites almost all the existing scenarios. In most cases it boils
down to splitting a scenario into 2+ more specific scenarios, where each
describes _exactly_ what happens in the GIVEN and THEN sections. This is
very important, as it:
- makes these scenarios ready to be implemented right away: 1 scenario =
1 test to write
  - helps with ensuring that we covered edge cases
- Adds new scenarios for handling missing base versions according to
[elastic#210358](elastic#210358).
- Adds a placeholder section for licensing scenarios according to
[elastic#11502](elastic/security-team#11502).
- Removes scenarios for the `overwrite` request parameter - this is
common importing logic which is not related to prebuilt rules.
- Addresses my own comments from [this
review](elastic#206893 (review)).

The new test plan should be in line with the changes discussed in
elastic#210358.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport:version Backport to applied version labels Feature:Prebuilt Detection Rules Security Solution Prebuilt Detection Rules area release_note:skip Skip the PR/issue when compiling release notes Team:Detection Rule Management Security Detection Rule Management Team Team:Detections and Resp Security Detection Response Team Team: SecuritySolution Security Solutions Team working on SIEM, Endpoint, Timeline, Resolver, etc. test-plan v8.18.0 v8.19.0 v9.0.0 v9.1.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants