Skip to content

Commit 742e66d

Browse files
dplumleeelasticmachinebanderror
committed
[Security Solution] Adds prebuilt rule import/export integration tests (#206893)
## Summary Adds integration tests in accordance to #204889 Adds on to the existing tests we have for rule import and export to include tests related to the prebuilt rule customization epic and the new functionality that will be shipping. All these tests are running behind the `prebuiltRulesCustomizationEnabled` feature flag. ### Checklist Check the PR satisfies following conditions. Reviewers should verify this PR satisfies this list as well. - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [x] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed - [x] ESS x100: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7921 - [x] Serverless x100: https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/7922 --------- Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Georgii Gorbachev <georgii.gorbachev@elastic.co> (cherry picked from commit 3e4ed6e)
1 parent 9fadbff commit 742e66d

3 files changed

Lines changed: 219 additions & 38 deletions

File tree

  • x-pack
    • solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules
    • test
      • security_solution_api_integration/test_suites/detections_response/rules_management/prebuilt_rules/prebuilt_rule_customization/customization_enabled
      • security_solution_cypress/cypress/tasks

‎x-pack/solutions/security/plugins/security_solution/docs/testing/test_plans/detection_response/prebuilt_rules/prebuilt_rule_import.md‎

Lines changed: 36 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -27,20 +27,20 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one
2727
- [Non-functional requirements](#non-functional-requirements)
2828
- [Scenarios](#scenarios)
2929
- [Core Functionality](#core-functionality)
30-
- [Scenario: Importing an unmodified prebuilt rule with a matching rule\_id and version](#scenario-importing-an-unmodified-prebuilt-rule-with-a-matching-rule_id-and-version)
31-
- [Scenario: Importing a customized prebuilt rule with a matching rule\_id and version](#scenario-importing-a-customized-prebuilt-rule-with-a-matching-rule_id-and-version)
32-
- [Scenario: Importing a custom rule with a matching rule\_id and version](#scenario-importing-a-custom-rule-with-a-matching-rule_id-and-version)
33-
- [Scenario: Importing a prebuilt rule with a matching rule\_id but no matching version](#scenario-importing-a-prebuilt-rule-with-a-matching-rule_id-but-no-matching-version)
34-
- [Scenario: Importing a prebuilt rule with a non-existent rule\_id](#scenario-importing-a-prebuilt-rule-with-a-non-existent-rule_id)
35-
- [Scenario: Importing a prebuilt rule without a rule\_id field](#scenario-importing-a-prebuilt-rule-without-a-rule_id-field)
36-
- [Scenario: Importing a prebuilt rule with a matching rule\_id but missing a version field](#scenario-importing-a-prebuilt-rule-with-a-matching-rule_id-but-missing-a-version-field)
30+
- [Scenario: Importing an unmodified prebuilt rule with a matching rule_id and version](#scenario-importing-an-unmodified-prebuilt-rule-with-a-matching-rule_id-and-version)
31+
- [Scenario: Importing a customized prebuilt rule with a matching rule_id and version](#scenario-importing-a-customized-prebuilt-rule-with-a-matching-rule_id-and-version)
32+
- [Scenario: Importing a custom rule with a matching rule_id and version](#scenario-importing-a-custom-rule-with-a-matching-rule_id-and-version)
33+
- [Scenario: Importing a prebuilt rule with a matching rule_id but no matching version](#scenario-importing-a-prebuilt-rule-with-a-matching-rule_id-but-no-matching-version)
34+
- [Scenario: Importing a prebuilt rule with a non-existent rule_id](#scenario-importing-a-prebuilt-rule-with-a-non-existent-rule_id)
35+
- [Scenario: Importing a prebuilt rule without a rule_id field](#scenario-importing-a-prebuilt-rule-without-a-rule_id-field)
36+
- [Scenario: Importing a prebuilt rule with a matching rule_id but missing a version field](#scenario-importing-a-prebuilt-rule-with-a-matching-rule_id-but-missing-a-version-field)
3737
- [Scenario: Importing an existing custom rule missing a version field](#scenario-importing-an-existing-custom-rule-missing-a-version-field)
3838
- [Scenario: Importing a new custom rule missing a version field](#scenario-importing-a-new-custom-rule-missing-a-version-field)
3939
- [Scenario: Importing a rule with overwrite flag set to true](#scenario-importing-a-rule-with-overwrite-flag-set-to-true)
4040
- [Scenario: Importing a rule with overwrite flag set to false](#scenario-importing-a-rule-with-overwrite-flag-set-to-false)
4141
- [Scenario: Importing both custom and prebuilt rules](#scenario-importing-both-custom-and-prebuilt-rules)
4242
- [Scenario: Importing prebuilt rules when the rules package is not installed](#scenario-importing-prebuilt-rules-when-the-rules-package-is-not-installed)
43-
- [Scenario: User imports a custom rule before a prebuilt rule asset is created with the same rule\_id](#scenario-user-imports-a-custom-rule-before-a-prebuilt-rule-asset-is-created-with-the-same-rule_id)
43+
- [Scenario: User imports a custom rule before a prebuilt rule asset is created with the same rule_id](#scenario-user-imports-a-custom-rule-before-a-prebuilt-rule-asset-is-created-with-the-same-rule_id)
4444

4545
## Useful information
4646

@@ -83,8 +83,8 @@ https://marketplace.visualstudio.com/items?itemName=yzhang.markdown-all-in-one
8383
**Automation**: 1 cypress test and 1 integration test.
8484

8585
```Gherkin
86-
Given the import payload contains a prebuilt rule with a matching rule_id and version, identical to the published rule
87-
When the user imports the rule
86+
Given the import payload contains an unmodified prebuilt rule
87+
And its rule_id and version match a rule asset from the installed package
8888
Then the rule should be created or updated
8989
And the ruleSource type should be "external"
9090
And isCustomized should be false
@@ -95,25 +95,34 @@ And isCustomized should be false
9595
**Automation**: 1 cypress test and 1 integration test.
9696

9797
```Gherkin
98-
Given the import payload contains a prebuilt rule with a matching rule_id and version, modified from the published version
99-
And the overwrite flag is set to true
98+
Given the import payload contains a modified prebuilt rule
99+
And its rule_id and version match a rule asset from the installed package
100100
When the user imports the rule
101101
Then the rule should be created or updated
102102
And the ruleSource type should be "external"
103103
And isCustomized should be true
104+
```
105+
106+
#### Scenario: Importing a custom rule with a matching prebuilt rule_id and version
104107

105-
CASE: Should work with older, newer, or identical version numbers
108+
**Automation**: 1 cypress test and 1 integration test.
109+
110+
```Gherkin
111+
Given the import payload contains a custom rule with a matching rule_id and version
112+
When the user imports the rule
113+
Then the rule should be created or updated
114+
And the ruleSource type should be "external"
106115
```
107116

108-
#### Scenario: Importing a custom rule with a matching rule_id and version
117+
#### Scenario: Importing a custom rule with a matching custom rule_id and version
109118

110119
**Automation**: 1 cypress test and 1 integration test.
111120

112121
```Gherkin
113122
Given the import payload contains a custom rule with a matching rule_id and version
114123
And the overwrite flag is set to true
115124
When the user imports the rule
116-
Then the rule should be updated
125+
Then the rule should be created or updated
117126
And the ruleSource type should be "internal"
118127
```
119128

@@ -122,10 +131,11 @@ And the ruleSource type should be "internal"
122131
**Automation**: 1 integration test.
123132

124133
```Gherkin
125-
Given the import payload contains a prebuilt rule with a matching rule_id but no matching version
126-
And the overwrite flag is set to true
134+
Given the import payload contains a prebuilt rule
135+
And its rule_id matches a rule asset from the installed package
136+
And the version does not match the rule asset's version
127137
When the user imports the rule
128-
Then the rule should be created
138+
Then the rule should be created or updated
129139
And the ruleSource type should be "external"
130140
And isCustomized should be true
131141
```
@@ -135,7 +145,8 @@ And isCustomized should be true
135145
**Automation**: 1 integration test.
136146

137147
```Gherkin
138-
Given the import payload contains a prebuilt rule with a non-existent rule_id
148+
Given the import payload contains a prebuilt rule
149+
And its rule_id does NOT match a rule asset from the installed package
139150
When the user imports the rule
140151
Then the rule should be created
141152
And the ruleSource type should be "internal"
@@ -190,19 +201,21 @@ And the "version" field should be set to 1
190201
**Automation**: 1 integration test.
191202

192203
```Gherkin
193-
Given the import payload contains a rule with an existing rule_id
204+
Given the import payload contains a rule
205+
And its rule_id matches a rule_id of one of the installed rules
194206
And the overwrite flag is set to true
195207
When the user imports the rule
196208
Then the rule should be overwritten
197-
And the ruleSource type should be calculated based on the rule_id and version
209+
And the ruleSource should be based on rule_id and version
198210
```
199211

200212
#### Scenario: Importing a rule with overwrite flag set to false
201213

202214
**Automation**: 1 integration test.
203215

204216
```Gherkin
205-
Given the import payload contains a rule with an existing rule_id
217+
Given the import payload contains a rule
218+
And its rule_id matches a rule_id of one of the installed rules
206219
And the overwrite flag is set to false
207220
When the user imports the rule
208221
Then the import should be rejected with a message "rule_id already exists"
@@ -230,7 +243,7 @@ And prebuilt rules missing versions should be rejected
230243
Given the import payload contains prebuilt rules
231244
And no rules package has been installed locally
232245
When the user imports the rule
233-
Then all rules should be created or updated as custom rules
246+
Then the latest prebuilt rules package should get installed automatically
234247
```
235248

236249
#### Scenario: User imports a custom rule before a prebuilt rule asset is created with the same rule_id

0 commit comments

Comments
 (0)