[Security Solution] Exceptions Cypress tests by MadameSheema · Pull Request #81759 · elastic/kibana

MadameSheema · 2020-10-27T12:39:52Z

Summary

In this PR we are adding some tests in order to check that exceptions feature is working fine.

These tests were complex to develop so I will appreciate a deep look to it, because I'm afraid I can be messing or not properly checking things.

For these tests we have created different archives:

rule_for_exceptions
rule_for_exceptions_from_alert
auditbeat_for_exceptions
auditbeat_for_exceptions2
auditbeat_for_exceptions3
auditbeat_for_exceptions_from_alert
auditbeat_for_exceptions_from_alert2
auditbeat_for_exceptions_from_alert3

The rule_for_exceptions archive contains a custom rule with query host.name:* and index pattern exceptions-*. As on CI all the Kibana tests are executed in parallel, this was the selected index pattern in order to avoid collisions with different tests from different teams using a common index pattern (i.e. auditbeat).

The rule_for_exceptions_from_alert archive contains a rule with archive contains a custom rule with query host.name:* and index pattern exceptionsalert-*. This index pattern was selected from the same reason as the previous one and also, to prevent collisions with the previous test.

auditbeat_for_exceptions, auditbeat_for_exceptions2 and auditbeat_for_exceptions3, each archive contains 2 different alerts that matches the exceptions-* pattern.

auditbeat_for_exceptions_from_alert, auditbeat_for_exceptions_from_alert2, auditbeat_for_exceptions_from_alert3, each archive contains 2 different alerts that thames the exceptionsalert-* pattern.

Creates an exception from rule details and deletes the exception implementation:

We load the rule
We activate it
We load the first set of data
We wait for the rule to be executed
We deactivate the rule
We check that the rule has populated the expected alerts
We add an exception from the rule details for all the host.names that matches with the alerts on the archive, checking the closing alerts option
We activate the rule again
We load a new set of data
We wait for the rule to be executed again
We navigate to closed alerts in order to check that the alerts have been closed
We navigate to opened alerts and we check that no new rules have been populated
We delete the exception
We load a new set of data
We wait for the rule to be executed
We check that the rule has populated the expected alerts

Creates an exception from an existing alert and deletes the exception implementation:

We load the rule
We activate it
We load the first set of data
We wait for the rule to be executed
We deactivate the rule
We check that the rule has populated the expected alerts
We add an exception from the first existing alert for all the host.names that matches with the alerts on the archive, checking the closing alerts option
We activate the rule again
We load a new set of data
We wait for the rule to be executed again
We navigate to closed alerts in order to check that the alerts have been closed
We navigate to opened alerts and we check that no new rules have been populated
We delete the exception
We load a new set of data
We wait for the rule to be executed
We check that the rule has populated the expected alerts

…n' test

# Conflicts: # x-pack/plugins/security_solution/cypress/integration/alerts_detection_rules_custom.spec.ts # x-pack/plugins/security_solution/cypress/integration/alerts_detection_rules_ml.spec.ts # x-pack/plugins/security_solution/cypress/integration/alerts_detection_rules_override.spec.ts # x-pack/plugins/security_solution/cypress/integration/alerts_detection_rules_threshold.spec.ts # x-pack/plugins/security_solution/cypress/screens/rule_details.ts # x-pack/plugins/security_solution/cypress/tasks/create_new_rule.ts

…into improves-detections

# Conflicts: # x-pack/plugins/security_solution/cypress/integration/alerts_detection_rules_custom.spec.ts # x-pack/plugins/security_solution/cypress/integration/alerts_detection_rules_ml.spec.ts # x-pack/plugins/security_solution/cypress/integration/alerts_detection_rules_override.spec.ts # x-pack/plugins/security_solution/cypress/integration/alerts_detection_rules_threshold.spec.ts # x-pack/plugins/security_solution/cypress/screens/rule_details.ts # x-pack/plugins/security_solution/cypress/tasks/create_new_rule.ts

…etion' implementation

…ion'

MadameSheema · 2020-11-25T14:03:28Z

@elasticmachine merge upstream

peluja1012

LGTM. Thanks for added coverage and for helping debug the "conflict" issue!

patrykkopycinski · 2020-11-27T17:03:44Z

@elasticmachine merge upstream

MadameSheema · 2020-11-30T07:55:47Z

@elasticmachine merge upstream

kibanamachine · 2020-11-30T09:34:44Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 6f57aae

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	8.0MB	8.0MB	+159.0B

Distributable file count

id	before	after	diff
`default`	43134	43139	+5

History

💚 Build #90552 succeeded ee2a570
💚 Build #90548 succeeded 661cd54766e049aec2aa71f82e415001d90141db
💚 Build #90540 succeeded 9fc8984
💔 Build #90534 failed df1b01e
💔 Build #90527 failed a32a240

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* improves 'Creates and activates a new custom rule' test * fixes constant problem * improves 'Creates and activates a new custom rule with override option' test * improves 'Creates and activates a new threshold rule' test * refactor * fixes type check issue * improves assertions * removes unused code * changes variables for constants * improves 'waitForTheRuleToBeExecuted' test * improves readability * fixes jenkins error * refactor * blah * more things * finishes 'Creates an exception from rule details and deletes the excpetion' implementation * implements 'Creates an exception from an alert and deletes the exception' * updates VALUES_INPUT locator * updates archiver * refactor * improves the code * fixes CI error * renames exceptions archive * refactor * fixes merge issue * fixes CI issue * debug * refactor * improves test data * removes signals index after the execution * removes unused line * removes unused variable * refactors 'numberOfauditbeatExceptionsAlerts' constant to camel case * simplifies the archive * waits for the rule to be executed after navigating to opened alerts tab * cleaning data * fixes tests flakiness * cleans test data * refactors code * removes unsused archives * cleans data * simplifies data * fixes CI issue Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

* master: [Security Solution] Exceptions Cypress tests (elastic#81759) [ML] Fix spaces job ID check (elastic#84404) [Security Solution][Detections] Handle dupes when processing threshold rules (elastic#83062) skip flaky suite (elastic#84440) skip flaky suite (elastic#84445) [APM] Fix missing `service.node.name` (elastic#84269) Upgrade fp-ts to 2.8.6 (elastic#83866) Added data streams privileges to better control delete actions in UI (elastic#83573) Improve short-url redirect validation (elastic#84366) TSVB offsets (elastic#83051) [Discover] Fix navigating back when changing index pattern (elastic#84061) [Logs UI] Polish the UI for the log entry examples in the anomaly table (elastic#82139) [Logs UI] Limit the height of the "view in context" container (elastic#83178) [Application Usage] Update `schema` with new `fleet` rename (elastic#84327) fix identation in list (elastic#84301)

* improves 'Creates and activates a new custom rule' test * fixes constant problem * improves 'Creates and activates a new custom rule with override option' test * improves 'Creates and activates a new threshold rule' test * refactor * fixes type check issue * improves assertions * removes unused code * changes variables for constants * improves 'waitForTheRuleToBeExecuted' test * improves readability * fixes jenkins error * refactor * blah * more things * finishes 'Creates an exception from rule details and deletes the excpetion' implementation * implements 'Creates an exception from an alert and deletes the exception' * updates VALUES_INPUT locator * updates archiver * refactor * improves the code * fixes CI error * renames exceptions archive * refactor * fixes merge issue * fixes CI issue * debug * refactor * improves test data * removes signals index after the execution * removes unused line * removes unused variable * refactors 'numberOfauditbeatExceptionsAlerts' constant to camel case * simplifies the archive * waits for the rule to be executed after navigating to opened alerts tab * cleaning data * fixes tests flakiness * cleans test data * refactors code * removes unsused archives * cleans data * simplifies data * fixes CI issue Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>

…bana into add-metadata-to-node-details * 'add-metadata-to-node-details' of github.com:phillipb/kibana: [APM] ML anomaly detection integration: Displaying anomaly job results in the Transaction duration chart is not as intended (elastic#84415) Support for painless language autocomplete within monaco (elastic#80577) [Lens] Time scale ui (elastic#83904) removing beta callouts (elastic#84510) [Lens] (Accessibility) add aria-label to chart type icon (elastic#84493) Trusted Apps signer API. (elastic#83661) increase stdout max listeners for legacy logging (elastic#84497) [APM] Service overview: Add throughput chart (elastic#84439) [Discover] Unskip main functional tests (elastic#84300) Uptime overview overhaul (elastic#83406) [APM] Adjust time formats based on the difference between start and end (elastic#84470) [ML] Renaming saved object repair to sync (elastic#84311) [UsageCollection] Remove `formatBulkUpload` and other unused APIs (elastic#84313) [Visualizations] Adds visConfig.title and uiState to build pipeline function (elastic#84456) [Elasticsearch Migration] Update docs re UsageCollection (elastic#84322) TSVB field list performance issue on using annotations (elastic#84407) [Security Solution] Exceptions Cypress tests (elastic#81759) [ML] Fix spaces job ID check (elastic#84404) [Security Solution][Detections] Handle dupes when processing threshold rules (elastic#83062)

MadameSheema and others added 30 commits September 15, 2020 21:46

improves 'Creates and activates a new custom rule' test

ee00b9f

fixes constant problem

5e3f5d5

improves 'Creates and activates a new custom rule with override optio…

0b5afe1

…n' test

improves 'Creates and activates a new threshold rule' test

0c34e9b

refactor

56c4c92

fixes type check issue

5ac8324

Merge branch 'master' into improves-detections

579fb54

improves assertions

9486073

Merge branch 'improves-detections' of github.com:MadameSheema/kibana …

f41a9e8

…into improves-detections

Merge branch 'master' into improves-detections

a1948b1

Merge branch 'master' into improves-detections

5a48b9b

removes unused code

578ce01

changes variables for constants

f9c3555

improves 'waitForTheRuleToBeExecuted' test

6fe2d60

improves readability

905ee85

Merge branch 'master' into improves-detections

c061b7a

Merge branch 'master' into improves-detections

da955b6

Merge branch 'master' into improves-detections

6a2ec50

Merge branch 'master' into improves-detections

26e6298

fixes jenkins error

492cf98

Merge branch 'master' into improves-detections

2fe6861

refactor

1e7dde8

Merge branch 'improves-detections' of github.com:MadameSheema/kibana …

3b8ca5e

…into improves-detections

blah

2d658a5

more things

1b65879

finishes 'Creates an exception from rule details and deletes the excp…

edd4e7b

…etion' implementation

implements 'Creates an exception from an alert and deletes the except…

85da013

…ion'

Merge branch 'master' into exceptions

1c5ff10

kibanamachine and others added 2 commits November 25, 2020 09:03

Merge branch 'master' into exceptions

40ed914

removes unsused archives

563e5cf

peluja1012 approved these changes Nov 25, 2020

View reviewed changes

MadameSheema added 2 commits November 25, 2020 18:07

cleans data

9916505

simplifies data

2ef6b65

MadameSheema force-pushed the exceptions branch from 69226bd to 626c639 Compare November 26, 2020 10:37

MadameSheema requested a review from a team as a code owner November 26, 2020 16:53

MadameSheema force-pushed the exceptions branch 9 times, most recently from df1b01e to 537aad6 Compare November 27, 2020 15:05

fixes CI issue

ee2a570

MadameSheema force-pushed the exceptions branch from 661cd54 to ee2a570 Compare November 27, 2020 17:53

Merge branch 'master' into exceptions

6f57aae

MadameSheema merged commit 4546352 into elastic:master Nov 30, 2020

MadameSheema deleted the exceptions branch November 30, 2020 09:37

MadameSheema mentioned this pull request Nov 30, 2020

[7.x] [Security Solution] Exceptions Cypress tests (#81759) #84469

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution] Exceptions Cypress tests#81759

[Security Solution] Exceptions Cypress tests#81759
MadameSheema merged 68 commits intoelastic:masterfrom
MadameSheema:exceptions

MadameSheema commented Oct 27, 2020 •

edited

Loading

Uh oh!

MadameSheema commented Nov 25, 2020

Uh oh!

peluja1012 left a comment

Uh oh!

patrykkopycinski commented Nov 27, 2020

Uh oh!

MadameSheema commented Nov 30, 2020

Uh oh!

kibanamachine commented Nov 30, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

Conversation

MadameSheema commented Oct 27, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Creates an exception from rule details and deletes the exception implementation:

Creates an exception from an existing alert and deletes the exception implementation:

Uh oh!

MadameSheema commented Nov 25, 2020

Uh oh!

peluja1012 left a comment

Choose a reason for hiding this comment

Uh oh!

patrykkopycinski commented Nov 27, 2020

Uh oh!

MadameSheema commented Nov 30, 2020

Uh oh!

kibanamachine commented Nov 30, 2020

💚 Build Succeeded

Metrics [docs]

Async chunks

Distributable file count

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

MadameSheema commented Oct 27, 2020 •

edited

Loading