[Security Solution] Fixes flaky test rules#81040
Merged
MadameSheema merged 6 commits intoelastic:masterfrom Oct 21, 2020
Merged
[Security Solution] Fixes flaky test rules#81040MadameSheema merged 6 commits intoelastic:masterfrom
MadameSheema merged 6 commits intoelastic:masterfrom
Conversation
Contributor
Author
|
@elasticmachine merge upstream |
Contributor
|
Pinging @elastic/siem (Team:SIEM) |
rylnd
approved these changes
Oct 21, 2020
Contributor
rylnd
left a comment
There was a problem hiding this comment.
I had one question about ordering and you've got some merge conflicts (from me, sorry 😄 ), but CI should take care of those so you get a pre-emptive approval! Thanks for this fix.
|
|
||
| after(() => { | ||
| esArchiverUnload('timeline'); | ||
| deleteRule(); |
Contributor
There was a problem hiding this comment.
Are these two operations totally independent? I assume so but wanted to call out the discrepancy in ordering.
Contributor
Author
There was a problem hiding this comment.
In this specific case, does not matter the order, but you are right, in general, it should not be like that, good catch! I need to update this 😊
Contributor
💚 Build SucceededMetrics [docs]
History
To update your PR or re-run it, just comment with: |
MadameSheema
added a commit
to MadameSheema/kibana
that referenced
this pull request
Oct 21, 2020
* deletes the created rule in the after hook * fixes failing test * changes rule deletion order Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
MadameSheema
added a commit
to MadameSheema/kibana
that referenced
this pull request
Oct 21, 2020
* deletes the created rule in the after hook * fixes failing test * changes rule deletion order Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
MadameSheema
added a commit
that referenced
this pull request
Oct 21, 2020
* deletes the created rule in the after hook * fixes failing test * changes rule deletion order Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
MadameSheema
added a commit
that referenced
this pull request
Oct 21, 2020
* deletes the created rule in the after hook * fixes failing test * changes rule deletion order Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com> Co-authored-by: Kibana Machine <42973632+kibanamachine@users.noreply.github.com>
jloleysens
added a commit
to jloleysens/kibana
that referenced
this pull request
Oct 21, 2020
…arm-phase-to-formlib * 'master' of github.com:elastic/kibana: (55 commits) [UX] Fix map color variance and apply proper filter for extended stats (elastic#81106) [User Experience] Use EuiSelect for percentiles instead of SuperSelect (elastic#81082) [DOCS] Add link for monitoring ssl settings (elastic#81057) [test] Await loading indicator in monitoring test (elastic#81279) [ILM] Minor copy and link additions to cloud CTA for cold phase (elastic#80512) [Mappings editor] Add scaled_float and date_range comp integration tests (elastic#81287) [Discover] Deangularize context.app (elastic#80851) [O11y Overview] Add code to display/hide UX section when appropriate (elastic#80873) [Discover] Extend DiscoverNoResults component to show different message on error (elastic#79671) Fix tagcloud word overlapping (elastic#81161) [Security Solution] Fixes flaky test rules (elastic#81040) Changed the code to avoid tech debt with hacky solutions after receiving comments on EUI issue reported about this problem. (elastic#81183) [Security Solution][All] Replace old markdown renderer with the new one (elastic#80301) Add namespaced version of the API call (elastic#81278) [ML] Data Frame Analytics: Fix race condition and support for feature influence legacy format. (elastic#81123) [Fleet] Fix POLICY_CHANGE action creation for new policy (elastic#81236) [Security Solution][Endpoint][Admin] Malware user notification checkbox (elastic#78084) [SecuritySolution][Unit Tests] - fix flakey unit test (elastic#81239) skip flaky suite (elastic#81264) [Maps] fix top-level Map page is called 'Kibana' (elastic#81238) ... # Conflicts: # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/hot_phase/hot_phase.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/shared/forcemerge_field.tsx # x-pack/plugins/index_lifecycle_management/public/application/sections/edit_policy/components/phases/warm_phase.tsx
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
We saw some rule test failing due to a not proper clean up after the execution. This is because right now our Cypress tests are executed 2 more times if there is a failure. If the failure it is due to an assertion or timeout after the rule creation, the next execution is going to fail due to a mismatch in the number of created rules.
In order to fix that flakiness now we are deleting the created rules at the after hook.