Fix "Snap account signatures" flaky tests#22614
Merged
HowardBraham merged 1 commit intodevelopfrom Jan 24, 2024
Merged
Conversation
Contributor
|
CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## develop #22614 +/- ##
========================================
Coverage 68.12% 68.12%
========================================
Files 1086 1086
Lines 42622 42624 +2
Branches 11341 11341
========================================
+ Hits 29033 29035 +2
Misses 13589 13589 ☔ View full report in Codecov by Sentry. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
I can't say that this is completely fixed, but it's much much better.
The changes
driver.clickElementon the newdata-testidthat became available in 1.1.1driver.delayTEST_SNAPS_SIMPLE_KEYRING_WEBSITE_URLin more places, which required moving it to a different file for Storybook usageMy testing procedure
I ran the 12 tests in the
test/e2e/accountsfolder 625 times each...yes, I seriously did run them 625 times because I ran the whole suite on
5 x 25 x 5 = 625 runs of the test suite
625 runs x 12 tests = 7,500 individual tests
Out of those 7,500 individual tests, there were only 13 failures. Not zero, but much much better than before.
You can see these results with 7,500 tests here: https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/67476/workflows/b1a5ccea-c241-41d9-8247-5e71909dde83
Related issues
Fixes: #22575
Manual testing steps
Screenshots/Recordings
Pre-merge author checklist
Pre-merge reviewer checklist