test: fix flaky test PPOM Blockaid Alert for ERC20 Approve by adding missing mocks by seaona · Pull Request #22617 · MetaMask/metamask-extension

seaona · 2024-01-22T14:25:42Z

Description

There were 2 issues with the test: PPOM Blockaid Alert for ERC20 Approve transactions.

The test was flaky
After checking the artifacts, I've realized that the test was not triggering the Approve ERC20 screen, but a generic Contract Interaction, which was also not expected

For fixing the test, I've added some missing mocks, this results in:

making the test more reliable
displaying the correct ERC20 approve screen (see pics below)

ci runs

Note: other flaky tests might appear ocasionally, but here we should look for the ppom test and make sure it's green all the time:

Run 1 test-e2e-chrome: https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/67549/workflows/96751fb6-3c09-413b-9d0c-f9810ff7475f/jobs/2222452
Run 2 test-e2e-chrome: https://app.circleci.com/pipelines/github/MetaMask/metamask-extension/67549/workflows/96751fb6-3c09-413b-9d0c-f9810ff7475f/jobs/2222551
Run 3 test-e2e-chrome: https://circleci.com/gh/MetaMask/metamask-extension/2222608
Run 4 test-e2e-firefox: https://circleci.com/gh/MetaMask/metamask-extension/2224983

Related issues

Fixes: #22577

Manual testing steps

Trigger ci run
Check test is passing

Screenshots/Recordings

Before

After

Pre-merge author checklist

Pre-merge reviewer checklist

I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

github-actions · 2024-01-22T14:25:54Z

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

seaona · 2024-01-22T14:31:36Z

test/e2e/tests/ppom-blockaid-alert-erc20-approval.spec.js

    ['eth_gasPrice'],
    ['eth_getBalance'],
    ['eth_getBlockByNumber'],
+    ['eth_createAccessList'],


adding a couple of more mocks to requests that are also triggered by ppom when performing the approve ERC20 action, to stabilize the test

seaona · 2024-01-22T16:15:15Z

test/e2e/tests/ppom-blockaid-alert-erc20-approval.spec.js

+    [
+      'eth_call',
+      {
+        methodResultVariant: 'balanceApprove',


this mock fixes the "not displaying the ERC20 approve screeen"

metamaskbot · 2024-01-22T16:41:00Z

Builds ready [8772b80]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4
- common: 0, 1, 2, 3, 4, 5, 6, 7
- content-script: 0
- contentscript: contentscript
- disable-console: disable-console
- inpage: inpage
- policy-load: policy-load
- sentry-install: sentry-install
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (1259 ± 146 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	86	307	164	68	33
		domContentLoaded	10	219	65	76	36
		load	809	1846	1259	303	146
		domInteractive	10	218	65	76	36

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

metamaskbot · 2024-01-23T06:51:47Z

Builds ready [49043c2]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4
- common: 0, 1, 2, 3, 4, 5, 6, 7
- content-script: 0
- contentscript: contentscript
- disable-console: disable-console
- inpage: inpage
- policy-load: policy-load
- sentry-install: sentry-install
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (1609 ± 117 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	99	318	162	53	26
		domContentLoaded	10	71	27	17	8
		load	1080	2154	1609	244	117
		domInteractive	10	71	27	17	8

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

codecov · 2024-01-23T06:52:19Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Comparison is base (35273da) 68.11% compared to head (49043c2) 68.11%.
Report is 3 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop   #22617   +/-   ##
========================================
  Coverage    68.11%   68.11%           
========================================
  Files         1086     1086           
  Lines        42628    42628           
  Branches     11340    11340           
========================================
+ Hits         29033    29035    +2     
+ Misses       13595    13593    -2

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

add missing mocks

be53b60

seaona added the team-confirmations-secure-ux-PR PRs from the confirmations team label Jan 22, 2024

seaona self-assigned this Jan 22, 2024

seaona commented Jan 22, 2024

View reviewed changes

seaona added 2 commits January 22, 2024 15:47

add balance approve mock

51353f9

fix: reset block number

8772b80

seaona marked this pull request as ready for review January 22, 2024 16:01

seaona requested a review from a team as a code owner January 22, 2024 16:01

seaona commented Jan 22, 2024

View reviewed changes

jpuri approved these changes Jan 22, 2024

View reviewed changes

blackdevelopa approved these changes Jan 22, 2024

View reviewed changes

Merge branch 'develop' into test-flaky-ppom-erc20

49043c2

digiwand approved these changes Jan 22, 2024

View reviewed changes

seaona merged commit f69e67d into develop Jan 23, 2024

seaona deleted the test-flaky-ppom-erc20 branch January 23, 2024 10:11

github-actions bot locked and limited conversation to collaborators Jan 23, 2024

metamaskbot added the release-11.10.0 Issue or pull request that will be included in release 11.10.0 label Jan 23, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: fix flaky test PPOM Blockaid Alert for ERC20 Approve by adding missing mocks#22617

test: fix flaky test PPOM Blockaid Alert for ERC20 Approve by adding missing mocks#22617
seaona merged 4 commits intodevelopfrom
test-flaky-ppom-erc20

seaona commented Jan 22, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Jan 22, 2024

Uh oh!

seaona Jan 22, 2024 •

edited

Loading

Uh oh!

seaona Jan 22, 2024

Uh oh!

metamaskbot commented Jan 22, 2024

Uh oh!

metamaskbot commented Jan 23, 2024

Uh oh!

codecov bot commented Jan 23, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

seaona commented Jan 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

ci runs

Related issues

Manual testing steps

Screenshots/Recordings

Before

After

Pre-merge author checklist

Pre-merge reviewer checklist

Uh oh!

github-actions bot commented Jan 22, 2024

Uh oh!

seaona Jan 22, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

seaona Jan 22, 2024

Choose a reason for hiding this comment

Uh oh!

metamaskbot commented Jan 22, 2024

Uh oh!

metamaskbot commented Jan 23, 2024

Uh oh!

codecov bot commented Jan 23, 2024

Codecov Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

seaona commented Jan 22, 2024 •

edited

Loading

seaona Jan 22, 2024 •

edited

Loading