Skip to content

test: fix flaky test Migrate Opensea to Blockaid Banner @no-mmi Shows up on Token Approval transaction confirmations#24739

Merged
seaona merged 1 commit intodevelopfrom
flaky-fix-opensea-banner
May 23, 2024
Merged

test: fix flaky test Migrate Opensea to Blockaid Banner @no-mmi Shows up on Token Approval transaction confirmations#24739
seaona merged 1 commit intodevelopfrom
flaky-fix-opensea-banner

Conversation

@seaona
Copy link
Copy Markdown
Member

@seaona seaona commented May 23, 2024

Description

This PR attempts to fix the flaky test Migrate Opensea to Blockaid Banner @no-mmi Shows up on Token Approval transaction confirmations

This test fails whenever we click Create Token for deploying an ERC20 token, as the popup is not triggered and we remain in the test dapp.

I haven't been able to reproduce this failure locally. However, after looking into the code and the context of flakiness, I can confirm that:

  • I've seen that we are doing several unnecessary steps for the test setup, like connecting to the test dapp, or deploying an ERC20 contract. This can be achieved by fixtures, so I've added the fixtures and got rid of a good chunk of the code. The good part is that the test failure precisely happens on the test setup, which is now removed
  • It might be that there is a race condition between the Connect to the test Dapp and the Create Token steps, where the dapp has not yet the connected address for triggering the Create Token but the button is already enabled, resulting in the error in the test dapp unknown account and popup not being opened (see video below). However, those 2 steps are now removed from the test, meaning any potential race condition at this point won't happen anymore
  • I've added the logInWithBalanceValidation instead of the unlockWallet to make sure the balance is loaded before triggering the interaction

For the above points, I have strong reasons for wanting to implement the changes on this PR, despite not being 100% sure of the above being the root cause.

Open in GitHub Codespaces

Related issues

Fixes: another item of this list #24603

Manual testing steps

  1. Run this test locally several times `yarn test:e2e:single test/e2e/tests/ppom/migrate-opensea-to-blockaid-banner.spec.js --browser=firefox --leave-running --retryUntilFailure --retries=10
  2. Check ci jobs

Screenshots/Recordings

Screenshot from 2024-05-23 11-14-32

image

This is not the test environment, however here I demonstrate how would look like if the create token is triggered before the dapp has the connected account resolved

unknown-accont.mp4

Pre-merge author checklist

  • I’ve followed MetaMask Coding Standards.
  • I've completed the PR template to the best of my ability
  • I’ve included tests if applicable
  • I’ve documented my code using JSDoc format if applicable
  • I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

@github-actions
Copy link
Copy Markdown
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

await connectAccountToTestDapp(driver);

await driver.switchToWindowWithTitle(WINDOW_TITLES.TestDApp);
await driver.findClickableElement('#createToken');
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the flaky test fails at this point - now we are removing completely this step as we use the deployed contract from fixtures

@seaona seaona added team-extension-platform Extension Platform team flaky tests labels May 23, 2024
@seaona seaona self-assigned this May 23, 2024
@metamaskbot
Copy link
Copy Markdown
Collaborator

Builds ready [afaceac]
Page Load Metrics (1610 ± 610 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint632411345024
domContentLoaded107222189
load52311616101270610
domInteractive107222189
Bundle size diffs
  • background: 0 Bytes (0.00%)
  • ui: 0 Bytes (0.00%)
  • common: 0 Bytes (0.00%)

@seaona seaona marked this pull request as ready for review May 23, 2024 10:02
@seaona seaona requested a review from a team as a code owner May 23, 2024 10:02

await connectAccountToTestDapp(driver);

await driver.switchToWindowWithTitle(WINDOW_TITLES.TestDApp);
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can skip these steps, since now we are already connected to the test dapp

@seaona seaona added the needs-dev-review PR needs reviews from other engineers (in order to receive required approvals) label May 23, 2024
@legobeat legobeat requested a review from a team May 23, 2024 11:36
@seaona seaona merged commit a8862df into develop May 23, 2024
@seaona seaona deleted the flaky-fix-opensea-banner branch May 23, 2024 12:26
@github-actions github-actions bot locked and limited conversation to collaborators May 23, 2024
@github-actions github-actions bot removed the needs-dev-review PR needs reviews from other engineers (in order to receive required approvals) label May 23, 2024
@metamaskbot metamaskbot added release-11.16.6 Issue or pull request that will be included in release 11.16.6 and removed release-11.18.0 labels Jun 4, 2024
@metamaskbot
Copy link
Copy Markdown
Collaborator

Missing release label release-11.16.6 on PR. Adding release label release-11.16.6 on PR and removing other release labels(release-11.18.0), as PR was cherry-picked in branch 11.16.6.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

flaky tests release-11.16.6 Issue or pull request that will be included in release 11.16.6 team-extension-platform Extension Platform team

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

4 participants