Skip to content

test: Fix flaky test Update Network: update network details and validate the ui elements#24790

Merged
DDDDDanica merged 1 commit intodevelopfrom
chloe-flaky-test-24628
May 27, 2024
Merged

test: Fix flaky test Update Network: update network details and validate the ui elements#24790
DDDDDanica merged 1 commit intodevelopfrom
chloe-flaky-test-24628

Conversation

@chloeYue
Copy link
Copy Markdown
Contributor

@chloeYue chloeYue commented May 27, 2024

Description

This PR addresses a flaky test issue related to the Chain ID input field.

Reason for flakiness: The root cause of the flakiness was identified as the rapid input of the entire Chain ID, which resulted in the error message appearing and persisting even after the correct Chain ID was fully entered.

Observation: When entering the Chain ID manually, the error message correctly displays when incomplete values like "0x53" are input but disappears upon entering the full Chain ID with "9". To mimic more human-like actions in the automated test, I adjusted the test behavior accordingly.

Key Changes:
Split Chain ID Input: The Chain ID input process has been split into two distinct parts (chainId_part1 and chainId_part2). This modification adopts a more human-like approach to entering the Chain ID, allowing the error message sufficient time to respond accurately.

Results:
Before the Fix: The flakiness could be reproduced locally on Firefox with a 50% occurrence rate.
After the Fix: After implementing the changes, I ran the test 20 consecutive times without encountering any flakiness, demonstrating the effectiveness of the fix.

This adjustment ensures that the automated test behaves in a way that closely resembles user interaction, and reduce test flakiness.

Open in GitHub Codespaces

Related issues

Fixes: #24628

Manual testing steps

  1. Run the test several times yarn test:e2e:single test/e2e/tests/request-queuing/ui.spec.js --browser=firefox --leave-running --retryUntilFailure --retries=10
  2. Check ci jobs

Screenshots/Recordings

Before

After

Pre-merge author checklist

  • I’ve followed MetaMask Coding Standards.
  • I've completed the PR template to the best of my ability
  • I’ve included tests if applicable
  • I’ve documented my code using JSDoc format if applicable
  • I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

@chloeYue chloeYue self-assigned this May 27, 2024
@chloeYue chloeYue requested a review from a team as a code owner May 27, 2024 14:52
@github-actions
Copy link
Copy Markdown
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@chloeYue chloeYue changed the title Fix flaky test Update Network: update network details and validate the ui elements test: Fix flaky test Update Network: update network details and validate the ui elements May 27, 2024
@metamaskbot
Copy link
Copy Markdown
Collaborator

Builds ready [327832c]
Page Load Metrics (909 ± 556 ms)
PlatformPageMetricMin (ms)Max (ms)Average (ms)StandardDeviation (ms)MarginOfError (ms)
ChromeHomefirstPaint6012485167
domContentLoaded9241232
load4828409091159556
domInteractive9241232
Bundle size diffs
  • background: 0 Bytes (0.00%)
  • ui: 0 Bytes (0.00%)
  • common: 0 Bytes (0.00%)

@MetaMask MetaMask deleted a comment from metamaskbot May 27, 2024
@MetaMask MetaMask deleted a comment from metamaskbot May 27, 2024
Copy link
Copy Markdown
Member

@seaona seaona left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@DDDDDanica DDDDDanica merged commit 7eb2f2b into develop May 27, 2024
@DDDDDanica DDDDDanica deleted the chloe-flaky-test-24628 branch May 27, 2024 16:32
@github-actions github-actions bot locked and limited conversation to collaborators May 27, 2024
@metamaskbot metamaskbot added release-11.16.6 Issue or pull request that will be included in release 11.16.6 and removed release-11.18.0 labels Jun 4, 2024
@metamaskbot
Copy link
Copy Markdown
Collaborator

Missing release label release-11.16.6 on PR. Adding release label release-11.16.6 on PR and removing other release labels(release-11.18.0), as PR was cherry-picked in branch 11.16.6.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

flaky tests release-11.16.6 Issue or pull request that will be included in release 11.16.6 team-extension-platform Extension Platform team

Projects

Archived in project

Development

Successfully merging this pull request may close these issues.

Flaky Test: "Update Network: update network details and validate the ui elements"

5 participants