chore: test loading multi accounts#16732
Conversation
|
CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes. |
… between long network list
|
There was a problem hiding this comment.
Bug: Keyring Account Mismatch and ID Collision
The withImportedHdKeyringAndTwoDefaultAccountsOneImportedHdAccountOneQrAccountOneSimpleKeyPairAccount method has two issues:
- The first HD Key Tree keyring now contains only one account, contradicting the "TwoDefaultAccounts" implied by the method's name.
- Two distinct keyrings (QR Hardware Wallet Device and an HD Key Tree) within this method are assigned the same non-unique
metadata.id.
e2e/fixtures/fixture-builder.js#L1497-L1516
metamask-mobile/e2e/fixtures/fixture-builder.js
Lines 1497 to 1516 in eb0ba47
Bug: iOS Test Thresholds Set Incorrectly
All iOS performance test thresholds are incorrectly set to 5000ms, instead of the intended 7500ms. This discrepancy causes iOS tests to fail prematurely.
e2e/specs/performance/account-list/render-account-list.spec.ts#L31-L38
e2e/specs/performance/account-list/render-account-list.spec.ts#L124-L131
Was this report helpful? Give feedback by reacting with 👍 or 👎
|
|
There was a problem hiding this comment.
Bug: Network List Test Uses Incorrect Thresholds
The "heavy load" network list performance test uses incorrect thresholds. Both Android and iOS are currently set to 8000ms, but should be 17500ms for Android and 6500ms for iOS, aligning with the pattern established in the first performance test in the same file.
e2e/specs/performance/network-list/render-network-list.spec.ts#L130-L137
Bug: Test Passes Despite Performance Issues
The performance test logs "Performance test passed!" unconditionally, even when the measured time exceeds the defined threshold and a warning is logged. This creates misleading output where failed tests appear to pass.
e2e/specs/performance/network-list/render-network-list.spec.ts#L88-L95
Bug: Network List Test Timing Inconsistency
The "handle network list performance with heavy token load" test has an inaccurate and inconsistent timing measurement. The timer starts after tapping the networks button, excluding navigation time. Crucially, the endTime is recorded before the network list is checked for visibility, meaning the time taken for the list to actually render is not included. This makes the reported performance metrics incomplete and inconsistent with other tests.
e2e/specs/performance/network-list/render-network-list.spec.ts#L239-L242
e2e/specs/performance/network-list/render-network-list.spec.ts#L159-L166
Was this report helpful? Give feedback by reacting with 👍 or 👎
|
There was a problem hiding this comment.
Bug: Account List Dismissal Performance Threshold Mismatch
The performance threshold for dismissing the account list in the 'benchmark switching accounts from the account list' test is incorrectly set to 4000ms for Android. According to quality gates documentation, Android's threshold should be 5000ms, while iOS's 4000ms is correct. This inconsistency may cause incorrect test failures on Android.
e2e/specs/performance/account-list/dismiss-account-list.spec.ts#L54-L60
Was this report helpful? Give feedback by reacting with 👍 or 👎
|
There was a problem hiding this comment.
Bug: Performance Test Thresholds Mismatch PR Specs
Performance thresholds are inconsistent with the PR description across all account list performance tests. The code currently sets both Android and iOS thresholds to 5000ms for general rendering, heavy token load, and minimal load baseline tests. However, the PR description specifies different thresholds:
- General Rendering: iOS should be 7500ms (Android 5000ms).
- Heavy Load: Android and iOS should have different thresholds (not both 5000ms).
- Minimal Load Baseline: Android should be 45000ms, iOS should be 15000ms.
This discrepancy leads to incorrect performance expectations and affects test reliability across platforms.
e2e/specs/performance/account-list/render-account-list.spec.ts#L31-L38
e2e/specs/performance/account-list/render-account-list.spec.ts#L124-L131
e2e/specs/performance/account-list/render-account-list.spec.ts#L193-L200
Bug: iOS Tests Misconfigured on Linux Stack
The ios_e2e_test_performance workflow is incorrectly configured to use the linux-docker-android-22.04 stack. iOS tests require a macOS stack to run simulators, making the current configuration invalid and causing the workflow to fail.
bitrise.yml#L1840-L1843
Lines 1840 to 1843 in 627096c
bitrise.yml#L1988-L1991
Lines 1988 to 1991 in 627096c
Was this report helpful? Give feedback by reacting with 👍 or 👎
|
There was a problem hiding this comment.
Bug: Token Load Test Has Incorrect Thresholds
The heavy token load performance test uses incorrect thresholds. Both Android and iOS are currently set to 8000ms, but should be 17500ms for Android and 6500ms for iOS, consistent with the render network list efficiently test. This discrepancy makes the Android test too strict and the iOS test too lenient, leading to inaccurate performance results.
e2e/specs/performance/network-list/render-network-list.spec.ts#L130-L137
Was this report helpful? Give feedback by reacting with 👍 or 👎
|
There was a problem hiding this comment.
Bug: iOS Account List Load Times Incorrectly Set
The iOS performance thresholds for account list loading tests (general, heavy load, and baseline) are incorrectly set to 5000ms. They should be 7500ms, and distinct from the Android thresholds.
e2e/specs/performance/account-list/render-account-list.spec.ts#L35-L37
e2e/specs/performance/account-list/render-account-list.spec.ts#L128-L130
Was this report helpful? Give feedback by reacting with 👍 or 👎
|



Description
🎯 Performance Quality Gates
This document outlines the performance thresholds (quality gates) for MetaMask Mobile's critical user flows. These thresholds ensure optimal user experience across different platforms and scenarios.
📱 Platform Overview
📊 Test Reporting
Performance tests automatically generate detailed reports using the
PerformanceTestReporterutility:🏠 Account List Performance Tests
Test:
render account list efficiently with multiple accounts and networksConfiguration: Multiple accounts, popular networks, profile syncing enabled
Test:
handle account list performance with heavy token loadConfiguration: Multiple accounts, popular networks, 10 tokens for stress testing
Test:
benchmark account list with minimal loadConfiguration: Minimal accounts, default network, 2 tokens (baseline measurement)
Test:
benchmark switching accounts from the account listConfiguration: Account switching/dismissal performance
🌐 Network List Performance Tests
Test:
render network list efficiently with multiple accounts and all popular networksConfiguration: Multiple accounts, all popular networks
Test:
handle network list performance with heavy token load on all popular networksConfiguration: Multiple accounts, popular networks, 10 tokens for stress testing
Test:
benchmark network list with minimal loadConfiguration: Minimal tokens, popular networks (baseline measurement)
Test:
benchmark switching networks from the network listConfiguration: Network switching/dismissal performance
📊 Test Summary
Account List Tests (4 tests)
Network List Tests (4 tests)
Total: 8 performance tests across critical user flows
🚨 Quality Gate Rules
Failure Criteria
Tests fail immediately when total time exceeds the maximum acceptable time for any scenario.
Performance Patterns
User Profile Testing
Tests run across different user states with varying account complexity:
User Profile Definitions
Current Test Coverage
📈 Reporting Features
Automated Reports
Report Usage
Performance reports can be used for:
What can we do to improve it
Bitrise Runs
Table if 8 bitrise runs results
Related issues
Fixes:
Manual testing steps
Screenshots/Recordings
Before
After
Pre-merge author checklist
Pre-merge reviewer checklist