Skip to content

test: fix ff default value#31070

Merged
racitores merged 1 commit into
mainfrom
raci/fix-flaky-test-perps-abtesting
Jun 4, 2026
Merged

test: fix ff default value#31070
racitores merged 1 commit into
mainfrom
raci/fix-flaky-test-perps-abtesting

Conversation

@racitores

@racitores racitores commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

Description

This PR makes the E2E remote feature-flag mock deterministic for the homepage
trending-sections A/B test.

The flaky behavior came from setupRemoteFeatureFlagsMock(mockServer, {})
serving the production registry defaults. For
homeTMCU470AbtestTrendingSections, the production default is a percentage
rollout array, not a fixed control value. RemoteFeatureFlagController
resolves that rollout at runtime using a deterministic threshold based on:

sha256(analyticsId + flagName)

That means the same build and the same mocked feature-flag endpoint can still
render different homepage UI if the runtime analytics ID differs:

  • control renders the regular Perpetuals section.
  • trendingSections renders the separate Trending perpetuals section.

The fix pins homeTMCU470AbtestTrendingSections to control in the shared
E2E-safe mock defaults. Individual tests can still explicitly opt into
trendingSections by passing an override to createRemoteFeatureFlagsMock or
setupRemoteFeatureFlagsMock.

Changelog

CHANGELOG entry: null

Related issues

Fixes:

Manual testing steps

Feature: E2E remote feature flag defaults

  Scenario: homepage trending sections A/B test is deterministic in E2E mocks
    Given the E2E remote feature flag mock is created with default overrides
    When the mock response is generated
    Then homeTMCU470AbtestTrendingSections resolves to control

  Scenario: a test intentionally opts into the treatment
    Given the E2E remote feature flag mock is created with a trendingSections override
    When the mock response is generated
    Then homeTMCU470AbtestTrendingSections resolves to trendingSections

Verified with:

NODE_OPTIONS=--max-old-space-size=8192 yarn jest tests/api-mocking/helpers/remoteFeatureFlagsHelper.test.ts --runInBand --watchman=false --forceExit

Result: 1 test suite passed, 23 tests passed.

Screenshots/Recordings

Not applicable. Test-only change.

Before

E2E default feature-flag mocks could return the production rollout array for
homeTMCU470AbtestTrendingSections, allowing runtime analytics ID bucketing to
choose either homepage variant.

After

E2E default feature-flag mocks pin homeTMCU470AbtestTrendingSections to
control, so existing specs continue to see the Perpetuals section unless a
spec explicitly overrides the variant.

Pre-merge author checklist

Performance checks (if applicable)

  • I've tested on Android
    • Ideally on a mid-range device; emulator is acceptable
  • I've tested with a power user scenario
    • Use these power-user SRPs to import wallets with many accounts and tokens
  • I've instrumented key operations with Sentry traces for production performance metrics

For performance guidelines and tooling, see the Performance Guide.

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Note

Low Risk
Test-only changes to API-mocking defaults; no production app or auth logic.

Overview
E2E remote feature-flag mocks now pin homeTMCU470AbtestTrendingSections to control in E2E_SAFE_DEFAULTS, alongside existing safe defaults like mobileMinimumVersions. Production’s percentage rollout made homepage section labels depend on the generated analytics ID; pinning keeps API-mocked E2E runs deterministic.

Two unit tests assert the default mock includes control and that createRemoteFeatureFlagsMock({ homeTMCU470AbtestTrendingSections: 'trendingSections' }) still overrides the pin.

Reviewed by Cursor Bugbot for commit 50d4df1. Bugbot is set up for automated code reviews on this repo. Configure here.

@racitores racitores requested a review from a team as a code owner June 4, 2026 12:03
@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@github-actions

github-actions Bot commented Jun 4, 2026

Copy link
Copy Markdown
Contributor

🔍 Smart E2E Test Selection

  • Selected E2E tags: SmokeWalletPlatform
  • Selected Performance tags: None (no tests recommended)
  • Risk Level: low
  • AI Confidence: 85%
click to see 🤖 AI reasoning details

E2E Test Selection:
The changes are confined to E2E test infrastructure files:

  1. remoteFeatureFlagsHelper.ts: Adds homeTMCU470AbtestTrendingSections: 'control' to the E2E_SAFE_DEFAULTS object. This pins the homepage trending sections A/B test flag to the control variant for all E2E tests, preventing non-deterministic behavior caused by percentage-based rollout that depends on the analytics ID.

  2. remoteFeatureFlagsHelper.test.ts: Adds two unit tests verifying the new default is applied and can be overridden.

Impact Assessment:

  • This flag directly affects the Trending homepage sections (SmokeWalletPlatform tests: trending-feed.spec.ts, trending-search.spec.ts, trending-browser.spec.ts)
  • The remoteFeatureFlagsHelper.ts is imported by 20 test files across many test suites, but the change only adds a new default — it doesn't modify existing defaults or behavior
  • The change is stabilizing (fixes potential flakiness) rather than breaking
  • The control variant is what the trending tests expect, as confirmed by the test file structure

Why SmokeWalletPlatform only:

  • The flag homeTMCU470AbtestTrendingSections is specifically about homepage trending sections
  • The trending spec files (trending-feed.spec.ts, trending-search.spec.ts, trending-browser.spec.ts) all use the SmokeWalletPlatform tag
  • Other test suites that import remoteFeatureFlagsHelper.ts are not affected by this specific flag addition
  • No app code was changed, only test infrastructure

No performance tests needed: This is a pure test infrastructure change with no impact on app rendering, data loading, or runtime performance.

Performance Test Selection:
This is a pure test infrastructure change (E2E mock helper and its unit tests). No app code was modified, so there is no performance impact on rendering, data loading, or any runtime behavior.

View GitHub Actions results

@github-actions github-actions Bot added the risk:high AI analysis: high risk label Jun 4, 2026
@racitores racitores enabled auto-merge June 4, 2026 13:14
@racitores racitores added this pull request to the merge queue Jun 4, 2026
Merged via the queue into main with commit d56f284 Jun 4, 2026
209 of 215 checks passed
@racitores racitores deleted the raci/fix-flaky-test-perps-abtesting branch June 4, 2026 13:40
@github-actions github-actions Bot locked and limited conversation to collaborators Jun 4, 2026
@metamaskbotv2 metamaskbotv2 Bot added the release-7.81.0 Issue or pull request that will be included in release 7.81.0 label Jun 4, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

release-7.81.0 Issue or pull request that will be included in release 7.81.0 risk:high AI analysis: high risk size-S team-qa QA team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants