Skip to content

ci: add automated E2E readiness label and CI checks - Part1#29305

Merged
jvbriones merged 40 commits into
mainfrom
mmqa-1738
Apr 29, 2026
Merged

ci: add automated E2E readiness label and CI checks - Part1#29305
jvbriones merged 40 commits into
mainfrom
mmqa-1738

Conversation

@jvbriones

@jvbriones jvbriones commented Apr 24, 2026

Copy link
Copy Markdown
Contributor

Description

Introduces an E2E readiness gate via the new pr-not-ready-for-e2e label which is auto-applied on PR open, causes E2E builds/tests to be skipped, and block merge until the label is removed (and E2E tests run and pass)

Also CI control flow for E2E tests is refactored:

flowchart TD
    CI[CI run starts - Android and iOS] --> GR[Check conditions]
    GR -->|Merge Queue | MQ[No E2E]
    GR -->|Fork PR| FK[No E2E]
    GR -->|PR label: skip-e2e| HS[No E2E]
    GR -->|PR label: pr-not-ready-for-e2e| L2[No E2E]
    L2 -->|ignorable-only changes| NoBlock[No merge block]
    L2 -->|non-ignorable changes| Skip2[Merge blocked]
    GR -->|PR ignorable-only changes| Ignorable[No E2E - merge not blocked]
    GR -->|PR has Android-only changes| Android[Android Build + Tests needed]
    GR -->|PR has iOS-only changes| iOS[iOS Build + Test needed]
    GR -->|PR other files changed| Both[Both Build + Tests needed]
    GR -->|Scheduled and Push to main | Full[Run all E2E Suites for Both]

    Android & iOS & Both --> LABEL{{PR label: skip-smart-e2e-selection ?}}
    LABEL -->|yes| AllTags[Run all E2E needed]
    LABEL -->|no| AI[🤖 AI selects test suites + confidence score]
    AI --> CONF{{Confidence >= 80% ?}}
    CONF -->|yes| SelectedTags[Run selected E2E suites]
    CONF -->|no| AllTagsFallback[Run all E2E needed]
Loading

NOTE: It will be activated in a follow up PR so we can test the whole approach in dedicated branches

Changelog

CHANGELOG entry:

Related issues

Fixes: https://consensyssoftware.atlassian.net/browse/MMQA-1738

Manual testing steps

  • Test labels to skip E2E tests ✅
  • Check CI graph in GH UI ✅
  • Test node 20 warnings related with checkout v4 ✅
  • Test auto-label to skip E2E ⏱️-> not activated in this PR. To be validated next.

Screenshots/Recordings

Before

After

Pre-merge author checklist

Performance checks (if applicable)

  • I've tested on Android
    • Ideally on a mid-range device; emulator is acceptable
  • I've tested with a power user scenario
    • Use these power-user SRPs to import wallets with many accounts and tokens
  • I've instrumented key operations with Sentry traces for production performance metrics

For performance guidelines and tooling, see the Performance Guide.

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Note

Medium Risk
Touches core CI/E2E gating and merge-blocking logic, so misconfiguration could skip required E2E runs or incorrectly block merges. Changes are mostly workflow refactors, but they affect critical automation paths.

Overview
Adds an E2E readiness gate driven by the new pr-not-ready-for-e2e label: a new workflow auto-applies the label on PR open (temporarily scoped to test-e2e-readiness-label-* branches), CI skips E2E while it’s present, and ci.yml fails the final gate to block merge unless the PR is ignorable-only.

Refactors CI control flow by replacing needs-e2e-build.yml and e2e-check-build-needed.mjs with a new reusable get-requirements.yml that computes skip/build/run flags (labels, [skip-e2e], forks, merge queue, and path filtering via new .github/rules/filter-rules.yml) and feeds smart-e2e-selection + platform E2E build/test jobs, including passing changed_files for flakiness detection.

Simplifies smart-e2e-selection to only respect skip-smart-e2e-selection (removing release-branch/draft/event gating and force_run output), updates comment formatting, and bumps multiple workflows to actions/checkout@v6 and actions/setup-node@v6. Documentation is updated with a new E2E_DECISION_TREE.md and expanded labeling guidelines; CODEOWNERS is adjusted accordingly.

Reviewed by Cursor Bugbot for commit 93b4427. Bugbot is set up for automated code reviews on this repo. Configure here.

@jvbriones jvbriones requested review from a team as code owners April 24, 2026 06:49
@github-actions

Copy link
Copy Markdown
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4ce382630c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .github/workflows/ci.yml Outdated
@jvbriones jvbriones changed the title Mmqa 1738 ci: add automated E2E readiness label and CI checks Apr 24, 2026
@jvbriones jvbriones added no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed skip-smart-e2e-selection Skip Smart E2E selection, i.e. select all E2E tests to run labels Apr 24, 2026
Comment thread .github/rules/filter-rules.yml Outdated
@jvbriones jvbriones added the team-qa QA team label Apr 24, 2026
Comment thread .github/workflows/ci.yml Outdated
Comment thread .github/workflows/get-requirements.yml
Comment thread .github/scripts/e2e-check-build-needed.mjs Outdated
Comment thread .github/workflows/needs-e2e-build.yml Outdated
Comment thread .github/rules/filter-rules.yml
@jvbriones jvbriones changed the title ci: add automated E2E readiness label and CI checks ci: add automated E2E readiness label and CI checks - Part1 Apr 29, 2026
Comment thread .github/workflows/ci.yml Outdated
racitores
racitores previously approved these changes Apr 29, 2026
@jvbriones jvbriones enabled auto-merge April 29, 2026 14:18

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 3fca3b7. Configure here.

Comment thread .github/workflows/get-requirements.yml
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Smart E2E Test Selection

  • Selected E2E tags: SmokeAccounts, SmokeConfirmations, SmokeNetworkAbstractions, SmokeWalletPlatform
  • Selected Performance tags: None (no tests recommended)
  • Risk Level: medium
  • AI Confidence: 78%
click to see 🤖 AI reasoning details

E2E Test Selection:
This PR is a pure CI/workflow infrastructure refactoring with no app code changes. The key changes are:

  1. New get-requirements.yml replaces needs-e2e-build.yml - centralizes E2E build decision logic with improved label handling (pr-not-ready-for-e2e), commit tag detection ([skip-e2e]), and file filtering
  2. New filter-rules.yml - centralized path filter rules
  3. New auto-label-not-ready-for-e2e.yml - auto-labeling workflow (scoped to test branches only)
  4. Deleted needs-e2e-build.yml and e2e-check-build-needed.mjs - replaced by new workflow
  5. Major ci.yml refactoring - replaces check-skip-merge-queue + needs_e2e_build with single get_requirements job, removes force_run output, adds merge-blocking for pr-not-ready-for-e2e label
  6. Updated smart-e2e-selection/action.yml - simplified, removed event-name, is-draft, force_run inputs/outputs
  7. Action version bumps across multiple workflows (v3/v4 → v6)
  8. Documentation updates - AGENTS.md, CODEOWNERS, guidelines

Since no app code changed, there is no risk to app functionality. However, these changes directly affect the E2E pipeline infrastructure - specifically how tests are triggered, which builds happen, and when E2E is skipped. The filter-rules.yml itself marks these CI workflow files as e2e_relevant_workflows, indicating the system recognizes they should trigger E2E.

Running a representative subset of E2E tests is appropriate to validate:

  • The new get-requirements.yml correctly determines E2E needs
  • The pipeline still correctly triggers builds and test runs
  • The smart-e2e-selection action still works with the simplified inputs
  • The overall CI flow functions end-to-end

I'm selecting a moderate set of smoke tests (SmokeAccounts, SmokeConfirmations, SmokeNetworkAbstractions, SmokeWalletPlatform) to validate the pipeline works correctly without running the full suite, since the app code itself is unchanged.

Performance Test Selection:
No app code changes were made in this PR. All changes are CI/workflow infrastructure (GitHub Actions workflows, scripts, filter rules, documentation). Performance tests measure app responsiveness and rendering, which are completely unaffected by CI pipeline changes.

View GitHub Actions results

@sonarqubecloud

Copy link
Copy Markdown

@github-actions

Copy link
Copy Markdown
Contributor

E2E Fixture Validation — Schema is up to date
12 value mismatches detected (expected — fixture represents an existing user).
View details

@Cal-L Cal-L left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@jvbriones jvbriones added this pull request to the merge queue Apr 29, 2026
Merged via the queue into main with commit 7b749d4 Apr 29, 2026
198 of 200 checks passed
@jvbriones jvbriones deleted the mmqa-1738 branch April 29, 2026 18:39
@github-actions github-actions Bot locked and limited conversation to collaborators Apr 29, 2026
@metamaskbotv2 metamaskbotv2 Bot added the release-7.76.0 Issue or pull request that will be included in release 7.76.0 label Apr 29, 2026
uses: actions/checkout@v6
with:
fetch-depth: 1
ref: ${{ env.HEAD_COMMIT_HASH }}

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're running these checks on the wrong commit.

pull_request triggered workflows run on the head of the merge branch (which is what github.sha points at, which is the default for actions/checkout). This merge branch includes the latest changes from the base branch. This is very different from the head commit hash, which never has the changes from the base branch merged in.

This is currently breaking all open PRs that don't yet include this commit, and it'll break again each time this workflow or the filters change. We need to be consistent about which commit we're using.

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed release-7.76.0 Issue or pull request that will be included in release 7.76.0 size-L team-qa QA team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants