Skip to content

test: add metametrics to collect qa stats#27696

Merged
jvbriones merged 4 commits into
mainfrom
raci/add-e2e-test-metametrics-coverage-for-observability
Mar 20, 2026
Merged

test: add metametrics to collect qa stats#27696
jvbriones merged 4 commits into
mainfrom
raci/add-e2e-test-metametrics-coverage-for-observability

Conversation

@racitores

@racitores racitores commented Mar 19, 2026

Copy link
Copy Markdown
Contributor

Description

This updates .github/scripts/collect-qa-stats.mjs so QA stats include E2E MetaMetrics coverage in its own top-level metametrics namespace (alongside unit, component_view, e2e, and performance).

The script statically scans event display names that tests assert: it walks tests/helpers/analytics/expectations/.ts and an explicit LEGACY_INLINE_METAMETRICS_PATHS list for specs/helpers that still use inline checks (getEventsPayloads, event.event ===, etc.). It resolves onboardingEvents. using tests/helpers/analytics/helpers.ts. Emitted keys are metametrics_events_checked_unique_count and metametrics_events_checked_names_json (sorted JSON array of names).

Changelog

CHANGELOG entry:

Related issues

Fixes:

Manual testing steps

Feature: my feature name

  Scenario: user [verb for user action]
    Given [describe expected initial app state]

    When user [verb for user action]
    Then [describe expected outcome]

Screenshots/Recordings

Before

After

Pre-merge author checklist

Pre-merge reviewer checklist

  • I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
  • I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

Note

Medium Risk
Changes the QA metrics pipeline by adding a new metametrics namespace and altering artifact sourcing (optional fixture mode), which could affect downstream dashboards/alerts if parsing or artifact naming assumptions break.

Overview
Extends .github/scripts/collect-qa-stats.mjs to emit a new top-level metametrics namespace in qa-stats.json, populated by a static scan of E2E sources to count and list unique MetaMetrics event names asserted in tests.

Adds an optional local fixture mode (QA_STATS_FIXTURE_ROOT) that bypasses the GitHub API/token and reads artifacts from extracted directories, and tweaks E2E logging/output to better handle runs without JUnit artifacts and to print the JSON body after writing.

Written by Cursor Bugbot for commit b371f3602bac194f6b4f55a3355bea7a9c1a6f01. This will update automatically on new commits. Configure here.

@racitores racitores requested a review from a team as a code owner March 19, 2026 14:49
@racitores racitores added no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed skip-e2e skip E2E test jobs labels Mar 19, 2026
@github-actions

Copy link
Copy Markdown
Contributor

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

@metamaskbot metamaskbot added the team-qa QA team label Mar 19, 2026
Comment thread .github/scripts/collect-qa-stats.mjs
@github-actions github-actions Bot added risk-low Low testing needed · Low bug introduction risk and removed risk-low Low testing needed · Low bug introduction risk labels Mar 19, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Comment thread .github/scripts/collect-qa-stats.mjs Outdated
@github-actions github-actions Bot added risk-low Low testing needed · Low bug introduction risk and removed risk-low Low testing needed · Low bug introduction risk labels Mar 19, 2026
@racitores racitores force-pushed the raci/add-e2e-test-metametrics-coverage-for-observability branch from dad1ab6 to 8bcf27b Compare March 19, 2026 15:15
@github-actions github-actions Bot added risk-low Low testing needed · Low bug introduction risk and removed risk-low Low testing needed · Low bug introduction risk labels Mar 19, 2026
@github-actions github-actions Bot added risk-low Low testing needed · Low bug introduction risk and removed risk-low Low testing needed · Low bug introduction risk labels Mar 19, 2026
@jvbriones jvbriones marked this pull request as draft March 20, 2026 08:29
@racitores racitores force-pushed the raci/add-e2e-test-metametrics-coverage-for-observability branch from b371f36 to 9f8c400 Compare March 20, 2026 16:02
@github-actions

Copy link
Copy Markdown
Contributor

🔍 Smart E2E Test Selection

⏭️ Smart E2E selection skipped - draft PR

All E2E tests pre-selected.

View GitHub Actions results

@racitores racitores marked this pull request as ready for review March 20, 2026 16:05
@jvbriones jvbriones enabled auto-merge March 20, 2026 16:12

@MarioAslau MarioAslau left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@sonarqubecloud

Copy link
Copy Markdown

@jvbriones jvbriones added this pull request to the merge queue Mar 20, 2026
Merged via the queue into main with commit 5ff018e Mar 20, 2026
127 checks passed
@jvbriones jvbriones deleted the raci/add-e2e-test-metametrics-coverage-for-observability branch March 20, 2026 16:50
@github-actions github-actions Bot locked and limited conversation to collaborators Mar 20, 2026
@metamaskbot metamaskbot added the release-7.72.0 Issue or pull request that will be included in release 7.72.0 label Mar 20, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

no-changelog no-changelog Indicates no external facing user changes, therefore no changelog documentation needed release-7.72.0 Issue or pull request that will be included in release 7.72.0 risk-low Low testing needed · Low bug introduction risk size-M skip-e2e skip E2E test jobs team-qa QA team

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants