test: add metametrics to collect qa stats#27696
Merged
jvbriones merged 4 commits intoMar 20, 2026
Merged
Conversation
Contributor
|
CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes. |
Contributor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.
dad1ab6 to
8bcf27b
Compare
b371f36 to
9f8c400
Compare
Contributor
🔍 Smart E2E Test Selection⏭️ Smart E2E selection skipped - draft PR All E2E tests pre-selected. |
jvbriones
approved these changes
Mar 20, 2026
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.




Description
This updates .github/scripts/collect-qa-stats.mjs so QA stats include E2E MetaMetrics coverage in its own top-level metametrics namespace (alongside unit, component_view, e2e, and performance).
The script statically scans event display names that tests assert: it walks tests/helpers/analytics/expectations/.ts and an explicit LEGACY_INLINE_METAMETRICS_PATHS list for specs/helpers that still use inline checks (getEventsPayloads, event.event ===, etc.). It resolves onboardingEvents. using tests/helpers/analytics/helpers.ts. Emitted keys are metametrics_events_checked_unique_count and metametrics_events_checked_names_json (sorted JSON array of names).
Changelog
CHANGELOG entry:
Related issues
Fixes:
Manual testing steps
Screenshots/Recordings
Before
After
Pre-merge author checklist
Pre-merge reviewer checklist
Note
Medium Risk
Changes the QA metrics pipeline by adding a new
metametricsnamespace and altering artifact sourcing (optional fixture mode), which could affect downstream dashboards/alerts if parsing or artifact naming assumptions break.Overview
Extends
.github/scripts/collect-qa-stats.mjsto emit a new top-levelmetametricsnamespace inqa-stats.json, populated by a static scan of E2E sources to count and list unique MetaMetrics event names asserted in tests.Adds an optional local fixture mode (
QA_STATS_FIXTURE_ROOT) that bypasses the GitHub API/token and reads artifacts from extracted directories, and tweaks E2E logging/output to better handle runs without JUnit artifacts and to print the JSON body after writing.Written by Cursor Bugbot for commit b371f3602bac194f6b4f55a3355bea7a9c1a6f01. This will update automatically on new commits. Configure here.