chore: exclude running git diff job for the e2e quality gate in `develop`, `master` and release branches by seaona · Pull Request #25605 · MetaMask/metamask-extension

seaona · 2024-07-01T08:55:49Z

Description

This PR excludes running the git-diff script for the e2e quality gate added here, for develop as well as for master and release branches.
We don't want to run this in develop, master or release branch, since there is no point on re-running the new/changed tests there, as the changes are already in develop branch. This could add up more credits and also slow down ci in those branches.

Context: in develop branch the current quality gate fails, since the diffResult is empty, and we were throwing the following error as we were entering in the !diffResult condition.
However, this PR fixes the issue on the higher level, by skipping entirely the git diff for the mentioned branches above.

  await fetchUntilMergeBaseFound();
  const { stdout: diffResult } = await exec(`git diff --name-only origin/HEAD...${process.env.CIRCLE_BRANCH}`);
  if (!diffResult) {
      throw new Error('Unable to get diff after full checkout.');
  }

Related issues

Fixes: current develop ci

Manual testing steps

ci should continue to work
once merged, fixes develop

Screenshots/Recordings

Before

After

Pre-merge author checklist

I've followed MetaMask Contributor Docs and MetaMask Extension Coding Standards.
I've completed the PR template to the best of my ability
I’ve included tests if applicable
I’ve documented my code using JSDoc format if applicable
I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

github-actions · 2024-07-01T08:56:02Z

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

metamaskbot · 2024-07-01T16:20:57Z

Builds ready [3783ebf]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (250 ± 245 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	69	268	114	40	19
		domContentLoaded	10	60	29	14	7
		load	43	1862	250	511	245
		domInteractive	10	60	29	14	7

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

codecov · 2024-07-01T16:21:46Z

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 69.60%. Comparing base (fe12ae4) to head (7d28ce5).
Report is 5 commits behind head on develop.

Additional details and impacted files

@@           Coverage Diff            @@
##           develop   #25605   +/-   ##
========================================
  Coverage    69.60%   69.60%           
========================================
  Files         1364     1364           
  Lines        48172    48172           
  Branches     13291    13291           
========================================
  Hits         33526    33526           
  Misses       14646    14646

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

seaona · 2024-07-02T07:15:39Z

test/e2e/changedFilesUtil.js

  } catch (error) {
    console.error('Error reading from file:', error);
-    return '';
+    return [];


returning an empty array, in order to handle properly the subsequent filter, when there is no file (in develop, master and RC branch)

seaona · 2024-07-02T07:22:43Z

.circleci/config.yml

                - /^Version-v(\d+)[.](\d+)[.](\d+)/
          requires:
            - prep-build
-            - get-changed-files-with-git-diff


this is not needed here, since the vault decrypt is a special test, that only runs on develop and RC branches, and only runs this single test (doesn't use the run-all script)

danjm · 2024-07-02T08:14:51Z

.circleci/config.yml

                - /^Version-v(\d+)[.](\d+)[.](\d+)/
          requires:
            - prep-build
-            - get-changed-files-with-git-diff


We can also remove get-changed-files-with-git-diff as a requirement for the test-e2e-firefox-flask and test-e2e-firefox-confirmation-redesign jobs, which also only run on develop/master/rc

that's a great point! Removed too 👍

danjm

One more suggested change from me

danjm · 2024-07-02T08:35:08Z

.circleci/config.yml

          at: .
      - run:
          name: Get changed files with git diff
-          command: npx tsx .circleci/scripts/git-diff-develop.ts


For simpler and easier to maintain code, I think that instead of this code change to the command, we can do the following change above:

diff --git a/.circleci/config.yml b/.circleci/config.yml index d9f0754335..9fe95a7f6b 100644 --- a/.circleci/config.yml +++ b/.circleci/config.yml @@ -106,6 +106,12 @@ workflows: - check-pr-tag - prep-deps - get-changed-files-with-git-diff: + filters: + branches: + ignore: + - develop + - master + - /^Version-v(\d+)[.](\d+)[.](\d+)/ requires: - prep-deps - test-deps-audit:

And the other jobs that depend on get-changed-files-with-git-diff should still run, see the docs on that points: "Note: When jobs in the current workflow that are listed as dependencies are not executed (due to a filter function for example), their requirement as a dependency for other jobs will be ignored by the requires option. However, if all dependencies of a job are filtered, then that job will not be executed either." https://circleci.com/docs/configuration-reference/#requires

wooa that's what I was looking for! I fully agree with this approach. I initially had it like this, but couldn't find anywhere if the jobs requiring the skipped job would succeed. Pushing the changes now. Thank you!!

sonarqubecloud · 2024-07-02T08:43:58Z

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarCloud

metamaskbot · 2024-07-02T09:23:18Z

Builds ready [7d28ce5]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (166 ± 205 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	75	300	114	62	30
		domContentLoaded	10	89	25	16	8
		load	40	2022	166	427	205
		domInteractive	9	88	25	16	8

Bundle size diffs

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 0 Bytes (0.00%)

ignore quality gate for develop, master and rc branches

59ce1be

seaona self-assigned this Jul 1, 2024

seaona added the team-extension-platform Extension Platform team label Jul 1, 2024

metamaskbot added the team-confirmations Push issues to confirmations team label Jul 1, 2024

add requires

afa75f9

seaona changed the title ~~chore: ignore quality gate for develop, master and RC branches~~ chore: fix quality gate for develop branch Jul 1, 2024

seaona added 3 commits July 1, 2024 11:25

fix develop git diff

37a05ec

add check for ci branch

cdf6025

remove error condition with git diff and just return the output

8aac280

seaona removed the team-confirmations Push issues to confirmations team label Jul 1, 2024

seaona marked this pull request as ready for review July 1, 2024 11:43

seaona requested review from a team and kumavis as code owners July 1, 2024 11:43

seaona marked this pull request as draft July 1, 2024 12:30

seaona and others added 3 commits July 1, 2024 14:57

do not run git diff in develop, master and release branches

9e19f5b

exclude develop master and rc for git diff

cfc8f72

Merge branch 'develop' into chore-quality-gate-ignore-branches

0ada3e8

seaona changed the title ~~chore: fix quality gate for develop branch~~ chore: exclude running git diff job for the e2e quality gate in develop, master and release branches Jul 1, 2024

seaona and others added 2 commits July 1, 2024 16:39

Merge branch 'develop' into chore-quality-gate-ignore-branches

82ef0f0

return empty array for empty file

3783ebf

clean up

726f86b

seaona commented Jul 2, 2024

View reviewed changes

fix requires job

b3f4510

seaona marked this pull request as ready for review July 2, 2024 07:42

seaona requested a review from a team as a code owner July 2, 2024 07:42

danjm reviewed Jul 2, 2024

View reviewed changes

do not run in test-e2e-firefox-confirmation-redesign and firefox-flask

e697d96

danjm suggested changes Jul 2, 2024

View reviewed changes

apply filter on workflow level

7d28ce5

danjm approved these changes Jul 2, 2024

View reviewed changes

chloeYue approved these changes Jul 2, 2024

View reviewed changes

seaona merged commit 4886a3f into develop Jul 2, 2024

seaona deleted the chore-quality-gate-ignore-branches branch July 2, 2024 12:18

github-actions bot locked and limited conversation to collaborators Jul 2, 2024

metamaskbot added the release-12.2.0 Issue or pull request that will be included in release 12.2.0 label Jul 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

chore: exclude running git diff job for the e2e quality gate in `develop`, `master` and release branches#25605

chore: exclude running git diff job for the e2e quality gate in `develop`, `master` and release branches#25605
seaona merged 14 commits intodevelopfrom
chore-quality-gate-ignore-branches

seaona commented Jul 1, 2024 •

edited

Loading

Uh oh!

github-actions bot commented Jul 1, 2024

Uh oh!

metamaskbot commented Jul 1, 2024

Uh oh!

codecov bot commented Jul 1, 2024 •

edited

Loading

Uh oh!

seaona Jul 2, 2024

Uh oh!

seaona Jul 2, 2024 •

edited

Loading

Uh oh!

danjm Jul 2, 2024

Uh oh!

seaona Jul 2, 2024

Uh oh!

danjm left a comment

Uh oh!

danjm Jul 2, 2024

Uh oh!

seaona Jul 2, 2024

Uh oh!

sonarqubecloud bot commented Jul 2, 2024

Uh oh!

metamaskbot commented Jul 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

seaona commented Jul 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related issues

Manual testing steps

Screenshots/Recordings

Before

After

Pre-merge author checklist

Pre-merge reviewer checklist

Uh oh!

github-actions bot commented Jul 1, 2024

Uh oh!

metamaskbot commented Jul 1, 2024

Uh oh!

codecov bot commented Jul 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

seaona Jul 2, 2024

Choose a reason for hiding this comment

Uh oh!

seaona Jul 2, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

danjm Jul 2, 2024

Choose a reason for hiding this comment

Uh oh!

seaona Jul 2, 2024

Choose a reason for hiding this comment

Uh oh!

danjm left a comment

Choose a reason for hiding this comment

Uh oh!

danjm Jul 2, 2024

Choose a reason for hiding this comment

Uh oh!

seaona Jul 2, 2024

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented Jul 2, 2024

Quality Gate passed

Uh oh!

metamaskbot commented Jul 2, 2024

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

seaona commented Jul 1, 2024 •

edited

Loading

codecov bot commented Jul 1, 2024 •

edited

Loading

seaona Jul 2, 2024 •

edited

Loading