feat: add api spec test infrastructure by shanejonas · Pull Request #24132 · MetaMask/metamask-extension

shanejonas · 2024-04-19T15:11:02Z

Description

This PR adds tests using api-specs via the @open-rpc/test-coverage tool.

Check out the test report on CircleCI as an artifact.

Related issues

Fixes: https://github.com/MetaMask/MetaMask-planning/issues/2304

Manual testing steps

run yarn build:test
run yarn test:api-specs
see test report

Screenshots/Recordings

Link to video from PI lane demos

Pre-merge author checklist

I’ve followed MetaMask Coding Standards.
I've completed the PR template to the best of my ability
I’ve included tests if applicable
I’ve documented my code using JSDoc format if applicable
I’ve applied the right labels on the PR (see labeling guidelines). Not required for external contributors.

Pre-merge reviewer checklist

I've manually tested the PR (e.g. pull and build branch, run the app, test code being changed).
I confirm that this PR addresses all acceptance criteria described in the ticket it closes and includes the necessary testing evidence such as recordings and or screenshots.

github-actions · 2024-04-19T15:11:13Z

CLA Signature Action: All authors have signed the CLA. You may need to manually re-run the blocking PR check if it doesn't pass in a few minutes.

shanejonas · 2024-04-19T15:19:19Z

@metamaskbot update-policies

metamaskbot · 2024-04-19T15:20:32Z

Policy update failed. You can review the logs or retry the policy update here

shanejonas · 2024-04-19T16:03:26Z

@metamaskbot update-policies

metamaskbot · 2024-04-19T16:13:34Z

Policies updated

shanejonas · 2024-04-19T18:48:49Z

@metamaskbot update-policies

shanejonas · 2024-04-19T18:52:42Z

@metamaskbot update-policies

metamaskbot · 2024-04-19T18:56:56Z

Policies updated

shanejonas · 2024-04-19T18:57:24Z

@metamaskbot update-policies

metamaskbot · 2024-04-19T19:00:48Z

No policy changes

metamaskbot · 2024-04-19T19:05:27Z

No policy changes

Co-authored-by: jiexi <jiexiluan@gmail.com>

metamaskbot · 2024-06-18T21:15:14Z

Builds ready [c63d737]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (139 ± 179 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	66	97	84	8	4
		domContentLoaded	9	18	11	2	1
		load	43	1767	139	374	179
		domInteractive	9	18	11	2	1

Bundle size diffs [🚨 Warning! Bundle size has increased!]

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 437 Bytes (0.01%)

mcmire · 2024-06-21T20:51:44Z

@shanejonas I'm having trouble running yarn test:api-specs locally. Does Ganache run there? It doesn't seem to be able to connect to localhost.

metamaskbot · 2024-06-24T16:54:02Z

Builds ready [ad13905]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (42 ± 3 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	57	164	78	25	12
		domContentLoaded	7	19	9	2	1
		load	37	56	42	5	3
		domInteractive	7	19	9	2	1

Bundle size diffs [🚨 Warning! Bundle size has increased!]

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 437 Bytes (0.01%)

metamaskbot · 2024-06-24T21:51:37Z

Builds ready [1cdb3d7]

builds: chrome, firefox
builds (beta): chrome
builds (flask): chrome, firefox
builds (MMI): chrome, firefox
builds (test): chrome, firefox
builds (test-flask): chrome, firefox
build viz: Build System
mv3: Background Module Init Stats
mv3: UI Init Stats
mv3: Module Load Stats
mv3: Bundle Size Stats
mv2: E2e Actions Stats
code coverage: Report
storybook: Storybook
typescript migration: Dashboard
all artifacts
bundle viz:
- background: 0, 1, 2, 3, 4, 5
- common: 0, 1, 2, 3, 4, 5, 6, 7, 8, 9
- content-script: 0
- offscreen: 0
- ui: 0, 1, 10, 11, 2, 3, 4, 5, 6, 7, 8, 9

Page Load Metrics (58 ± 5 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	62	111	93	12	6
		domContentLoaded	10	23	13	3	2
		load	47	82	58	10	5
		domInteractive	10	23	13	3	2

Bundle size diffs [🚨 Warning! Bundle size has increased!]

background: 0 Bytes (0.00%)
ui: 0 Bytes (0.00%)
common: 437 Bytes (0.01%)

mcmire · 2024-06-24T23:45:40Z

@shanejonas I added:

        await new Promise((resolve) => {
          setTimeout(resolve, 4000);
        });

after server.start() and got:

[UnhandledPromiseRejection: This error originated either by throwing inside of an async function without a catch block, or by rejecting a promise which was not handled with .catch(). The promise rejected with the reason "Error: can't resolve reference #/components/schemas/BlockNumberOrTag from id #".] {
  code: 'ERR_UNHANDLED_REJECTION'
}

and then the test just ended without doing anything else.

Is it possible that something inside of @open-rpc/mock-server is not being awaited properly? Or that openrpcDocument is somehow invalid?

mcmire

The tests do succeed on CI, so that's good. Would be nice to figure out why it's failing locally but it could be my machine. Will approve in the meantime.

shanejonas requested review from a team as code owners April 19, 2024 15:11

metamaskbot added the INVALID-PR-TEMPLATE PR's body doesn't match template label Apr 19, 2024

shanejonas closed this Apr 19, 2024

shanejonas reopened this Apr 19, 2024

github-actions bot locked and limited conversation to collaborators Apr 19, 2024

shanejonas closed this Apr 19, 2024

shanejonas reopened this Apr 19, 2024

shanejonas closed this Apr 19, 2024

shanejonas reopened this Apr 19, 2024

shanejonas closed this Apr 19, 2024

shanejonas reopened this Apr 19, 2024

shanejonas force-pushed the sj/api-spec-tests branch from c7e7ac2 to 398b838 Compare April 19, 2024 17:43

shanejonas added the team-wallet-api-platform-deprecated DEPRECATED: please use "team-wallet-integrations" instead label Apr 19, 2024

shanejonas changed the title ~~Sj/api spec tests~~ feat: add api spec tests Apr 19, 2024

metamaskbot removed the INVALID-PR-TEMPLATE PR's body doesn't match template label Apr 19, 2024

shanejonas requested review from brad-decker and kumavis as code owners April 19, 2024 20:08

shanejonas and others added 16 commits June 18, 2024 16:06

fix: use ACCOUNT_1 over hardcoded address

a2a2e60

fix: rebase

c30af48

fix: linting

30ee78b

fix: dedupe

5136d56

fix: issue with commenting on PR when no report was generated

b22e7cd

fix: linting

342b5d1

fix: more linting

6064470

fix: lint order

ceb7352

fix: yarn version on CI

2a88900

fix: yarn lock updates after rebase

d360858

fix: add switchToOrOpenDapp to transport

26e5efd

fix: added taskQueue to deal with a race condition around window handles

359a69e

Update .circleci/config.yml

ba9911f

Co-authored-by: jiexi <jiexiluan@gmail.com>

fix: remove verbose reporter

5c03586

fix: yarn lock

43327f9

fix: removed unused vars

c63d737

shanejonas force-pushed the sj/api-spec-tests branch from 65730c9 to c63d737 Compare June 18, 2024 20:07

adonesky1 approved these changes Jun 19, 2024

View reviewed changes

jiexi approved these changes Jun 21, 2024

View reviewed changes

Merge branch 'develop' into sj/api-spec-tests

ad13905

Merge branch 'develop' into sj/api-spec-tests

1cdb3d7

mcmire approved these changes Jun 24, 2024

View reviewed changes

adonesky1 merged commit c9fb2ab into develop Jun 25, 2024

adonesky1 deleted the sj/api-spec-tests branch June 25, 2024 00:32

metamaskbot added the release-12.1.0 Issue or pull request that will be included in release 12.1.0 label Jun 25, 2024

Conversation

shanejonas commented Apr 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Related issues

Manual testing steps

Screenshots/Recordings

Pre-merge author checklist

Pre-merge reviewer checklist

Uh oh!

github-actions bot commented Apr 19, 2024

Uh oh!

shanejonas commented Apr 19, 2024

Uh oh!

metamaskbot commented Apr 19, 2024

Uh oh!

shanejonas commented Apr 19, 2024

Uh oh!

metamaskbot commented Apr 19, 2024

Uh oh!

shanejonas commented Apr 19, 2024

Uh oh!

shanejonas commented Apr 19, 2024

Uh oh!

metamaskbot commented Apr 19, 2024

Uh oh!

shanejonas commented Apr 19, 2024

Uh oh!

metamaskbot commented Apr 19, 2024

Uh oh!

metamaskbot commented Apr 19, 2024

Uh oh!

metamaskbot commented Jun 18, 2024

Uh oh!

mcmire commented Jun 21, 2024

Uh oh!

metamaskbot commented Jun 24, 2024

Uh oh!

metamaskbot commented Jun 24, 2024

Uh oh!

mcmire commented Jun 24, 2024

Uh oh!

mcmire left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

shanejonas commented Apr 19, 2024 •

edited

Loading