Build - refactor for bundle factoring and swappable runtime by kumavis · Pull Request #11080 · MetaMask/metamask-extension

kumavis · 2021-05-13T06:47:21Z

this refactor unblocks incremental lavamoat protections for the webapp
this is does not add lavamoat yet

some additional benefits:

automatically factors bundles into entry (phishing/ui/background) and "common" module groups
module groups are then reduced to subgroups based on a size limit, and then compose into bundles
bundles call into a swappable runtime api that allows the same module definition to be used in lavamoat or as in here a normal commonjs runtime. eg background could use lavamoat protections while ui does not, even as both consume the same "common" bundles

additional watchify and patch-package note:
in order to remove an incompatible watchify feature (binary output statistics), we used patch-package. it was determined to be the easiest and least hacky way to disable the non-optional feature.

… build-module-groups

development/build/scripts.js

development/stream-flat-map.js

development/build/scripts.js

kumavis

im happy with it now

metamaskbot · 2021-07-13T20:04:48Z

Builds ready [287097f]

builds: chrome, firefox, opera
bundle viz: background, ui, inpage, contentscript, ui-libs, bg-libs, phishing-detect
build viz: Build System
code coverage: Report
storybook: Storybook
all artifacts

Page Load Metrics (693 ± 40 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	54	91	69	10	5
		domContentLoaded	605	890	691	83	40
		load	606	891	693	83	40
		domInteractive	604	890	691	83	40

development/stream-flat-map.js

kumavis · 2021-07-14T20:56:08Z

browserify dedupe option?
set here

metamask-extension/development/build/scripts.js

Line 210 in 26f7673

Object.assign(bundlerOpts, bifyModuleGroups.plugin.args);

hmm i expected it to have an args in the exports -- digging
metamask ci bot links (eg bundle viz)
(@ryanml on it)

Co-authored-by: Brad Decker <git@braddecker.dev>

kumavis · 2021-07-14T20:56:55Z

desire for stats on total shipped code size

metamaskbot · 2021-07-14T21:14:10Z

Builds ready [26f7673]

builds: chrome, firefox, opera
bundle viz: background, ui, inpage, contentscript, ui-libs, bg-libs, phishing-detect
build viz: Build System
code coverage: Report
storybook: Storybook
all artifacts

Page Load Metrics (686 ± 36 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	54	88	67	9	4
		domContentLoaded	436	785	685	76	36
		load	437	787	686	76	36
		domInteractive	436	785	685	76	36

brad-decker

The changes we discussed are good to have but not required for this to land imo. Even the broken links in the build output bot notification are not an issue due to the all artifacts link working.

kumavis · 2021-07-15T00:05:20Z

is this the right pack

metamask-extension/development/build/scripts.js

Line 237 in 26f7673

lavaPack({ raw: true, hasExports: true, includePrelude: false }),

edit: yes, bc we're excluding the prelude

kumavis · 2021-07-15T00:18:20Z

verify bifyModuleGroups.groupByFactor

metamaskbot · 2021-07-15T01:38:12Z

Builds ready [626c8f3]

builds: chrome, firefox, opera
bundle viz: background-0, background-1, background-2, background-3, background-4, background-5, background-6, background-7, background-8, background-9, common-0, common-1, common-2, common-3, contentscript, disable-console, inpage, phishing-detect-0, sentry-install, ui-0, ui-1, ui-2, ui-3, ui-4, ui-5
build viz: Build System
code coverage: Report
storybook: Storybook
all artifacts

Page Load Metrics (922 ± 78 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	64	200	103	33	16
		domContentLoaded	528	1230	920	163	78
		load	530	1232	922	163	78
		domInteractive	527	1230	919	163	78

metamaskbot · 2021-07-15T01:38:40Z

Builds ready [626c8f3]

builds: chrome, firefox, opera
bundle viz: background-0, background-1, background-2, background-3, background-4, background-5, background-6, background-7, background-8, background-9, common-0, common-1, common-2, common-3, contentscript, disable-console, inpage, phishing-detect-0, sentry-install, ui-0, ui-1, ui-2, ui-3, ui-4, ui-5
build viz: Build System
code coverage: Report
storybook: Storybook
all artifacts

Page Load Metrics (922 ± 78 ms)

Platform	Page	Metric	Min (ms)	Max (ms)	Average (ms)	StandardDeviation (ms)	MarginOfError (ms)
Chrome	Home	firstPaint	64	200	103	33	16
		domContentLoaded	528	1230	920	163	78
		load	530	1232	922	163	78
		domInteractive	527	1230	919	163	78

ryanml · 2021-07-15T01:40:18Z

Bundle links have been fixed, this could be formatted a bit cleaner but that can be resolved in a follow up

kumavis · 2021-07-15T17:59:17Z

ok im ready

kumavis added 13 commits May 11, 2021 17:09

wip

922b8d3

build - breakout sentry-install bundle

5a3be45

deps - move new build sys deps to published versions

5bcda98

chore: lint fix

a426571

clean - remove unused file

d4cc61f

clean - remove unsused package script

a539646

lavamoat - update build system policy

39a0e4d

build - render html to all platforms

56ead09

development - improve sourcemap debugger output

952252a

deps - update lavapack

c7ec1cc

lint - fix

5053cee

deps - update lavapack for bugfix

c983a82

deps - update lavapack for bugfix

308ef8f

This comment has been minimized.

Sign in to view

kumavis added 2 commits May 20, 2021 13:30

deps - bump lavapack for line ending normalization

12b788a

sourcemap explorer - disable boundary validation

9411220

This comment has been minimized.

Sign in to view

Merge branch 'develop' of github.com:MetaMask/metamask-extension into…

7660029

… build-module-groups

This comment has been minimized.

Sign in to view

ci - reset normal ci flow

14649d4

kumavis marked this pull request as ready for review May 20, 2021 06:21

kumavis requested a review from a team as a code owner May 20, 2021 06:21

kumavis requested a review from adonesky1 May 20, 2021 06:21

This comment has been minimized.

Sign in to view

kumavis changed the title ~~Build - refactor for bundle runtime~~ Build - refactor for bundle factoring and swappable runtime May 20, 2021

ryanml reviewed May 21, 2021

View reviewed changes

development/build/scripts.js Outdated Show resolved Hide resolved

ryanml reviewed May 22, 2021

View reviewed changes

development/stream-flat-map.js Show resolved Hide resolved

build - re-enable minification on prod

51691d9

kumavis commented May 24, 2021

View reviewed changes

development/build/scripts.js Outdated Show resolved Hide resolved

kumavis mentioned this pull request May 24, 2021

LavaMoat - protect app process background #10838

Closed

7 tasks

kumavis added 2 commits July 13, 2021 12:41

ui - add comment about conditional import

14e6d5c

build - improve comment

287097f

kumavis commented Jul 13, 2021

View reviewed changes

brad-decker reviewed Jul 14, 2021

View reviewed changes

development/stream-flat-map.js Outdated Show resolved Hide resolved

Update development/stream-flat-map.js

26f7673

Co-authored-by: Brad Decker <git@braddecker.dev>

brad-decker previously approved these changes Jul 14, 2021

View reviewed changes

darkwing previously approved these changes Jul 14, 2021

View reviewed changes

ryanml dismissed stale reviews from darkwing and brad-decker via 71d3589 July 15, 2021 00:35

ryanml force-pushed the build-module-groups branch from 71d3589 to 6973dd3 Compare July 15, 2021 01:09

MetaMask deleted a comment from metamaskbot Jul 15, 2021

Outputting all bundle file links (metamaskbot)

626c8f3

ryanml force-pushed the build-module-groups branch from 6973dd3 to 626c8f3 Compare July 15, 2021 01:10

ryanml requested review from brad-decker and darkwing July 15, 2021 01:39

darkwing approved these changes Jul 15, 2021

View reviewed changes

brad-decker approved these changes Jul 15, 2021

View reviewed changes

kumavis merged commit 9f4820e into develop Jul 15, 2021

kumavis deleted the build-module-groups branch July 15, 2021 17:59

github-actions bot locked and limited conversation to collaborators Jul 15, 2021

Conversation

kumavis commented May 13, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Uh oh!

Uh oh!

Uh oh!

kumavis left a comment

Choose a reason for hiding this comment

Uh oh!

metamaskbot commented Jul 13, 2021

Uh oh!

Uh oh!

kumavis commented Jul 14, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kumavis commented Jul 14, 2021

Uh oh!

metamaskbot commented Jul 14, 2021

Uh oh!

brad-decker left a comment

Choose a reason for hiding this comment

Uh oh!

kumavis commented Jul 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kumavis commented Jul 15, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

metamaskbot commented Jul 15, 2021

Uh oh!

metamaskbot commented Jul 15, 2021

Uh oh!

ryanml commented Jul 15, 2021

Uh oh!

kumavis commented Jul 15, 2021

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

kumavis commented May 13, 2021 •

edited

Loading

kumavis commented Jul 14, 2021 •

edited

Loading

kumavis commented Jul 15, 2021 •

edited

Loading

kumavis commented Jul 15, 2021 •

edited

Loading