stress-test: add cli application by spmonahan · Pull Request #24551 · microsoft/fluentui

spmonahan · 2022-08-26T22:07:31Z

Current Behavior

Running Stress Test requires lots of npm scripts which are difficult to maintain and understand.

New Behavior

Stress Test is run by a CLI application that eliminates the need for most npm scripts (the remaining ones are focused on development).

Testing

Pull this branch
Run yarn
Run yarn workspace @fluentui/stress-test build:local
Run yarn workspace @fluentui/stress-test stress-test run --scenario simple-stress --sample-size 2 --test-cases mount --browsers firefox --sizes xs

Related Issue(s)

Fixes #24309

Adds a CLI application for Stress Test that allows test configurations to easily be generated and run. This makes running large test suites as simple as a single command and eliminates the need for most npm scripts that previously existed for this package.

codesandbox-ci · 2022-08-26T22:17:57Z

This pull request is automatically built and testable in CodeSandbox.

To see build info of the built libraries, click here or the icon next to each commit SHA.

Latest deployment of this branch, based on commit c2d3b20:

Sandbox	Source
@fluentui/react 8 starter	Configuration
@fluentui/react-components 9 starter	Configuration

fabricteam · 2022-08-26T22:19:38Z

📊 Bundle size report

Package & Exports	Baseline (minified/GZIP)	PR	Change
`react-alert` Alert	`83.797 kB` `20.853 kB`	`83.852 kB` `20.905 kB`	`55 B` `52 B`
`react-button` Button	`36.405 kB` `9.588 kB`	`36.46 kB` `9.602 kB`	`55 B` `14 B`
`react-button` CompoundButton	`43.431 kB` `10.804 kB`	`43.486 kB` `10.823 kB`	`55 B` `19 B`
`react-button` MenuButton	`39.023 kB` `10.467 kB`	`39.078 kB` `10.482 kB`	`55 B` `15 B`
`react-button` SplitButton	`46.553 kB` `11.85 kB`	`46.608 kB` `11.866 kB`	`55 B` `16 B`
`react-button` ToggleButton	`51.919 kB` `11.011 kB`	`51.974 kB` `11.025 kB`	`55 B` `14 B`
`react-components` react-components: Accordion, Button, FluentProvider, Image, Menu, Popover	`189.317 kB` `52.005 kB`	`189.372 kB` `52.027 kB`	`55 B` `22 B`

Unchanged fixtures

Package & Exports	Size (minified/GZIP)
`global-context` createContext	`533 B` `341 B`
`global-context` createContextSelector	`554 B` `348 B`
`priority-overflow` createOverflowManager	`2.936 kB` `1.212 kB`
`react-accordion` Accordion (including children components)	`79.418 kB` `24.09 kB`
`react-avatar` Avatar	`48.339 kB` `13.676 kB`
`react-avatar` AvatarGroup	`14.908 kB` `5.971 kB`
`react-avatar` AvatarGroupItem	`68.307 kB` `19.019 kB`
`react-badge` Badge	`22.558 kB` `7.187 kB`
`react-badge` CounterBadge	`23.461 kB` `7.478 kB`
`react-badge` PresenceBadge	`24.006 kB` `7.049 kB`
`react-card` Card - All	`67.516 kB` `19.291 kB`
`react-card` Card	`63.198 kB` `18.202 kB`
`react-card` CardFooter	`8.519 kB` `3.582 kB`
`react-card` CardHeader	`9.562 kB` `3.922 kB`
`react-card` CardPreview	`8.62 kB` `3.639 kB`
`react-combobox` Combobox (including child components)	`73.821 kB` `24.003 kB`
`react-combobox` Dropdown (including child components)	`73.408 kB` `23.911 kB`
`react-components` react-components: FluentProvider & webLightTheme	`33.317 kB` `10.983 kB`
`react-dialog` Dialog (including children components)	`81.485 kB` `24.209 kB`
`react-divider` Divider	`16.417 kB` `5.883 kB`
`react-image` Image	`10.738 kB` `4.245 kB`
`react-input` Input	`23.556 kB` `7.642 kB`
`react-label` Label	`9.296 kB` `3.843 kB`
`react-link` Link	`12.289 kB` `4.953 kB`
`react-menu` Menu (including children components)	`116.237 kB` `35.452 kB`
`react-menu` Menu (including selectable components)	`119.436 kB` `35.934 kB`
`react-overflow` hooks only	`10.685 kB` `4.104 kB`
`react-popover` Popover	`102.894 kB` `31.525 kB`
`react-portal` Portal	`10.576 kB` `3.875 kB`
`react-positioning` usePositioning	`19.7 kB` `7.404 kB`
`react-provider` FluentProvider	`15.713 kB` `5.864 kB`
`react-radio` Radio	`36.083 kB` `11.944 kB`
`react-radio` RadioGroup	`14.206 kB` `5.683 kB`
`react-select` Select	`20.804 kB` `7.327 kB`
`react-slider` Slider	`32.128 kB` `10.06 kB`
`react-spinbutton` SpinButton	`43.901 kB` `12.364 kB`
`react-spinner` Spinner	`19.935 kB` `6.42 kB`
`react-switch` Switch	`32.62 kB` `10.282 kB`
`react-text` Text - Default	`11.74 kB` `4.587 kB`
`react-text` Text - Wrappers	`15.05 kB` `5.025 kB`
`react-textarea` Textarea	`23.732 kB` `7.86 kB`
`react-theme` Single theme token import	`69 B` `89 B`
`react-theme` Teams: all themes	`29.548 kB` `6.434 kB`
`react-theme` Teams: Light theme	`17.452 kB` `5.054 kB`
`react-tooltip` Tooltip	`41.502 kB` `14.623 kB`
`react-utilities` SSRProvider	`180 B` `159 B`

_{🤖 This report was generated against 22de26fac5b939224884052b0bf86cecc5f22fee}

size-auditor · 2022-08-26T22:21:30Z

Asset size changes

⚠️ Insufficient baseline data to detect size changes

Unable to find bundle size details for Baseline commit: 28f5559

Possible causes

The baseline build 28f5559 is broken
The Size Auditor run for the baseline build 28f5559 was not triggered

Recommendations

Please merge your branch for this Pull request with the latest master build and commit your changes once again

apps/stress-test/scripts/utils/configureYargs.js

apps/stress-test/README.md

apps/stress-test/scenarios/default.js

apps/stress-test/scripts/stressTest.js

apps/stress-test/scripts/commands/buildTestConfig.js

layershifter · 2022-09-07T08:30:56Z

apps/stress-test/scripts/utils/tachometer.js

+    // Make sure we run these in sequence.
+    // We don't want to run several tests at once as they
+    // may unduly influence each other.
+    const res = await execAsync(cmd);
+    console.log(res.stdout);


FYI: you can use execSync from @fluentui/scripts.

What do I need to do to use @fluentui/scripts? I added the package to Stress Test's devDeps but I get

Error: Cannot find module '@fluentui/scripts'

When attempting to require() the module.

FYI: you can use execSync from @fluentui/scripts.

note: I would recommend to not use that scripts abstraction, it's kinda shady and not proper part of public API of this package

apps/stress-test/package.json

fabricteam · 2022-09-07T16:20:36Z

Perf Analysis (`@fluentui/react-components`)

No significant results to display.

All results

Scenario	Render type	Master Ticks	PR Ticks	Iterations
Avatar	mount	1313	1297	5000
Button	mount	877	955	5000
FluentProvider	mount	1466	1536	5000
FluentProviderWithTheme	mount	626	627	10
FluentProviderWithTheme	virtual-rerender	594	587	10
FluentProviderWithTheme	virtual-rerender-with-unmount	622	628	10
MakeStyles	mount	1854	1930	50000
SpinButton	mount	2457	2492	5000

fabricteam · 2022-09-07T16:31:57Z

Perf Analysis (`@fluentui/react-northstar`)

Perf tests with no regressions

Scenario	Current PR Ticks	Baseline Ticks	Ratio
AccordionMinimalPerf.default	126	115	1.1:1
AttachmentMinimalPerf.default	123	116	1.06:1
RefMinimalPerf.default	186	176	1.06:1
IconMinimalPerf.default	558	527	1.06:1
AvatarMinimalPerf.default	157	150	1.05:1
ChatDuplicateMessagesPerf.default	250	237	1.05:1
AnimationMinimalPerf.default	442	427	1.04:1
CarouselMinimalPerf.default	390	375	1.04:1
ListNestedPerf.default	458	442	1.04:1
LoaderMinimalPerf.default	566	543	1.04:1
SegmentMinimalPerf.default	282	272	1.04:1
ButtonSlotsPerf.default	466	453	1.03:1
ChatMinimalPerf.default	600	580	1.03:1
DividerMinimalPerf.default	290	282	1.03:1
GridMinimalPerf.default	278	269	1.03:1
ReactionMinimalPerf.default	306	296	1.03:1
TextMinimalPerf.default	284	276	1.03:1
ButtonMinimalPerf.default	135	133	1.02:1
DropdownManyItemsPerf.default	549	539	1.02:1
FlexMinimalPerf.default	235	231	1.02:1
FormMinimalPerf.default	311	304	1.02:1
HeaderMinimalPerf.default	291	285	1.02:1
ListMinimalPerf.default	423	413	1.02:1
PortalMinimalPerf.default	141	138	1.02:1
SliderMinimalPerf.default	1336	1310	1.02:1
TableManyItemsPerf.default	1567	1535	1.02:1
TooltipMinimalPerf.default	1952	1922	1.02:1
CheckboxMinimalPerf.default	1737	1717	1.01:1
DropdownMinimalPerf.default	2253	2229	1.01:1
ItemLayoutMinimalPerf.default	967	955	1.01:1
LabelMinimalPerf.default	310	307	1.01:1
MenuButtonMinimalPerf.default	1413	1399	1.01:1
PopupMinimalPerf.default	526	522	1.01:1
ProviderMinimalPerf.default	332	330	1.01:1
TableMinimalPerf.default	326	322	1.01:1
TreeMinimalPerf.default	667	660	1.01:1
TreeWith60ListItems.default	127	126	1.01:1
AttachmentSlotsPerf.default	902	906	1:1
BoxMinimalPerf.default	274	275	1:1
DialogMinimalPerf.default	634	631	1:1
HeaderSlotsPerf.default	622	625	1:1
InputMinimalPerf.default	940	936	1:1
ListWith60ListItems.default	509	507	1:1
MenuMinimalPerf.default	702	701	1:1
RosterPerf.default	1768	1768	1:1
ProviderMergeThemesPerf.default	1050	1055	1:1
SplitButtonMinimalPerf.default	3624	3612	1:1
CustomToolbarPrototype.default	2231	2236	1:1
VideoMinimalPerf.default	585	584	1:1
AlertMinimalPerf.default	224	226	0.99:1
ButtonOverridesMissPerf.default	1089	1097	0.99:1
EmbedMinimalPerf.default	3028	3051	0.99:1
ImageMinimalPerf.default	315	318	0.99:1
LayoutMinimalPerf.default	286	288	0.99:1
ListCommonPerf.default	509	514	0.99:1
ToolbarMinimalPerf.default	754	758	0.99:1
CardMinimalPerf.default	431	442	0.98:1
DatepickerMinimalPerf.default	4685	4768	0.98:1
SkeletonMinimalPerf.default	277	284	0.98:1
StatusMinimalPerf.default	556	565	0.98:1
RadioGroupMinimalPerf.default	349	358	0.97:1
TextAreaMinimalPerf.default	369	381	0.97:1
ChatWithPopoverPerf.default	288	318	0.91:1

fabricteam · 2022-09-07T16:32:00Z

Perf Analysis (`@fluentui/react`)

No significant results to display.

All results

Scenario	Render type	Master Ticks	PR Ticks	Iterations
BaseButton	mount	760	770	5000
Breadcrumb	mount	2462	2471	1000
Checkbox	mount	2227	2236	5000
CheckboxBase	mount	1942	1953	5000
ChoiceGroup	mount	3959	3967	5000
ComboBox	mount	836	818	1000
CommandBar	mount	9144	9241	1000
ContextualMenu	mount	11204	11049	1000
DefaultButton	mount	964	960	5000
DetailsRow	mount	3282	3262	5000
DetailsRowFast	mount	3235	3258	5000
DetailsRowNoStyles	mount	3133	3141	5000
Dialog	mount	2651	2695	1000
DocumentCardTitle	mount	140	152	1000
Dropdown	mount	2825	2814	5000
FocusTrapZone	mount	1629	1566	5000
FocusZone	mount	1536	1542	5000
IconButton	mount	1519	1492	5000
Label	mount	301	289	5000
Layer	mount	3811	3833	5000
Link	mount	401	399	5000
MenuButton	mount	1276	1251	5000
MessageBar	mount	1885	1823	5000
Nav	mount	2891	2867	1000
OverflowSet	mount	937	942	5000
Panel	mount	2092	2126	1000
Persona	mount	845	850	1000
Pivot	mount	1212	1219	1000
PrimaryButton	mount	1108	1106	5000
Rating	mount	6569	6599	5000
SearchBox	mount	1095	1095	5000
Shimmer	mount	2466	2480	5000
Slider	mount	1674	1712	5000
SpinButton	mount	4274	4289	5000
Spinner	mount	356	375	5000
SplitButton	mount	2678	2681	5000
Stack	mount	428	441	5000
StackWithIntrinsicChildren	mount	1929	1917	5000
StackWithTextChildren	mount	4394	4365	5000
SwatchColorPicker	mount	10019	10068	5000
TagPicker	mount	2228	2204	5000
TeachingBubble	mount	88427	93178	5000
Text	mount	377	361	5000
TextField	mount	1177	1191	5000
ThemeProvider	mount	1112	1101	5000
ThemeProvider	virtual-rerender	657	647	5000
ThemeProvider	virtual-rerender-with-unmount	1741	1735	5000
Toggle	mount	691	692	5000
buttonNative	mount	113	108	5000

spmonahan · 2022-09-07T17:11:15Z

/azp run

azure-pipelines · 2022-09-07T17:11:35Z

Azure Pipelines successfully started running 3 pipeline(s).

layershifter · 2022-09-07T17:25:06Z

apps/stress-test/scripts/utils/configureYargs.js

+const { getBrowsers } = require('./getBrowsers');
+
+/**
+ * @typedef {Object.<string, import('yargs').Options} YargsOptions


Do this really work? O_o

It passed type-check but I see a missing > so I'm not sure how thoroughly the JSDocs are actually being checked. 🤔🤔🤔

I have a follow up PR where I need to do some more work with types so I'll investigate further there.

apps/stress-test/scripts/commands/runTachometer.js

apps/stress-test/scripts/commands/runServer.js

apps/stress-test/scripts/commands/run.js

spmonahan · 2022-09-07T18:58:40Z

/azp run

azure-pipelines · 2022-09-07T18:59:00Z

Azure Pipelines successfully started running 3 pipeline(s).

msft-fluent-ui-bot assigned micahgodbolt Aug 26, 2022

msft-fluent-ui-bot added the Area: Performance label Aug 26, 2022

github-actions bot added this to the July Project Cycle Q3 2022 milestone Aug 26, 2022

spmonahan requested review from a team, layershifter and ling1726 August 26, 2022 22:08

sopranopillow reviewed Aug 26, 2022

View reviewed changes

apps/stress-test/scripts/utils/configureYargs.js Outdated Show resolved Hide resolved

add missing 'getBrowsers' script.

d923b5b

spmonahan mentioned this pull request Aug 29, 2022

Stress Test: add build commands #24575

Merged

layershifter reviewed Sep 7, 2022

View reviewed changes

spmonahan added 2 commits September 7, 2022 08:43

pr feedback

257953f

readme updates

9d1c11e

spmonahan requested a review from a team as a code owner September 7, 2022 16:06

layershifter reviewed Sep 7, 2022

View reviewed changes

apps/stress-test/scripts/commands/runTachometer.js Outdated Show resolved Hide resolved

layershifter reviewed Sep 7, 2022

View reviewed changes

apps/stress-test/scripts/commands/runServer.js Outdated Show resolved Hide resolved

layershifter reviewed Sep 7, 2022

View reviewed changes

apps/stress-test/scripts/commands/run.js Outdated Show resolved Hide resolved

layershifter approved these changes Sep 7, 2022

View reviewed changes

clean up comments

c2d3b20

spmonahan merged commit 76e7b65 into microsoft:master Sep 7, 2022

Conversation

spmonahan commented Aug 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Current Behavior

New Behavior

Testing

Related Issue(s)

Uh oh!

codesandbox-ci bot commented Aug 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

fabricteam commented Aug 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📊 Bundle size report

Uh oh!

size-auditor bot commented Aug 26, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Asset size changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

layershifter Sep 7, 2022

Choose a reason for hiding this comment

Uh oh!

spmonahan Sep 7, 2022

Choose a reason for hiding this comment

Uh oh!

Hotell Sep 13, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

fabricteam commented Sep 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Perf Analysis (@fluentui/react-components)

Uh oh!

fabricteam commented Sep 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Perf Analysis (@fluentui/react-northstar)

Uh oh!

fabricteam commented Sep 7, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Perf Analysis (@fluentui/react)

Uh oh!

spmonahan commented Sep 7, 2022

Uh oh!

azure-pipelines bot commented Sep 7, 2022

Uh oh!

layershifter Sep 7, 2022

Choose a reason for hiding this comment

Uh oh!

spmonahan Sep 7, 2022

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

spmonahan commented Sep 7, 2022

Uh oh!

azure-pipelines bot commented Sep 7, 2022

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

spmonahan commented Aug 26, 2022 •

edited

Loading

codesandbox-ci bot commented Aug 26, 2022 •

edited

Loading

fabricteam commented Aug 26, 2022 •

edited

Loading

size-auditor bot commented Aug 26, 2022 •

edited

Loading

Hotell Sep 13, 2022 •

edited

Loading

fabricteam commented Sep 7, 2022 •

edited

Loading

Perf Analysis (`@fluentui/react-components`)

fabricteam commented Sep 7, 2022 •

edited

Loading

Perf Analysis (`@fluentui/react-northstar`)

fabricteam commented Sep 7, 2022 •

edited

Loading

Perf Analysis (`@fluentui/react`)