-
Notifications
You must be signed in to change notification settings - Fork 4.8k
Perf Testing: Inconsistent methodology for testing typing performance #51383
Description
I've been working on migrating the performance tests to Playwright and noticed that the way we test typing performance is different for the Post Editor vs. Site Editor. For example, for the Site Editor, we simply hit x 200 times and record the metrics:
gutenberg/packages/e2e-tests/specs/performance/site-editor.test.js
Lines 166 to 167 in 524d6ef
| // Type "x" 200 times. | |
| await page.keyboard.type( new Array( 200 ).fill( 'x' ).join( '' ) ); |
...while for Post Editor, we make an arbitrary pause before each stroke (depending on the test):
Typing
gutenberg/packages/e2e-tests/specs/performance/post-editor.test.js
Lines 158 to 165 in 524d6ef
| while ( i-- ) { | |
| // Wait for the browser to be idle before starting the monitoring. | |
| // The timeout should be big enough to allow all async tasks tor run. | |
| // And also to allow Rich Text to mark the change as persistent. | |
| // eslint-disable-next-line no-restricted-syntax | |
| await page.waitForTimeout( 2000 ); | |
| await page.keyboard.type( 'x' ); | |
| } |
Typing within containers
gutenberg/packages/e2e-tests/specs/performance/post-editor.test.js
Lines 208 to 213 in 524d6ef
| while ( i-- ) { | |
| // Wait for the browser to be idle before starting the monitoring. | |
| // eslint-disable-next-line no-restricted-syntax | |
| await page.waitForTimeout( 500 ); | |
| await page.keyboard.type( 'x' ); | |
| } |
I'd like to verify if that pause is necessary, and if yes, if we can replace it with waiting for some UI element. This would simulate a real-user experience better and produce more consistent results, as wait timers with arbitrary values are known to be unstable.