Fix flaky 'publish panel' e2e test#78082
Conversation
|
The following accounts have interacted with this PR and/or linked issues. I will continue to update these lists as activity occurs. You can also manually ask me to refresh this list by adding the If you're merging code through a pull request on GitHub, copy and paste the following into the bottom of the merge commit message. To understand the WordPress project's expectations around crediting contributors, please review the Contributor Attribution page in the Core Handbook. |
|
Size Change: 0 B Total Size: 7.92 MB ℹ️ View Unchanged
|
187193a to
b6d2951
Compare
|
Flaky tests detected in b6d2951. 🔍 Workflow run URL: https://github.com/WordPress/gutenberg/actions/runs/25537829697
|
b6d2951 to
5693621
Compare
| const cancelButton = page | ||
| .getByRole( 'region', { name: 'Editor publish' } ) | ||
| .getByRole( 'button', { name: 'Cancel' } ); | ||
| await expect( cancelButton ).toBeEnabled(); |
There was a problem hiding this comment.
This is the actual flaky e2e test fix.
|
@Mamaduka I was also looking at this in the last few days, and also noticed that To trigger My findings described in https://github.com/WordPress/gutenberg/pull/77893/changes#r3217213285 indicate that there is a slide-in animation in progress when the test tries to click "Cancel". On a moving button. Very strange, as animations should be disabled with |
|
Thanks for additional info, @jsnajdr! I’ve not even considered animations because of that global plugin. This seem to reduce flakiness of the test, but that might be due extra check just giving animations enough time to finish. There’s no harm in this change, IMO. |
I agree, there's no harm checking if the button is enabled. But #77721 was automatically reopened just 6 hours after closing. I'm not sure if this can be caused by a PR branch that is still based on an unfixed |
|
Reported a Playwright bug here: microsoft/playwright#40772 Not sure if it's really the cause of this flaky test, but definitely a bug. |
|
@Mamaduka This patch, waiting for the Cancel button to be enabled, didn't fix the flaky test. And neither did @danluu's AI fuzzed #77893, which waits for the Publish button to be But I think I found the cause, even I don't understand yet how it happens. If you look at this video from one of the failed runs, notice that the "Cancel" button is enabled, then gets disabled for a brief moment, and then gets enabled again: video.webmThat means that The wait for |
|
@jsnajdr do you think it could be RTC related? Some other flaky test (like classic block) I tried to look into became more unstable very recently and only major change we have to data store is RTC. |
Maybe, but does RTC possibly affect
I was looking into the failure where classic "convert to block" loses selection after undo and there is no block toolbar to click. That could be possibly related to RTC, because RTC manages the undo stack which also does the selection restoration job. But I still understand RTC only very superficially. |
What?
Closes #77721.
PR fixes flaky
should move focus back to the Publish panel toggle button when cancelinge2e test. Ensure theCancelbutton is enabled before clicking on it.Cancel button uses
disabled={ isSavingNonPostEntityChanges } + accessibleWhenDisabled. When both are true, the Button rendersaria-disabledand swallows clicks viastopPropagation/preventDefault, but Playwright still sees it as interactable. On CI, transient non-post entity saves (e.g., preferences) can flipdisabledto true right whenpage.click('… Cancel')fires; the click dispatches, butonClosenever runs, the panel stays open, and the focus assertion fails.Awaiting
toBeEnabled()(which treatsaria-disabledas disabled) prevents the button's click from being actionable.I've also migrated locators to use
getByRole, which fixes a couple of ESLint warnings in the file.How?
Testing Instructions
CI checks should be green.