Conversation
4360c0f to
3aa165c
Compare
|
OK, I just produced 3 immediately successful browser CI runs in a row:
Either BrowserStack is in an extraordinarily good mood right at the moment or the webdriver.io based tests are indeed less flaky than my custom solution. |
|
Note: Should we decide to merge this one I will have to adjust the branch protection settings with respect to the required CI runs. |
|
Update: I re-ran the failed commits on master, had to re-run them a couple of times until being lucky and all 5 browser tests passed. I reran the webdriver tests, still consistently passing 🎉 ! |
broofa
left a comment
There was a problem hiding this comment.
Tested this locally using my personal browserstack account and it worked first time. Well done!
To be fair though: The old setup als usually passed when running from my machine. My suspicion is that something with the networking connection between GitHub actions and BrowserStack is off… Anyways the new setup looks much more stable. 9 successful builds from GitHub actions in a row and they are also reasonably fast (~2mins). |
Our current browser tests were incredibly flaky. Webdriver.io seems to contain retry logic and other safeguards that we are currently lacking in our custom browserstack-local / selenium-webdriver setup. Under the hood things should still work similarly, let's see if they are now more stable.
Our current browser tests were incredibly flaky.
Webdriver.io seems to contain retry logic and other safeguards that we are currently lacking in our custom browserstack-local / selenium-webdriver setup.
Under the hood things should still work similarly, let's see if they are now more stable.