Skip to content

[UI tests] Schedule runs for full UI tests suite#2115

Merged
maxme merged 11 commits intodevelopfrom
try/e2e/scheduled-runs
Apr 30, 2020
Merged

[UI tests] Schedule runs for full UI tests suite#2115
maxme merged 11 commits intodevelopfrom
try/e2e/scheduled-runs

Conversation

@Stojdza
Copy link
Copy Markdown
Contributor

@Stojdza Stojdza commented Apr 6, 2020

Add an option for UI tests to run every 12 hours against develop branch.

Let's see will it work 🤞

NOTE: This PR includes changes from #2099

To test:
Make sure that tests are passing.

PR submission checklist:

  • I have considered adding unit tests where possible.
  • I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

@Stojdza Stojdza added the Testing Anything related to automated tests label Apr 6, 2020
@Stojdza Stojdza requested review from ceyhun, maxme and mchowning April 6, 2020 16:53
@Stojdza Stojdza self-assigned this Apr 6, 2020
Copy link
Copy Markdown
Contributor

@maxme maxme left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How will we get notified of a failure of the scheduled ios-device-checks or android-device-checks tests?

@Stojdza
Copy link
Copy Markdown
Contributor Author

Stojdza commented Apr 8, 2020

How will we get notified of a failure of the scheduled ios-device-checks or android-device-checks tests?

@maxme Whoever follows this project in CircleCI will receive notifications and can manage them.

@rachelmcr
Copy link
Copy Markdown
Contributor

@maxme Will that CircleCI notification be sufficient or would you like something more visible?

We could also set up Slack notifications like in the WPAndroid and WPiOS repos using the CircleCI Slack orb. I could help with that setup; we'd just need to know where you'd like those notifications to go.

@maxme
Copy link
Copy Markdown
Contributor

maxme commented Apr 15, 2020

@maxme Will that CircleCI notification be sufficient or would you like something more visible?
We could also set up Slack notifications like in the WPAndroid and WPiOS repos using the CircleCI Slack orb. I could help with that setup; we'd just need to know where you'd like those notifications to go.

Yes I think this would be better to setup Slack notifications. Can we target the #mobile-gutenberg channel?

@rachelmcr
Copy link
Copy Markdown
Contributor

I added Slack notifications that will fire only for the scheduled test runs. When the tests fail, a Slack message will be posted to the #mobile-gutenberg channel depending on which device the failure happened on:

🔴 Scheduled tests failed on Android device!

🔴 Scheduled tests failed on iOS device!

The message will also include a button linking to the failed CircleCI job.

@Stojdza
Copy link
Copy Markdown
Contributor Author

Stojdza commented Apr 17, 2020

This looks great! Thanks @rachelmcr!

Copy link
Copy Markdown
Contributor

@rachelmcr rachelmcr left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving because it looks like this PR has stalled. The workflow changes look good to me, although it feels a little funny to approve a PR that also has my own changes in it. @maxme is this PR intentionally on hold or is it ok to ship?

@maxme maxme merged commit c9b11f4 into develop Apr 30, 2020
@maxme maxme deleted the try/e2e/scheduled-runs branch April 30, 2020 07:43
@maxme
Copy link
Copy Markdown
Contributor

maxme commented Apr 30, 2020

:shipit:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Testing Anything related to automated tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants