Skip to content

[UI tests] Run only canary tests on a PR#2099

Closed
Stojdza wants to merge 6 commits intodevelopfrom
try/e2e/subset-of-tests
Closed

[UI tests] Run only canary tests on a PR#2099
Stojdza wants to merge 6 commits intodevelopfrom
try/e2e/subset-of-tests

Conversation

@Stojdza
Copy link
Copy Markdown
Contributor

@Stojdza Stojdza commented Apr 2, 2020

Run a subset of tests instead of a full suite against every PR. This is an experiment and the purpose of it is to reduce waiting time for CircleCI jobs and avoid running full suite against every commit. More info - p9ugOq-106-p2

Things to consider before we go with it:

  • What tests/functionalities we want to cover with canary tests?
  • Number of workers? With a smaller number of tests the jobs will wait less time when we have high traffic. But, should we reduce number of workers? Right now it's three.

To test:
Look at the tests in CircleCI jobs. Are they green?
What about execution time? Are they faster than before?

Note:
This option should not go to develop before we add the option to run the full suite against PR.

PR submission checklist:

  • I have considered adding unit tests where possible.
  • I have considered if this change warrants user-facing release notes and have added them to RELEASE-NOTES.txt if necessary.

@Stojdza Stojdza added the Testing Anything related to automated tests label Apr 2, 2020
@Stojdza Stojdza self-assigned this Apr 2, 2020
@Stojdza
Copy link
Copy Markdown
Contributor Author

Stojdza commented Apr 3, 2020

UI tests jobs' names should be updated in Branch protection rules once this PR is ready for merge https://github.com/wordpress-mobile/gutenberg-mobile/settings/branches

@Stojdza Stojdza requested review from ceyhun, maxme and mchowning April 3, 2020 15:17
@Stojdza Stojdza marked this pull request as ready for review April 3, 2020 15:17
@Stojdza
Copy link
Copy Markdown
Contributor Author

Stojdza commented Apr 9, 2020

Changes from this PR are merged to #2137

@Stojdza Stojdza closed this Apr 9, 2020
@Stojdza Stojdza deleted the try/e2e/subset-of-tests branch April 9, 2020 11:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Testing Anything related to automated tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant