Skip to content

UI tests: Add MVP e2e configuration#12403

Merged
brbrr merged 17 commits intomasterfrom
add/e2e-tests-for-ci
May 23, 2019
Merged

UI tests: Add MVP e2e configuration#12403
brbrr merged 17 commits intomasterfrom
add/e2e-tests-for-ci

Conversation

@brbrr
Copy link
Copy Markdown
Contributor

@brbrr brbrr commented May 17, 2019 •
edited
Loading

This time for real. This PR adds a way to run ngrok powered WP installation in CI. Also includes a dummy test which tries to login into newly created WP site.

I plan to have these tests disabled after this PR will be merged since it does not include any actual tests. In further PR's I'll plan to add some real tests

Few things that I not really sure about, and would like to get some feedback on these:

  • travis config: stages vs one more parallel build. Former approach doubles build execution time (6min total). The overall e2e build time is 3 minutes, but it will grow as we'll add more tests. UPD: I updated travis config to run e2e job in parallel, and made it optional (e.g. failing job won't fail the build) Switched the job to run in parallel as the other jobs
  • dependencies in root package.json. E2E tests don't have much in common with root dependencies so it might be a good idea to move them into its own file. Not sure that we really need that. Decided not to complicate things for now.

Changes proposed in this Pull Request:

  • add bash script to provision ngrok powered WP site in CI context
  • add a dummy test to login into a newly created site

Is this a new feature or does it add/remove features to an existing part of Jetpack?

  • p9dueE-Ju-p2

Testing instructions:

  • All the magic should happen inside CI build. In a job log, you should see something similar to:
$ JEST_PUPPETEER_CONFIG=tests/e2e/jest-puppeteer.config.js jest --config tests/e2e/jest.config.js
 PASS  tests/e2e/specs/dummy.test.js (22.885s)
  First test
    ✓ Can login to wp-admin and click Connect on Jetpack page (22205ms)

  console.log tests/e2e/lib/pageHelper.js:10
    Found element by locator: #welcome-panel

Test Suites: 1 passed, 1 total
Tests:       1 passed, 1 total
Snapshots:   0 total
Time:        22.943s
Ran all test suites matching /tests\/e2e\/specs\/dummy.test.js/i.
✨  Done in 24.25s.

Proposed changelog entry for your changes:

  • none

@brbrr brbrr added [Status] In Progress Unit Tests [Focus] Blocks Issues related to the block editor, aka Gutenberg, and its extensions developed in Jetpack labels May 17, 2019
@brbrr brbrr requested a review from a team May 17, 2019 11:08
@brbrr brbrr self-assigned this May 17, 2019
@matticbot
Copy link
Copy Markdown
Contributor

matticbot commented May 17, 2019

Caution: This PR has changes that must be merged to WordPress.com
Hello brbrr! These changes need to be synced to WordPress.com - If you 're an a11n, please commandeer, review, and approve D28358-code before merging this PR. Thank you!

This was referenced May 17, 2019
Closed
Closed
Closed
simison
simison reviewed May 17, 2019
View reviewed changes
simison
simison reviewed May 17, 2019
View reviewed changes
@brbrr brbrr added [Status] Needs Review This PR is ready for review. and removed [Status] In Progress labels May 17, 2019
brbrr
brbrr commented May 17, 2019
View reviewed changes
package.json
"nock": "10.0.6",
"node-wp-i18n": "1.2.3",
"prettier": "https://github.com/Automattic/wp-prettier/releases/download/wp-1.17.0/wp-prettier-1.17.0.tgz",
"puppeteer": "1.16.0",
Copy link
Copy Markdown
Contributor Author

@brbrr brbrr May 17, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Will quote @simison's comment from other PR(#12346 (comment)):

Noting that this is a bit heavier dependency (discussed p1554884363738500-slack-jetpack-gutenberg).

$ yarn why puppeteer
...
info Disk size without dependencies: "196.74MB"
info Disk size with unique dependencies: "197.57MB"
info Disk size with transitive dependencies: "199.36MB"
I don't think it's soooo heavy that it's worth the hassle to make pulling it optional (https://docs.npmjs.com/files/package.json#optionaldependencies, https://docs.npmjs.com/files/package.json#peerdependencies) but just wanted to give heads up.

@jetpackbot
Copy link
Copy Markdown
Collaborator

jetpackbot commented May 20, 2019 •
edited
Loading

Thank you for the great PR description!

When this PR is ready for review, please apply the [Status] Needs Review label. If you are an a11n, please have someone from your team review the code if possible. The Jetpack team will also review this PR and merge it to be included in the next Jetpack release.

Scheduled Jetpack release: June 4, 2019.
Scheduled code freeze: May 28, 2019

Generated by 🚫 dangerJS against 531a924

@brbrr brbrr force-pushed the add/e2e-tests-for-ci branch from 7f64e5e to 955f9ce Compare May 23, 2019 14:12
@matticbot
Copy link
Copy Markdown
Contributor

matticbot commented May 23, 2019

brbrr, Your synced wpcom patch D28358-code has been updated.

dereksmart
dereksmart reviewed May 23, 2019
View reviewed changes
@kraftbj kraftbj added this to the 7.4 milestone May 23, 2019
kraftbj
kraftbj previously approved these changes May 23, 2019
View reviewed changes
@brbrr brbrr merged commit 9eb18f9 into master May 23, 2019
@brbrr brbrr deleted the add/e2e-tests-for-ci branch May 23, 2019 15:00
@matticbot matticbot removed the [Status] Needs Review This PR is ready for review. label May 23, 2019
@kraftbj kraftbj added [Status] Ready to Merge Go ahead, you can push that green button! and removed [Status] Ready to Merge Go ahead, you can push that green button! labels May 23, 2019
sirreal
sirreal reviewed May 24, 2019
View reviewed changes
package.json
"babel-loader": "8.0.6",
"chai": "4.2.0",
"commander": "2.20.0",
"config": "3.1.0",
Copy link
Copy Markdown
Member

@sirreal sirreal May 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Adding this dependency screws up this import:

jetpack/_inc/client/lib/analytics/index.js

Line 10 in 597bc7c

import config from 'config';

@sirreal sirreal mentioned this pull request May 24, 2019
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@simison simison simison left review comments

@sirreal sirreal sirreal left review comments

@dereksmart dereksmart dereksmart left review comments

@kraftbj kraftbj kraftbj approved these changes

Assignees

@brbrr brbrr

Labels

[Focus] Blocks Issues related to the block editor, aka Gutenberg, and its extensions developed in Jetpack Touches WP.com Files Unit Tests

Projects

None yet

Milestone

7.4

Development

Successfully merging this pull request may close these issues.

7 participants

@brbrr @matticbot @jetpackbot @simison @kraftbj @sirreal @dereksmart