[APM] E2E: Zero config for running e2e locally by sorenlouv · Pull Request #59152 · elastic/kibana

sorenlouv · 2020-03-03T15:01:52Z

Changes

Use env vars for cypress (instead of importing kibana.dev.yml)
Use --config to load kibana.e2e.yml instead of the user's kibana.dev.yml.
Added run-e2e.sh script to automatically setup environment and run tests

TODO:

apm-integration-testing should return a non-zero exit code if it encounters errors (necessary to stop subsequent steps) (feat: return the exit code of docker compose in case of failure apm-integration-testing#803)
Kibana should be started in --dev mode. This is currently not possible because it disables the --config flag.

Helpful error messages

sorenlouv · 2020-03-03T15:03:09Z

x-pack/legacy/plugins/apm/e2e/README.md

Enabling Security and ES and APM Server on non-default ports to avoid clashing with local development

sorenlouv · 2020-03-03T15:04:14Z

x-pack/legacy/plugins/apm/e2e/README.md

Refer to e2e specific config file (instead of moving it around)

sorenlouv · 2020-03-03T15:04:43Z

x-pack/legacy/plugins/apm/e2e/README.md

This is now specified in kibana.e2e.yml. Not something the user has to worry about.

sorenlouv · 2020-03-03T15:05:38Z

x-pack/legacy/plugins/apm/e2e/ci/kibana.e2e.yml

Running kibana on a non-default port to avoid clashing with local development

sorenlouv · 2020-03-03T15:06:55Z

x-pack/legacy/plugins/apm/e2e/ci/kibana.e2e.yml

Added all apm configurations to ensure they don't break without us noticing

sorenlouv · 2020-03-03T15:08:00Z

x-pack/legacy/plugins/apm/e2e/cypress.json

Provide credentials to Cypress via environment vars

sorenlouv · 2020-03-03T23:59:35Z

retest

v1v · 2020-03-04T09:07:12Z

x-pack/legacy/plugins/apm/e2e/ci/kibana.e2e.yml

kibana_system_user didn't work when I tried, so maybe admin could be a good one, what do you think?

Suggested change

elasticsearch.username: 'kibana_system_user'

elasticsearch.username: 'admin'

Kibana shouldn't connect to ES with superuser privileges. According to the docs kibana_system_user should exist.

I can confirm that kibana_system_user can access ES like:

curl --user kibana_system_user:changeme -X GET "localhost:9201/_security/user/_has_privileges?pretty" -H 'Content-Type: application/json' -d'{ "cluster": [ "monitor", "manage" ] }'

which returns:

{ "username" : "kibana_system_user", "has_all_requested" : false, "cluster" : { "monitor" : true, "manage" : false }, "index" : { }, "application" : { } }

Can you try running that and LMK if it works?

smith

👍 once conflicts resolved and up to date and tests pass.

x-pack/legacy/plugins/apm/e2e/ci/prepare-kibana.sh

Co-Authored-By: Victor Martinez <victormartinezrubio@gmail.com>

cachedout · 2020-03-24T09:36:56Z

Hi @sqren

I'm helping to triage high-priority issues for @elastic/observablt-robots . Just so I understand, are the TODO items listed above part of this PR or is the intention to address them in follow-up PRs?

sorenlouv · 2020-03-24T11:14:11Z

Hey @cachedout,

I've just pushed a commit that fixes most of the issues. Only remaining issue is that apm-integration-testing does not return a non-zero exit code upon error. This should be fixed in https://github.com/elastic/apm-integration-testing but not something that should block this PR.

I'll merge this as soon as CI is green

kuisathaverat · 2020-03-24T12:22:56Z

@sqren Do exist an issue for that on the https://github.com/elastic/apm-integration-testing? I do not see any. Would you mind fill one with the details of what happens?

sorenlouv · 2020-03-24T12:38:34Z

@kuisathaverat elastic/apm-integration-testing#800

sorenlouv · 2020-03-24T15:10:59Z

retest

v1v · 2020-03-24T15:29:56Z

x-pack/legacy/plugins/apm/e2e/ci/prepare-kibana.sh

 ## Might help to avoid FATAL ERROR: Ineffective mark-compacts near heap limit Allocation failed - JavaScript heap out of memory
 export NODE_OPTIONS="--max-old-space-size=4096"
-nohup node scripts/kibana --no-base-path --csp.strict=false --optimize.watch=false> kibana.log 2>&1 &
+nohup node scripts/kibana --config "${E2E_DIR}/ci/kibana.e2e.yml" --no-base-path --optimize.watch=false> kibana.log 2>&1 &


What's the difference with https://github.com/elastic/kibana/pull/59152/files#diff-ccfe59ee0ec66fe86f6596aaf1fce60cR24? Are those flags also required in the CI?

This starts in dev mode

node ./scripts/kibana --no-base-path --dev --no-dev-config --config x-pack/legacy/plugins/apm/e2e/ci/kibana.e2e.yml

This starts in prod mode

node scripts/kibana --no-base-path --optimize.watch=false --config "${E2E_DIR}/ci/kibana.e2e.yml"

I think we should choose whatever is faster on CI. I guess we can test by running it 100 times or so on the CI in each mode.
I imagine dev compiles faster (because it doesn't need to compress resources) but prod has faster browser load times because assets are smaller. So could go both ways.

The reason I opted for using dev mode locally is because this is what developers is normally running, and I saw some issues when running prod mode without cleaning up after having run dev mode.

Thanks for the clarification, I'd say to use the same as the local one to avoid any misleading, then it will be easy to reproduce it anyway. Thanks again

sorenlouv · 2020-03-24T15:43:24Z

@elasticmachine merge upstream

then we don't need to trap the errorlevel

kibanamachine · 2020-03-26T00:12:44Z

💚 Build Succeeded

continuous-integration/kibana-ci/pull-request
Commit: 46b518c

History

💚 Build #35850 succeeded 46b518c
💚 Build #35830 succeeded e3d0391
💔 Build #35796 failed e3d0391
💚 Build #35772 succeeded 1bd70fa
💚 Build #30774 succeeded bdff7d06316a5168d162f5038d839b99f000f8b5

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

sorenlouv requested review from a team and v1v March 3, 2020 15:01

sorenlouv added release_note:skip Skip the PR/issue when compiling release notes v7.7.0 labels Mar 3, 2020

sorenlouv commented Mar 3, 2020

View reviewed changes

sorenlouv force-pushed the use-env-variables-e2e branch from 3ca9e26 to 1dfe5a3 Compare March 3, 2020 15:03

sorenlouv commented Mar 3, 2020

View reviewed changes

v1v approved these changes Mar 4, 2020

View reviewed changes

smith approved these changes Mar 18, 2020

View reviewed changes

sorenlouv changed the title ~~[APM] E2E: Use env vars over kibana.yml~~ [APM] E2E: Improve local DX by running all with single command Mar 20, 2020

sorenlouv changed the title ~~[APM] E2E: Improve local DX by running all with single command~~ [APM] E2E: Zero config for running e2e locally Mar 20, 2020

v1v reviewed Mar 21, 2020

View reviewed changes

x-pack/legacy/plugins/apm/e2e/ci/prepare-kibana.sh Outdated Show resolved Hide resolved

v1v added a commit to v1v/kibana that referenced this pull request Mar 21, 2020

cherry-pick some changes from elastic#59152

7139757

v1v added a commit to v1v/kibana that referenced this pull request Mar 22, 2020

cherry-pick from elastic#59152

26d1761

sorenlouv and others added 5 commits March 24, 2020 09:13

[APM] E2E: Use env vars over kibana.yml

51083c0

Add ./run-e2e script

27cf088

Add ./run-e2e script

5d456e9

Comment out debug stuff

3f2d266

Update x-pack/legacy/plugins/apm/e2e/ci/prepare-kibana.sh

d0b8706

Co-Authored-By: Victor Martinez <victormartinezrubio@gmail.com>

Fix cypress issues

4e8d51a

sorenlouv force-pushed the use-env-variables-e2e branch from e2dbda9 to 4e8d51a Compare March 24, 2020 11:12

Improve formatting

1bd70fa

Add --no-dev-config to kibana command

e3d0391

v1v reviewed Mar 24, 2020

View reviewed changes

v1v added a commit to v1v/kibana that referenced this pull request Mar 24, 2020

cherry-pick from elastic#59152

d5aa1db

Merge branch 'master' into use-env-variables-e2e

46b518c

v1v added a commit to v1v/kibana that referenced this pull request Mar 24, 2020

Cherry-pick from elastic#59152

bb22a3a

then we don't need to trap the errorlevel

sorenlouv merged commit 0fc0440 into elastic:master Mar 24, 2020

sorenlouv deleted the use-env-variables-e2e branch March 24, 2020 19:28

sorenlouv mentioned this pull request Mar 24, 2020

[7.x] [APM] E2E: Zero config for running e2e locally (#59152) #61146

Merged

sorenlouv added a commit that referenced this pull request Mar 24, 2020

[APM] E2E: Zero config for running e2e locally (#59152) (#61146)

ad57ddb

v1v mentioned this pull request Mar 30, 2020

ci(jenkins): use e2e in the CI #61803

Merged

2 tasks

	elasticsearch.username: 'kibana_system_user'
	elasticsearch.username: 'admin'

Conversation

sorenlouv commented Mar 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sorenlouv Mar 3, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sorenlouv commented Mar 3, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

smith left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cachedout commented Mar 24, 2020

Uh oh!

sorenlouv commented Mar 24, 2020

Uh oh!

kuisathaverat commented Mar 24, 2020

Uh oh!

sorenlouv commented Mar 24, 2020

Uh oh!

sorenlouv commented Mar 24, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sorenlouv Mar 24, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sorenlouv commented Mar 24, 2020

Uh oh!

kibanamachine commented Mar 26, 2020

💚 Build Succeeded

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

sorenlouv commented Mar 3, 2020 •

edited

Loading

sorenlouv Mar 3, 2020 •

edited

Loading

sorenlouv Mar 24, 2020 •

edited

Loading