unskip maps tests by rashmivkulkarni · Pull Request #90323 · elastic/kibana

rashmivkulkarni · 2021-02-04T19:50:35Z

unskip functional maps tests.
Part of #89788
Fixes: #38137
Fixes: #36011

Note: had to skip the layer_error test for reasons mentioned below in comments.
Ref: #69617

elasticmachine · 2021-02-04T19:50:38Z

Pinging @elastic/kibana-gis (Team:Geo)

rashmivkulkarni · 2021-02-05T22:48:55Z

https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/1249/ ( running it on Flaky test runner as well on CI group9 where this test belongs- for 50 times).

nreese

Would you also mind updating the baseline/flights_map.png?

LGTM, thanks for turning on these skipped tests.
code review

rashmivkulkarni · 2021-02-09T22:21:24Z

This test is run on :

cloud - https://internal-ci.elastic.co/view/All/job/elastic+estf-cloud-kibana-flaky-test-runner/210/

Error: expected 'Unable to find EMS tile configuration for id: . Kibana is unable to access Elastic Maps Service. Contact your system administrator.' to equal 'Unable to find EMS tile configuration for id: idThatDoesNotExitForEMSTile. Kibana is unable to access Elastic Maps Service. Contact your system administrator.'

Kibana CI - https://kibana-ci.elastic.co/job/elastic+kibana+pipeline-pull-request/105344/

passed

ESTF on Windows: https://internal-ci.elastic.co/view/All/job/elastic+estf-kibana+7.11-snapshot+multijob-windows/47/

Error: expected 'Unable to find EMS tile configuration for id: . Kibana is unable to access Elastic Maps Service. Contact your system administrator.' to equal 'Unable to find EMS tile configuration for id: idThatDoesNotExitForEMSTile. Kibana is unable to access Elastic Maps Service. Contact your system administrator.'

LeeDr · 2021-02-09T22:24:16Z

On my local Windows 10 running not headless the first 2 of the 3 sample_data test passed and the 3rd one failed with these diffs;

           │ debg percent different: 0.013915625
           └- √ pass  (2.2s) "maps app  maps loaded from sample data ecommerce should load layers"
           │ debg percent different: 0.018916875
           └- √ pass  (2.7s) "maps app  maps loaded from sample data flights should load saved object and display layers"
            │ debg percent different: 0.03130125
           └- × fail: maps app  maps loaded from sample data web logs should load saved object and display layers
           │      Error: expected 0.03130125 to be below 0.02

But running the test with TEST_HEADLESS_BROWSER=1 (Jenkins runs headless) gave better results with all 3 passing (percent different < .01);

           │ debg percent different: 0.005636875
           └- √ pass  (1.7s) "maps app  maps loaded from sample data ecommerce should load layers"
           │ debg percent different: 0.00162
           └- √ pass  (1.6s) "maps app  maps loaded from sample data flights should load saved object and display layers"
           │ debg percent different: 0.007640625
           └- √ pass  (1.4s) "maps app  maps loaded from sample data web logs should load saved object and display layers"

LeeDr

LGTM - If it passes on the cloud flaky test runner you're all good. If it doesn't pass let me know.

LeeDr · 2021-02-10T17:23:53Z

The layer_errors test passed for me locally on Windows running from source on this PR. But it fails on both our Cloud and Windows Jenkins jobs (which both use the snapshot build). We could try it locally against a snapshot build.

LeeDr

Can we skip just this one layer_errors test please.

x-pack/test/functional/apps/maps/layer_errors.js

kibanamachine · 2021-02-11T02:07:39Z

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`maps`	2.8MB	2.8MB	+14.0B
`triggersActionsUi`	1.5MB	1.5MB	-26.9KB
total			-26.9KB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
`triggersActionsUi`	163.1KB	163.2KB	+102.0B

Unknown metric groups

async chunk count

id	before	after	diff
`triggersActionsUi`	31	32	+1

History

💔 Build #105807 failed 6fc0795
💚 Build #105344 succeeded 9d719a4
💔 Build #105284 failed bad3cd9
💔 Build #104972 failed bc46367
💔 Build #104937 failed b9f38a7

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

* unskip maps tests * checking the baseline images * updated the test to move the mouse away and close the Legend * more changes to the test * reducing the threshold limit * updating the baseline images * added a comment about the baseline images * updating flights baseline image and adjusting threshold * updated threshold and baseline image for web log * session image of weblogsmap * skipping layer_errors test * skip the test - as it fails on cloud and windows on snapshot

…errors) and also added baseline images (#91205) * fixes #74449 * unskip maps tests (#90323) * unskip maps tests * checking the baseline images * updated the test to move the mouse away and close the Legend * more changes to the test * reducing the threshold limit * updating the baseline images * added a comment about the baseline images * updating flights baseline image and adjusting threshold * updated threshold and baseline image for web log * session image of weblogsmap * skipping layer_errors test * skip the test - as it fails on cloud and windows on snapshot

…errors) and also added baseline images (elastic#91205) * fixes elastic#74449 * unskip maps tests (elastic#90323) * unskip maps tests * checking the baseline images * updated the test to move the mouse away and close the Legend * more changes to the test * reducing the threshold limit * updating the baseline images * added a comment about the baseline images * updating flights baseline image and adjusting threshold * updated threshold and baseline image for web log * session image of weblogsmap * skipping layer_errors test * skip the test - as it fails on cloud and windows on snapshot

…errors) and also added baseline images (#91205) (#91973) * fixes #74449 * unskip maps tests (#90323) * unskip maps tests * checking the baseline images * updated the test to move the mouse away and close the Legend * more changes to the test * reducing the threshold limit * updating the baseline images * added a comment about the baseline images * updating flights baseline image and adjusting threshold * updated threshold and baseline image for web log * session image of weblogsmap * skipping layer_errors test * skip the test - as it fails on cloud and windows on snapshot

…errors) and also added baseline images (#91205) (#91972) * fixes #74449 * unskip maps tests (#90323) * unskip maps tests * checking the baseline images * updated the test to move the mouse away and close the Legend * more changes to the test * reducing the threshold limit * updating the baseline images * added a comment about the baseline images * updating flights baseline image and adjusting threshold * updated threshold and baseline image for web log * session image of weblogsmap * skipping layer_errors test * skip the test - as it fails on cloud and windows on snapshot

unskip maps tests

a49c0d6

rashmivkulkarni added Team:Geo Former Team Label for Geo Team. Now use Team:Presentation test_xpack_functional v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.11.0 labels Feb 4, 2021

rashmivkulkarni self-assigned this Feb 4, 2021

rashmivkulkarni mentioned this pull request Feb 4, 2021

unskip functional tests which are skipped. #89788

Closed

rashmivkulkarni added 2 commits February 5, 2021 10:05

checking the baseline images

57b7de7

updated the test to move the mouse away and close the Legend

2ef4cb4

rashmivkulkarni requested review from LeeDr and nreese February 5, 2021 22:44

nreese approved these changes Feb 5, 2021

View reviewed changes

rashmivkulkarni added 8 commits February 5, 2021 17:23

more changes to the test

228ba83

reducing the threshold limit

d105692

updating the baseline images

fbd6745

added a comment about the baseline images

a398f7d

updating flights baseline image and adjusting threshold

b9f38a7

updated threshold and baseline image for web log

bc46367

session image of weblogsmap

bad3cd9

Merge branch '7.11' of github.com:elastic/kibana into unskip-maps-7.11

9d719a4

LeeDr approved these changes Feb 9, 2021

View reviewed changes

LeeDr suggested changes Feb 10, 2021

View reviewed changes

x-pack/test/functional/apps/maps/layer_errors.js Outdated Show resolved Hide resolved

rashmivkulkarni mentioned this pull request Feb 10, 2021

kibana/x-pack/test/functional/apps/maps/layer_errors.js failing on cloud and windows on a snapshot build #91043

Closed

skipping layer_errors test

6fc0795

skip the test - as it fails on cloud and windows on snapshot

f503be5

rashmivkulkarni merged commit b1134bb into elastic:7.11 Feb 11, 2021

rashmivkulkarni mentioned this pull request Feb 11, 2021

Unskip and Fix maps x-pack functional test - ( sample data and layer_errors) and also added baseline images #91205

Merged

nreese mentioned this pull request Nov 23, 2021

Failing test: Chrome X-Pack UI Functional Tests.x-pack/test/functional/apps/maps/layer_errors·js - maps app layer errors ESJoinSource with missing index pattern id should diplay error message in layer panel #69617

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

unskip maps tests#90323

unskip maps tests#90323
rashmivkulkarni merged 13 commits intoelastic:7.11from
rashmivkulkarni:unskip-maps-7.11

rashmivkulkarni commented Feb 4, 2021 •

edited

Loading

Uh oh!

elasticmachine commented Feb 4, 2021

Uh oh!

rashmivkulkarni commented Feb 5, 2021

Uh oh!

nreese left a comment

Uh oh!

rashmivkulkarni commented Feb 9, 2021 •

edited by LeeDr

Loading

Uh oh!

LeeDr commented Feb 9, 2021

Uh oh!

LeeDr left a comment

Uh oh!

LeeDr commented Feb 10, 2021

Uh oh!

LeeDr left a comment

Uh oh!

Uh oh!

kibanamachine commented Feb 11, 2021

async chunk count

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

rashmivkulkarni commented Feb 4, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticmachine commented Feb 4, 2021

Uh oh!

rashmivkulkarni commented Feb 5, 2021

Uh oh!

nreese left a comment

Choose a reason for hiding this comment

Uh oh!

rashmivkulkarni commented Feb 9, 2021 • edited by LeeDr Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

LeeDr commented Feb 9, 2021

Uh oh!

LeeDr left a comment

Choose a reason for hiding this comment

Uh oh!

LeeDr commented Feb 10, 2021

Uh oh!

LeeDr left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kibanamachine commented Feb 11, 2021

💚 Build Succeeded

Metrics [docs]

Async chunks

Page load bundle

async chunk count

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

rashmivkulkarni commented Feb 4, 2021 •

edited

Loading

rashmivkulkarni commented Feb 9, 2021 •

edited by LeeDr

Loading