[7.10] Make alert status fetching more resilient (#84676) by chrisronline · Pull Request #84753 · elastic/kibana

chrisronline · 2020-12-02T12:25:02Z

Backports the following commits to 7.10:

Make alert status fetching more resilient ([Monitoring] Make alert status fetching more resilient #84676)

kibanamachine · 2020-12-02T13:54:17Z

💛 Build succeeded, but was flaky

continuous-integration/kibana-ci/pull-request
Commit: 26d466a
Flaky suites:
- xpack-firefox

Test Failures

Firefox XPack UI Functional Tests.x-pack/test/functional/apps/grok_debugger/grok_debugger·js.logstash grok debugger app "before all" hook in "grok debugger app"

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 8 times on tracked branches: https://github.com/elastic/kibana/issues/84440

[00:00:00]       │
[00:02:44]         └-: logstash
[00:02:44]           └-> "before all" hook
[00:02:44]           └-: grok debugger app
[00:02:44]             └-> "before all" hook
[00:02:44]             └-> "before all" hook
[00:02:44]               │ info [empty_kibana] Loading "mappings.json"
[00:02:44]               │ proc [kibana]   log   [12:56:41.045] [warning][ui-settings-service] Ignore invalid UiSettings value. Error: [validation [defaultRoute]]: Must be a relative URL..
[00:02:44]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xxl-1606911944706241145] [.kibana_1/0RR24cdpSPG8y60-3WN_6w] deleting index
[00:02:44]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xxl-1606911944706241145] [.kibana_2/9eNzr9wJQ0iXNwA8no1wSg] deleting index
[00:02:44]               │ info [empty_kibana] Deleted existing index [".kibana_2",".kibana_1"]
[00:02:44]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1606911944706241145] [.kibana] creating index, cause [api], templates [], shards [1]/[1]
[00:02:44]               │ info [empty_kibana] Created index ".kibana"
[00:02:44]               │ debg [empty_kibana] ".kibana" settings {"index":{"number_of_replicas":"1","number_of_shards":"1"}}
[00:02:44]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xxl-1606911944706241145] [.kibana/nJC7nOeORTiqoXhKMZ9wBA] update_mapping [_doc]
[00:02:44]               │ debg Migrating saved objects
[00:02:44]               │ proc [kibana]   log   [12:56:41.214] [info][savedobjects-service] Creating index .kibana_2.
[00:02:44]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1606911944706241145] [.kibana_2] creating index, cause [api], templates [], shards [1]/[1]
[00:02:44]               │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xxl-1606911944706241145] updating number_of_replicas to [0] for indices [.kibana_2]
[00:02:44]               │ proc [kibana]   log   [12:56:41.285] [info][savedobjects-service] Reindexing .kibana to .kibana_1
[00:02:44]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-debian-tests-xxl-1606911944706241145] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1]
[00:02:44]               │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-debian-tests-xxl-1606911944706241145] updating number_of_replicas to [0] for indices [.kibana_1]
[00:02:44]               │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-debian-tests-xxl-1606911944706241145] 4680 finished with response BulkByScrollResponse[took=2.1ms,timed_out=false,sliceId=null,updated=0,created=0,deleted=0,batches=0,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:02:44]               │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-debian-tests-xxl-1606911944706241145] [.kibana/nJC7nOeORTiqoXhKMZ9wBA] deleting index
[00:02:44]               │ proc [kibana]   log   [12:56:41.647] [info][savedobjects-service] Migrating .kibana_1 saved objects to .kibana_2
[00:02:44]               │ proc [kibana]   log   [12:56:41.654] [info][savedobjects-service] Pointing alias .kibana to .kibana_2.
[00:02:44]               │ proc [kibana]   log   [12:56:41.681] [info][savedobjects-service] Finished in 469ms.
[00:02:44]               │ debg applying update to kibana config: {"accessibility:disableAnimations":true,"dateFormat:tz":"UTC"}
[00:02:44]               │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-debian-tests-xxl-1606911944706241145] [.kibana_2/hJc5eXMUToCWxlQo5tf0-A] update_mapping [_doc]
[00:02:46]               │ debg navigating to grokDebugger url: http://localhost:6131/app/dev_tools#/grokdebugger
[00:02:46]               │ debg navigate to: http://localhost:6131/app/dev_tools#/grokdebugger
[00:02:46]               │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:02:46]               │ debg ... sleep(700) start
[00:02:47]               │ debg ... sleep(700) end
[00:02:47]               │ debg returned from get, calling refresh
[00:02:47]               │ERROR browser[error] (new AbortError("The operation was aborted. ", "http://localhost:6131/36064/bundles/core/core.entry.js", 6))
[00:02:47]               │ debg browser[log] "Detected an unhandled Promise rejection.\nEmptyError: no elements in sequence"
[00:02:47]               │ debg browser[log] "Detected an unhandled Promise rejection.\nAbortError: The operation was aborted. "
[00:02:47]               │      console.log: "Detected an unhandled Promise rejection.\nEmptyError: no elements in sequence"
[00:02:48]               │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:02:48]               │ debg currentUrl = http://localhost:6131/login?next=%2Fapp%2Fdev_tools%3F_t%3D1606913803287#/grokdebugger
[00:02:48]               │          appUrl = http://localhost:6131/app/dev_tools#/grokdebugger
[00:02:48]               │ debg TestSubjects.find(kibanaChrome)
[00:02:48]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:02:49]               │ debg Found login page
[00:02:49]               │ debg TestSubjects.setValue(loginUsername, test_user)
[00:02:49]               │ debg TestSubjects.click(loginUsername)
[00:02:49]               │ debg Find.clickByCssSelector('[data-test-subj="loginUsername"]') with timeout=10000
[00:02:49]               │ debg Find.findByCssSelector('[data-test-subj="loginUsername"]') with timeout=10000
[00:02:50]               │ debg TestSubjects.setValue(loginPassword, changeme)
[00:02:50]               │ debg TestSubjects.click(loginPassword)
[00:02:50]               │ debg Find.clickByCssSelector('[data-test-subj="loginPassword"]') with timeout=10000
[00:02:50]               │ debg Find.findByCssSelector('[data-test-subj="loginPassword"]') with timeout=10000
[00:02:50]               │ debg TestSubjects.click(loginSubmit)
[00:02:50]               │ debg Find.clickByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:02:50]               │ debg Find.findByCssSelector('[data-test-subj="loginSubmit"]') with timeout=10000
[00:02:50]               │ proc [kibana]   log   [12:56:47.517] [info][plugins][routes][security] Logging in with provider "basic" (basic)
[00:02:50]               │ debg Find.waitForDeletedByCssSelector('.kibanaWelcomeLogo') with timeout=10000
[00:02:51]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:02:51]               │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"] nav:not(.ng-hide)') with timeout=60000
[00:02:51]               │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:02:53]               │ debg Finished login process currentUrl = http://localhost:6131/app/dev_tools#/grokdebugger
[00:02:53]               │ debg ... sleep(501) start
[00:02:53]               │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:02:53]               │ debg ... sleep(501) end
[00:02:53]               │ debg in navigateTo url = http://localhost:6131/app/dev_tools#/grokdebugger
[00:02:53]               │ debg TestSubjects.exists(statusPageContainer)
[00:02:53]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:02:56]               │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:02:56]               │ debg TestSubjects.exists(grokDebugger)
[00:02:56]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:02:59]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:02:59]               │ debg --- retry.try error: Expected to find the grok debugger
[00:03:00]               │ debg TestSubjects.exists(grokDebugger)
[00:03:00]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:02]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:03]               │ debg --- retry.try failed again with the same message...
[00:03:03]               │ debg TestSubjects.exists(grokDebugger)
[00:03:03]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:06]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:07]               │ debg --- retry.try failed again with the same message...
[00:03:07]               │ debg TestSubjects.exists(grokDebugger)
[00:03:07]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:10]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:10]               │ debg --- retry.try failed again with the same message...
[00:03:11]               │ debg TestSubjects.exists(grokDebugger)
[00:03:11]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:13]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:14]               │ debg --- retry.try failed again with the same message...
[00:03:14]               │ debg TestSubjects.exists(grokDebugger)
[00:03:14]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:17]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:17]               │ debg --- retry.try failed again with the same message...
[00:03:18]               │ debg TestSubjects.exists(grokDebugger)
[00:03:18]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:20]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:21]               │ debg --- retry.try failed again with the same message...
[00:03:21]               │ debg TestSubjects.exists(grokDebugger)
[00:03:21]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:24]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:24]               │ debg --- retry.try failed again with the same message...
[00:03:25]               │ debg TestSubjects.exists(grokDebugger)
[00:03:25]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:27]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:28]               │ debg --- retry.try failed again with the same message...
[00:03:28]               │ debg TestSubjects.exists(grokDebugger)
[00:03:28]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:31]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:31]               │ debg --- retry.try failed again with the same message...
[00:03:32]               │ debg TestSubjects.exists(grokDebugger)
[00:03:32]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:34]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:35]               │ debg --- retry.try failed again with the same message...
[00:03:35]               │ debg TestSubjects.exists(grokDebugger)
[00:03:35]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:38]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:38]               │ debg --- retry.try failed again with the same message...
[00:03:39]               │ debg TestSubjects.exists(grokDebugger)
[00:03:39]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:41]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:42]               │ debg --- retry.try failed again with the same message...
[00:03:42]               │ debg TestSubjects.exists(grokDebugger)
[00:03:42]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:45]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:45]               │ debg --- retry.try failed again with the same message...
[00:03:46]               │ debg TestSubjects.exists(grokDebugger)
[00:03:46]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:48]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:49]               │ debg --- retry.try failed again with the same message...
[00:03:49]               │ debg TestSubjects.exists(grokDebugger)
[00:03:49]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:03:52]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:03:52]               │ debg --- retry.try failed again with the same message...
[00:03:53]               │ debg TestSubjects.exists(grokDebugger)
[00:03:53]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:05]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:06]               │ debg --- retry.try failed again with the same message...
[00:04:06]               │ debg TestSubjects.exists(grokDebugger)
[00:04:06]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:09]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:09]               │ debg --- retry.try failed again with the same message...
[00:04:10]               │ debg TestSubjects.exists(grokDebugger)
[00:04:10]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:12]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:13]               │ debg --- retry.try failed again with the same message...
[00:04:13]               │ debg TestSubjects.exists(grokDebugger)
[00:04:13]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:16]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:16]               │ debg --- retry.try failed again with the same message...
[00:04:17]               │ debg TestSubjects.exists(grokDebugger)
[00:04:17]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:19]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:20]               │ debg --- retry.try failed again with the same message...
[00:04:20]               │ debg TestSubjects.exists(grokDebugger)
[00:04:20]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:23]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:23]               │ debg --- retry.try failed again with the same message...
[00:04:24]               │ debg TestSubjects.exists(grokDebugger)
[00:04:24]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:26]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:27]               │ debg --- retry.try failed again with the same message...
[00:04:27]               │ debg TestSubjects.exists(grokDebugger)
[00:04:27]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:30]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:30]               │ debg --- retry.try failed again with the same message...
[00:04:31]               │ debg TestSubjects.exists(grokDebugger)
[00:04:31]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:33]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:34]               │ debg --- retry.try failed again with the same message...
[00:04:34]               │ debg TestSubjects.exists(grokDebugger)
[00:04:34]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:37]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:37]               │ debg --- retry.try failed again with the same message...
[00:04:38]               │ debg TestSubjects.exists(grokDebugger)
[00:04:38]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:40]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:41]               │ debg --- retry.try failed again with the same message...
[00:04:41]               │ debg TestSubjects.exists(grokDebugger)
[00:04:41]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:44]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:44]               │ debg --- retry.try failed again with the same message...
[00:04:45]               │ debg TestSubjects.exists(grokDebugger)
[00:04:45]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:47]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:48]               │ debg --- retry.try failed again with the same message...
[00:04:49]               │ debg TestSubjects.exists(grokDebugger)
[00:04:49]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:51]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:52]               │ debg --- retry.try failed again with the same message...
[00:04:52]               │ debg TestSubjects.exists(grokDebugger)
[00:04:52]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:55]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:55]               │ debg --- retry.try failed again with the same message...
[00:04:56]               │ debg TestSubjects.exists(grokDebugger)
[00:04:56]               │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="grokDebugger"]') with timeout=2500
[00:04:58]               │ debg --- retry.tryForTime error: [data-test-subj="grokDebugger"] is not displayed
[00:04:59]               │ debg --- retry.try failed again with the same message...
[00:04:59]               │ info Taking screenshot "/dev/shm/workspace/parallel/3/kibana/x-pack/test/functional/screenshots/failure/logstash grok debugger app _before all_ hook.png"
[00:04:59]               │ info Current URL is: http://localhost:6131/app/dev_tools#/grokdebugger
[00:04:59]               │ info Saving page source to: /dev/shm/workspace/parallel/3/kibana/x-pack/test/functional/failure_debug/html/logstash grok debugger app _before all_ hook.html
[00:04:59]               └- ✖ fail: logstash grok debugger app "before all" hook in "grok debugger app"
[00:04:59]               │      retry.try timeout: Error: Expected to find the grok debugger
[00:04:59]               │     at retry.try (/dev/shm/workspace/parallel/3/kibana/x-pack/test/functional/services/grok_debugger.js:53:17)
[00:04:59]               │   Error: retry.try timeout: Error: Expected to find the grok debugger
[00:04:59]               │       at retry.try (test/functional/services/grok_debugger.js:53:17)
[00:04:59]               │       at onFailure (/dev/shm/workspace/parallel/3/kibana/test/common/services/retry/retry_for_success.ts:28:9)
[00:04:59]               │       at retryForSuccess (/dev/shm/workspace/parallel/3/kibana/test/common/services/retry/retry_for_success.ts:68:13)
[00:04:59]               │ 
[00:04:59]               │

Stack Trace

Error: retry.try timeout: Error: Expected to find the grok debugger
    at retry.try (test/functional/services/grok_debugger.js:53:17)
    at onFailure (/dev/shm/workspace/parallel/3/kibana/test/common/services/retry/retry_for_success.ts:28:9)
    at retryForSuccess (/dev/shm/workspace/parallel/3/kibana/test/common/services/retry/retry_for_success.ts:68:13)

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`monitoring`	1.1MB	1.1MB	+23.0B

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Make alert status fetching more resilient (elastic#84676)

26d466a

chrisronline added the backport This PR is a backport of another PR label Dec 2, 2020

chrisronline merged commit 753474c into elastic:7.10 Dec 2, 2020

chrisronline deleted the backport/7.10/pr-84676 branch December 2, 2020 14:03

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[7.10] Make alert status fetching more resilient (#84676)#84753

[7.10] Make alert status fetching more resilient (#84676)#84753
chrisronline merged 1 commit intoelastic:7.10from
chrisronline:backport/7.10/pr-84676

chrisronline commented Dec 2, 2020

Uh oh!

kibanamachine commented Dec 2, 2020

Standard Out

Stack Trace

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

chrisronline commented Dec 2, 2020

Uh oh!

kibanamachine commented Dec 2, 2020

💛 Build succeeded, but was flaky

Test Failures

Standard Out

Stack Trace

Metrics [docs]

Async chunks

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants