Skip to content

[Uptime] Set ML anomaly look-back to 2w (from 24h) / Add spinner#65055

Merged
shahzad31 merged 3 commits intoelastic:masterfrom
shahzad31:ml-job-time
May 5, 2020
Merged

[Uptime] Set ML anomaly look-back to 2w (from 24h) / Add spinner#65055
shahzad31 merged 3 commits intoelastic:masterfrom
shahzad31:ml-job-time

Conversation

@shahzad31
Copy link
Copy Markdown
Contributor

Summary

Fixes: #64657

Increase anomaly job time analyzing period to 2 weeks.
Also added loading state while job is being created or loaded.

@shahzad31 shahzad31 requested a review from andrewvc May 4, 2020 10:37
@shahzad31 shahzad31 self-assigned this May 4, 2020
@shahzad31 shahzad31 added release_note:fix Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability v7.8.0 v8.0.0 and removed release_note:fix labels May 4, 2020
@elasticmachine
Copy link
Copy Markdown
Contributor

Pinging @elastic/uptime (Team:uptime)

@shahzad31 shahzad31 added the release_note:skip Skip the PR/issue when compiling release notes label May 4, 2020
@andrewvc andrewvc changed the title [Uptime] Increase anomaly job time analyzing period to 2 weeks [Uptime] Set ML anomaly look-back to 2w (from 24h) / Add spinner May 5, 2020
Copy link
Copy Markdown
Contributor

@andrewvc andrewvc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Generally looks great. Is there a place in ML where I can verify that the right time value is set to 2 weeks?

@shahzad31
Copy link
Copy Markdown
Contributor Author

@andrewvc yes you can, see the datafeed starting time in below screenshot

image

@kibanamachine
Copy link
Copy Markdown
Contributor

💛 Build succeeded, but was flaky


Test Failures

Kibana Pipeline / kibana-oss-agent / Firefox UI Functional Tests.test/functional/apps/visualize/input_control_vis/chained_controls·js.visualize app input controls chained controls should disable child control when parent control is not set

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:11:38]         └-: visualize app
[00:11:38]           └-> "before all" hook
[00:11:38]           └-> "before all" hook
[00:11:38]             │ debg Starting visualize before method
[00:11:38]             │ info [logstash_functional] Loading "mappings.json"
[00:11:38]             │ info [logstash_functional] Loading "data.json.gz"
[00:11:38]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:11:38]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [logstash-2015.09.22] creating index, cause [api], templates [], shards [1]/[0], mappings [_doc]
[00:11:38]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[logstash-2015.09.22][0]]])." previous.health="YELLOW" reason="shards started [[logstash-2015.09.22][0]]"
[00:11:38]             │ info [logstash_functional] Created index "logstash-2015.09.22"
[00:11:38]             │ debg [logstash_functional] "logstash-2015.09.22" settings {"index":{"analysis":{"analyzer":{"url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:11:38]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:11:38]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [logstash-2015.09.20] creating index, cause [api], templates [], shards [1]/[0], mappings [_doc]
[00:11:38]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[logstash-2015.09.20][0]]])." previous.health="YELLOW" reason="shards started [[logstash-2015.09.20][0]]"
[00:11:38]             │ info [logstash_functional] Created index "logstash-2015.09.20"
[00:11:38]             │ debg [logstash_functional] "logstash-2015.09.20" settings {"index":{"analysis":{"analyzer":{"url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:11:38]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:11:38]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [logstash-2015.09.21] creating index, cause [api], templates [], shards [1]/[0], mappings [_doc]
[00:11:38]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[logstash-2015.09.21][0]]])." previous.health="YELLOW" reason="shards started [[logstash-2015.09.21][0]]"
[00:11:38]             │ info [logstash_functional] Created index "logstash-2015.09.21"
[00:11:38]             │ debg [logstash_functional] "logstash-2015.09.21" settings {"index":{"analysis":{"analyzer":{"url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:11:48]             │ info progress: 13546
[00:11:48]             │ info [logstash_functional] Indexed 4633 docs into "logstash-2015.09.22"
[00:11:48]             │ info [logstash_functional] Indexed 4757 docs into "logstash-2015.09.20"
[00:11:48]             │ info [logstash_functional] Indexed 4614 docs into "logstash-2015.09.21"
[00:11:49]             │ info [long_window_logstash] Loading "mappings.json"
[00:11:49]             │ info [long_window_logstash] Loading "data.json.gz"
[00:11:49]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:11:49]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [long-window-logstash-0] creating index, cause [api], templates [], shards [1]/[0], mappings [_doc]
[00:11:49]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[long-window-logstash-0][0]]])." previous.health="YELLOW" reason="shards started [[long-window-logstash-0][0]]"
[00:11:49]             │ info [long_window_logstash] Created index "long-window-logstash-0"
[00:11:49]             │ debg [long_window_logstash] "long-window-logstash-0" settings {"index":{"analysis":{"analyzer":{"makelogs_url":{"max_token_length":"1000","tokenizer":"uax_url_email","type":"standard"}}},"number_of_replicas":"0","number_of_shards":"1"}}
[00:11:59]             │ info progress: 11740
[00:12:00]             │ info [long_window_logstash] Indexed 14005 docs into "long-window-logstash-0"
[00:12:01]             │ info [visualize] Loading "mappings.json"
[00:12:01]             │ info [visualize] Loading "data.json"
[00:12:01]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_2/7ytHDntrSl6UO8iNC4BMgw] deleting index
[00:12:01]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_1/It6fO_jbTn2VcabpFBTiDQ] deleting index
[00:12:01]             │ info [visualize] Deleted existing index [".kibana_2",".kibana_1"]
[00:12:01]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:12:01]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:12:01]             │ info [visualize] Created index ".kibana"
[00:12:01]             │ debg [visualize] ".kibana" settings {"index":{"number_of_replicas":"1","number_of_shards":"1"}}
[00:12:01]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana/W7_rPeMHSpi5iZwOvFC1Bg] update_mapping [_doc]
[00:12:01]             │ info [visualize] Indexed 12 docs into ".kibana"
[00:12:01]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana/W7_rPeMHSpi5iZwOvFC1Bg] update_mapping [_doc]
[00:12:01]             │ debg Migrating saved objects
[00:12:01]             │ proc [kibana]   log   [10:28:12.286] [info][savedobjects-service] Creating index .kibana_2.
[00:12:01]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:12:01]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_2] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:12:01]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] updating number_of_replicas to [0] for indices [.kibana_2]
[00:12:01]             │ proc [kibana]   log   [10:28:12.332] [info][savedobjects-service] Reindexing .kibana to .kibana_1
[00:12:01]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] applying create index request using v1 templates []
[00:12:01]             │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1], mappings [_doc]
[00:12:01]             │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] updating number_of_replicas to [0] for indices [.kibana_1]
[00:12:01]             │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] 9154 finished with response BulkByScrollResponse[took=25ms,timed_out=false,sliceId=null,updated=0,created=12,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:12:01]             │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana/W7_rPeMHSpi5iZwOvFC1Bg] deleting index
[00:12:01]             │ proc [kibana]   log   [10:28:12.659] [info][savedobjects-service] Migrating .kibana_1 saved objects to .kibana_2
[00:12:01]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_2/ZH-BNSAUTM6dTyoW820K0Q] update_mapping [_doc]
[00:12:01]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_2/ZH-BNSAUTM6dTyoW820K0Q] update_mapping [_doc]
[00:12:01]             │ proc [kibana]   log   [10:28:12.742] [info][savedobjects-service] Pointing alias .kibana to .kibana_2.
[00:12:01]             │ proc [kibana]   log   [10:28:12.781] [info][savedobjects-service] Finished in 497ms.
[00:12:01]             │ debg applying update to kibana config: {"accessibility:disableAnimations":true,"dateFormat:tz":"UTC"}
[00:12:02]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_2/ZH-BNSAUTM6dTyoW820K0Q] update_mapping [_doc]
[00:12:03]             │ debg replacing kibana config doc: {"defaultIndex":"logstash-*","format:bytes:defaultPattern":"0,0.[000]b"}
[00:12:03]             │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-ubuntu-16-tests-xl-1588672597224915651] [.kibana_2/ZH-BNSAUTM6dTyoW820K0Q] update_mapping [_doc]
[00:12:04]           └-: 
[00:12:04]             └-> "before all" hook
[00:12:04]             └-: input controls
[00:12:04]               └-> "before all" hook
[00:12:04]               └-: chained controls
[00:12:04]                 └-> "before all" hook
[00:12:04]                 └-> "before all" hook
[00:12:04]                   │ debg navigating to visualize url: http://localhost:6111/app/kibana#/visualize
[00:12:04]                   │ debg navigate to: http://localhost:6111/app/kibana#/visualize
[00:12:04]                   │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:12:04]                   │ debg ... sleep(700) start
[00:12:05]                   │ debg ... sleep(700) end
[00:12:05]                   │ debg returned from get, calling refresh
[00:12:06]                   │ debg browser[log] "^ A single error about an inline script not firing due to content security policy is expected!"
[00:12:07]                   │ debg currentUrl = http://localhost:6111/app/kibana#/visualize
[00:12:07]                   │          appUrl = http://localhost:6111/app/kibana#/visualize
[00:12:07]                   │ debg TestSubjects.find(kibanaChrome)
[00:12:07]                   │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=60000
[00:12:09]                   │ debg browser[info] "INFO: 2020-05-05T10:28:20Z
[00:12:09]                   │        Adding connection to http://localhost:6111/elasticsearch
[00:12:09]                   │
[00:12:09]                   │      "
[00:12:12]                   │ debg TestSubjects.find(kibanaChrome)
[00:12:12]                   │ debg Find.findByCssSelector('[data-test-subj="kibanaChrome"]') with timeout=10000
[00:12:12]                   │ debg ... sleep(501) start
[00:12:12]                   │ debg ... sleep(501) end
[00:12:12]                   │ debg in navigateTo url = http://localhost:6111/app/kibana#/visualize?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))
[00:12:12]                   │ debg --- retry.try error: URL changed, waiting for it to settle
[00:12:13]                   │ debg ... sleep(501) start
[00:12:13]                   │ debg ... sleep(501) end
[00:12:13]                   │ debg in navigateTo url = http://localhost:6111/app/kibana#/visualize?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))
[00:12:13]                   │ debg TestSubjects.exists(statusPageContainer)
[00:12:13]                   │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="statusPageContainer"]') with timeout=2500
[00:12:16]                   │ debg --- retry.tryForTime error: [data-test-subj="statusPageContainer"] is not displayed
[00:12:16]                   │ debg clickVisualizationByLinkText(chained input control)
[00:12:16]                   │ debg Find.clickByPartialLinkText('chained input control') with timeout=10000
[00:12:16]                   │ debg Find.byPartialLinkText('chained input control')  with timeout=10000
[00:12:17]                   │ debg isGlobalLoadingIndicatorVisible
[00:12:17]                   │ debg TestSubjects.exists(globalLoadingIndicator)
[00:12:17]                   │ debg Find.existsByDisplayedByCssSelector('[data-test-subj="globalLoadingIndicator"]') with timeout=1500
[00:12:17]                   │ debg TestSubjects.exists(globalLoadingIndicator-hidden)
[00:12:17]                   │ debg Find.existsByCssSelector('[data-test-subj="globalLoadingIndicator-hidden"]') with timeout=100000
[00:12:17]                 └-> should disable child control when parent control is not set
[00:12:17]                   └-> "before each" hook: global before each
[00:12:17]                   │ debg comboBox.getOptionsList, comboBoxSelector: listControlSelect0
[00:12:17]                   │ debg TestSubjects.find(listControlSelect0)
[00:12:17]                   │ debg Find.findByCssSelector('[data-test-subj="listControlSelect0"]') with timeout=10000
[00:12:18]                   │ debg TestSubjects.click(listControlSelect0)
[00:12:18]                   │ debg Find.clickByCssSelector('[data-test-subj="listControlSelect0"]') with timeout=10000
[00:12:18]                   │ debg Find.findByCssSelector('[data-test-subj="listControlSelect0"]') with timeout=10000
[00:12:20]                   │ debg TestSubjects.exists(~comboBoxOptionsList)
[00:12:20]                   │ debg Find.existsByDisplayedByCssSelector('[data-test-subj~="comboBoxOptionsList"]') with timeout=2500
[00:12:20]                   │ debg TestSubjects.find(~comboBoxOptionsList)
[00:12:20]                   │ debg Find.findByCssSelector('[data-test-subj~="comboBoxOptionsList"]') with timeout=10000
[00:12:20]                   │ warn WebElementWrapper.getVisibleText: The element reference of <div class="euiPanel euiComboBoxOptionsList euiComboBoxOptionsList--bottom"> is stale; either the element is no longer attached to the DOM, it is not in the current frame context, or the document has been refreshed
[00:12:20]                   │ debg finding element 'By(css selector, [data-test-subj~="comboBoxOptionsList"])' again, 2 attempts left
[00:12:31]                   │ info Taking screenshot "/dev/shm/workspace/kibana/test/functional/screenshots/failure/visualize app  input controls chained controls should disable child control when parent control is not set.png"
[00:12:31]                   │ info Current URL is: http://localhost:6111/app/kibana#/visualize/edit/68305470-87bc-11e9-a991-3b492a7c3e09?_g=(filters:!(),refreshInterval:(pause:!t,value:0),time:(from:now-15m,to:now))&_a=(filters:!(),linked:!f,query:(language:kuery,query:%27%27),uiState:(),vis:(aggs:!(),params:(controls:!((fieldName:geo.src,id:%271559757816862%27,indexPattern:%27logstash-*%27,label:%27%27,options:(dynamicOptions:!t,multiselect:!t,order:desc,size:5,type:terms),parent:%27%27,type:list),(fieldName:clientip,id:%271559757836347%27,indexPattern:%27logstash-*%27,label:%27%27,options:(dynamicOptions:!t,multiselect:!t,order:desc,size:5,type:terms),parent:%271559757816862%27,type:list)),pinFilters:!f,updateFiltersOnChange:!f,useTimeFilter:!f),title:%27chained%20input%20control%27,type:input_control_vis))
[00:12:31]                   │ info Saving page source to: /dev/shm/workspace/kibana/test/functional/failure_debug/html/visualize app  input controls chained controls should disable child control when parent control is not set.html
[00:12:31]                   └- ✖ fail: "visualize app  input controls chained controls should disable child control when parent control is not set"
[00:12:31]                   │

Stack Trace

{ NoSuchElementError: Unable to locate element: [data-test-subj~="comboBoxOptionsList"]
    at Object.throwDecodedError (node_modules/selenium-webdriver/lib/error.js:550:15)
    at parseHttpResponse (node_modules/selenium-webdriver/lib/http.js:565:13)
    at Executor.execute (node_modules/selenium-webdriver/lib/http.js:491:26)
    at process._tickCallback (internal/process/next_tick.js:68:7)
  name: 'NoSuchElementError',
  remoteStacktrace:
   'WebDriverError@chrome://marionette/content/error.js:175:5\nNoSuchElementError@chrome://marionette/content/error.js:387:5\nelement.find/</<@chrome://marionette/content/element.js:331:16\n' }

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Copy link
Copy Markdown
Contributor

@andrewvc andrewvc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@shahzad31 shahzad31 merged commit 62574f3 into elastic:master May 5, 2020
@shahzad31 shahzad31 deleted the ml-job-time branch May 5, 2020 12:33
shahzad31 added a commit to shahzad31/kibana that referenced this pull request May 5, 2020
gmmorris added a commit to gmmorris/kibana that referenced this pull request May 5, 2020
* master: (133 commits)
  Cleanup Typescript index pattern field editor / Expression functions for bucket agg (elastic#65254)
  Removes legacy infra plugin and moves saved objects registration to NP (elastic#64848)
  Added support for docLinks plugin in Connectors forms and missing save capabilities for modal dialog (elastic#64986)
  [SIEM] Removes prebuilt rules number dependency (elastic#65128)
  [Maps] add categorical palettes with 20 and 30 categories (elastic#64701)
  [CI] Slack alerts - Elasticsearch snapshot failures (elastic#64724)
  [Uptime] Console errors in case index missing (elastic#65115)
  [SIEM][CASE] Dynamic fields mapping based on connector (elastic#64412)
  [test/functional] Tsfy page objects (elastic#64887)
  [Maps] [Telemetry] Track geo_point and geo_shape index patterns separately (elastic#65195)
  [Maps] Add global fit to data (elastic#64702)
  Visualize: Reload on ui state change and fix ui state for tsvb (elastic#63699)
  [SIEM] [Cases] External service selection per case (elastic#64775)
  [Uptime] Set ML anomaly look-back to 2w (from 24h) / Add spinner (elastic#65055)
  [Metrics UI] Remove APM Hard Dependency (elastic#64952)
  [Ingest] Datastream list: add icons and dashboard links (elastic#65048)
  disable plugins. they could access ES via SO repository (elastic#65242)
  Feature fleet enrollment instructions (elastic#65176)
  [SIEM] Adds 'Configure connector' Cypress test (elastic#64807)
  [TSVB] Fix std deviation band mode (elastic#64413)
  ...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

release_note:skip Skip the PR/issue when compiling release notes Team:Uptime - DEPRECATED Synthetics & RUM sub-team of Application Observability v7.8.0 v8.0.0

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Uptime] Increase ML Window from 24h to 2 Weeks

4 participants