Re-enable session lifespan, idle timeout api integration tests and use unique names for the security test reports. by azasypkin · Pull Request #77746 · elastic/kibana

azasypkin · 2020-09-17T11:35:52Z

Re-enable session lifespan api integration tests.

In this PR I also updated report names for some of our tests (made them unique) as it seems CI's Test Results only picks up tests from the first config with a specific report name. Trying to figure that out with Operations Team, not sure if it's just CI UI glitch or these tests aren't run at all.

Fixes: #76223, #76239

azasypkin · 2020-09-17T13:21:33Z

Flaky Test Runner Job (42 executions): https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/834/

azasypkin · 2020-09-17T18:12:11Z

Result from https://kibana-ci.elastic.co/job/kibana+flaky-test-suite-runner/834/: 42 runs, 0 failures.

elasticmachine · 2020-09-17T18:12:25Z

Pinging @elastic/kibana-security (Team:Security)

jportner

LGTM! 🎉

kibanamachine · 2020-09-18T09:06:21Z

💛 Build succeeded, but was flaky

continuous-integration/kibana-ci/pull-request
Commit: f60ba7d
Flaky suites:
- xpack-kibana-ciGroup1

Test Failures

X-Pack APM API integration tests (basic).x-pack/test/apm_api_integration/basic/tests/metrics_charts/metrics_charts·ts.APM specs (basic) Metrics when data is loaded for opbeans-node returns metrics data CPU usage has correct series overall values

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:00:00]         └-: APM specs (basic)
[00:00:00]           └-> "before all" hook
[00:02:09]           └-: Metrics
[00:02:09]             └-> "before all" hook
[00:02:09]             └-: when data is loaded
[00:02:09]               └-> "before all" hook
[00:02:09]               └-> "before all" hook
[00:02:10]                 │ info [metrics_8.0.0] Loading "mappings.json"
[00:02:10]                 │ info [metrics_8.0.0] Loading "data.json.gz"
[00:02:10]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-16-tests-xxl-1600412581729425782] [apm-8.0.0-metric-000001] creating index, cause [api], templates [], shards [1]/[0]
[00:02:10]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-ubuntu-16-tests-xxl-1600412581729425782] current.health="GREEN" message="Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[apm-8.0.0-metric-000001][0]]])." previous.health="YELLOW" reason="shards started [[apm-8.0.0-metric-000001][0]]"
[00:02:10]                 │ info [metrics_8.0.0] Created index "apm-8.0.0-metric-000001"
[00:02:10]                 │ debg [metrics_8.0.0] "apm-8.0.0-metric-000001" settings {"index":{"codec":"best_compression","lifecycle":{"name":"apm-rollover-30-days","rollover_alias":"apm-8.0.0-metric"},"mapping":{"total_fields":{"limit":"2000"}},"max_docvalue_fields_search":"200","number_of_replicas":"0","number_of_shards":"1","priority":"100","refresh_interval":"1ms"}}
[00:02:11]                 │ info [metrics_8.0.0] Indexed 2323 docs into "apm-8.0.0-metric-000001"
[00:02:11]               └-: for opbeans-node
[00:02:11]                 └-> "before all" hook
[00:02:11]                 └-: returns metrics data
[00:02:11]                   └-> "before all" hook
[00:02:11]                   └-> "before all" hook
[00:02:11]                   └-> contains CPU usage and System memory usage chart data
[00:02:11]                     └-> "before each" hook: global before each
[00:02:11]                     └-> "before each" hook
[00:02:11]                     └- ✓ pass  (1ms) "APM specs (basic) Metrics when data is loaded for opbeans-node returns metrics data contains CPU usage and System memory usage chart data"
[00:02:11]                   └-> "after each" hook
[00:02:11]                   └-: CPU usage
[00:02:11]                     └-> "before all" hook
[00:02:11]                     └-> "before all" hook
[00:02:11]                     └-> has correct series
[00:02:11]                       └-> "before each" hook: global before each
[00:02:11]                       └-> "before each" hook
[00:02:11]                       └- ✓ pass  (1ms) "APM specs (basic) Metrics when data is loaded for opbeans-node returns metrics data CPU usage has correct series"
[00:02:11]                     └-> "after each" hook
[00:02:11]                     └-> has correct series overall values
[00:02:11]                       └-> "before each" hook: global before each
[00:02:11]                       └-> "before each" hook
[00:02:11]                       └- ✖ fail: APM specs (basic) Metrics when data is loaded for opbeans-node returns metrics data CPU usage has correct series overall values
[00:02:11]                       │       Error: expect(received).toMatchInlineSnapshot(snapshot)
[00:02:11]                       │ 
[00:02:11]                       │ Snapshot name: `when data is loaded for opbeans-node returns metrics data CPU usage has correct series overall values 1`
[00:02:11]                       │ 
[00:02:11]                       │ - Snapshot  - 1
[00:02:11]                       │ + Received  + 1
[00:02:11]                       │ 
[00:02:11]                       │   Array [
[00:02:11]                       │     0.714,
[00:02:11]                       │ -   0.38770000000000004,
[00:02:11]                       │ +   0.3877,
[00:02:11]                       │     0.75,
[00:02:11]                       │     0.2543,
[00:02:11]                       │   ]
[00:02:11]                       │       + expected - actual
[00:02:11]                       │ 
[00:02:11]                       │       -false
[00:02:11]                       │       +true
[00:02:11]                       │       
[00:02:11]                       │       at Assertion.assert (/dev/shm/workspace/parallel/22/kibana/packages/kbn-expect/expect.js:100:11)
[00:02:11]                       │       at Assertion.toMatchInline [as eql] (/dev/shm/workspace/parallel/22/kibana/packages/kbn-expect/expect.js:244:8)
[00:02:11]                       │       at Context.apply (test/apm_api_integration/basic/tests/metrics_charts/metrics_charts.ts:69:16)
[00:02:11]                       │       at Object.apply (/dev/shm/workspace/parallel/22/kibana/packages/kbn-test/src/functional_test_runner/lib/mocha/wrap_function.js:84:30)
[00:02:11]                       │ 
[00:02:11]                       │

Stack Trace

{ Error: expect(received).toMatchInlineSnapshot(snapshot)

Snapshot name: `when data is loaded for opbeans-node returns metrics data CPU usage has correct series overall values 1`

- Snapshot  - 1
+ Received  + 1

  Array [
    0.714,
-   0.38770000000000004,
+   0.3877,
    0.75,
    0.2543,
  ]
    at Assertion.assert (/dev/shm/workspace/parallel/22/kibana/packages/kbn-expect/expect.js:100:11)
    at Assertion.toMatchInline [as eql] (/dev/shm/workspace/parallel/22/kibana/packages/kbn-expect/expect.js:244:8)
    at Context.apply (test/apm_api_integration/basic/tests/metrics_charts/metrics_charts.ts:69:16)
    at Object.apply (/dev/shm/workspace/parallel/22/kibana/packages/kbn-test/src/functional_test_runner/lib/mocha/wrap_function.js:84:30) actual: 'false', expected: 'true', showDiff: true }

Build metrics

✅ unchanged

History

💚 Build #75203 succeeded 1db41a5
💚 Build #75145 succeeded 5a8622b

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

azasypkin · 2020-09-18T11:42:03Z

7.x/7.10.0: 31cb622

* master: (92 commits) [ILM] Data tiers for 7.10 (elastic#76126) [ML] Transforms: Fixes styling of preview grid pagination in summary step (elastic#77789) [Drilldowns] Beta badge support. Mark URL Drilldown as Beta (elastic#75654) Re-enable session lifespan, idle timeout api integration tests and use unique names for the security test reports. (elastic#77746) [Alerting] renames code in alerting RBAC exemption to make it easier to maintain (elastic#77598) [Alerting & Actions] Overwrite SOs when updating instead of partially updating (elastic#73688) fixed react warning in Suspense in alert flyout (elastic#77777) [APM] Track usage of Gold+ features (elastic#77630) Visualize: Bad request when working with histogram aggregation (elastic#77684) remove legacy ES plugin (elastic#77703) [Lens] change name of custom query to filters (elastic#77725) skip flaky suite (elastic#76239) remove visual aspects of baseline job (elastic#77815) skip flaky suite (elastic#77835) Fixes typo in data recognizer text (elastic#77691) management/update trusted_apps jest snapshot [build] Use Elastic hosted UBI minimal base image (elastic#77776) [APM] Add transaction error rate alert (elastic#76933) [Security Solution] [Detections] Remove file validation on import route (elastic#77770) [Enterprise Search][tech debt] Add Kea logic paths for easier debugging/defaults (elastic#77698) ...

Re-enable session lifespan api integration tests.

5a8622b

azasypkin added test Team:Security Platform Security: Auth, Users, Roles, Spaces, Audit Logging, etc t// Feature:Security/Authentication Platform Security - Authentication v8.0.0 release_note:skip Skip the PR/issue when compiling release notes v7.10.0 labels Sep 17, 2020

Use unique report names per test config.

1db41a5

azasypkin marked this pull request as ready for review September 17, 2020 18:12

azasypkin requested a review from a team as a code owner September 17, 2020 18:12

jportner approved these changes Sep 17, 2020

View reviewed changes

azasypkin added 2 commits September 18, 2020 09:00

Merge branch 'master' into issue-76223-session-cleanup-test

9b7239d

Re-enable idle timeout tests.

f60ba7d

azasypkin changed the title ~~Re-enable session lifespan api integration tests.~~ Re-enable session lifespan, idle timeout api integration tests and use unique names for the security test reports. Sep 18, 2020

azasypkin merged commit 9a0dfcf into elastic:master Sep 18, 2020

azasypkin deleted the issue-76223-session-cleanup-test branch September 18, 2020 09:08

azasypkin mentioned this pull request Sep 18, 2020

[7.x] Re-enable session lifespan, idle timeout api integration tests and use unique names for the security test reports. (#77746) #77859

Merged

azasypkin added the backported label Sep 18, 2020

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Re-enable session lifespan, idle timeout api integration tests and use unique names for the security test reports.#77746

Re-enable session lifespan, idle timeout api integration tests and use unique names for the security test reports.#77746
azasypkin merged 4 commits intoelastic:masterfrom
azasypkin:issue-76223-session-cleanup-test

azasypkin commented Sep 17, 2020 •

edited

Loading

Uh oh!

azasypkin commented Sep 17, 2020 •

edited

Loading

Uh oh!

azasypkin commented Sep 17, 2020

Uh oh!

elasticmachine commented Sep 17, 2020

Uh oh!

jportner left a comment

Uh oh!

kibanamachine commented Sep 18, 2020

Standard Out

Stack Trace

Uh oh!

azasypkin commented Sep 18, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

azasypkin commented Sep 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

azasypkin commented Sep 17, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

azasypkin commented Sep 17, 2020

Uh oh!

elasticmachine commented Sep 17, 2020

Uh oh!

jportner left a comment

Choose a reason for hiding this comment

Uh oh!

kibanamachine commented Sep 18, 2020

💛 Build succeeded, but was flaky

Test Failures

Standard Out

Stack Trace

Build metrics

History

Uh oh!

azasypkin commented Sep 18, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

azasypkin commented Sep 17, 2020 •

edited

Loading

azasypkin commented Sep 17, 2020 •

edited

Loading