Make Index Management functional and API integration tests robust against side effects introduced by Ingest Manager. by cjcenizal · Pull Request #70533 · elastic/kibana

cjcenizal · 2020-07-01T23:09:05Z

Fixes #70532

Addresses failures that were occurring in CI due to entities created by Ingest Manager:

…inst side effects introduced by Ingest Manager.

elasticmachine · 2020-07-01T23:09:07Z

Pinging @elastic/es-ui (Team:Elasticsearch UI)

spalger

LGTM, but would you mind reverting 0f418bb in this PR and running against the latest snapshots (instructions)

x-pack/test/api_integration/apis/management/index_management/templates.js

spalger

Those failures are the ones tracked by #70535, LGTM!

sebelga

LGTM! Although as I commented I would even remove the functional test that could potentially break in the future and does not bring more confidence that our component integration tests provide.

sebelga · 2020-07-02T07:09:03Z

x-pack/test/functional/apps/index_management/home_page.ts

-        expect(componentTemplateEmptyPrompt).to.be(true);
+        // Verify content. Component templates may have been created by other apps, e.g. Ingest Manager,
+        // so we don't make any assertion about the presence or absence of component templates.
+        const componentTemplateList = await testSubjects.exists('componentTemplateList');


If no components are created automatically then this test would fail, right? I am not sure that we need this test, as this use case is covered by our component integration tests (mocked empty list = empty prompt, results = template list).

This reverts commit fecd11f.

cjcenizal · 2020-07-02T15:27:20Z

Thanks @sebelga! The entire content area has that data test subject selector applied to it, so it will pass even if the list is empty and displays an empty prompt (or if it's showing the loading state or if there's an error with the request). I agree with you that it might not add any value beyond our component integration tests but I'd like to uncouple that change from the other changes in this PR. @cuff-links I'd like to hear your thoughts on this at some point.

cjcenizal · 2020-07-02T17:27:40Z

@sebelga Actually you're right, I overlooked the fact that the empty state will return early, so the selector isn't present. I'm going to fix this in another PR.

kibanamachine · 2020-07-02T17:37:56Z

💔 Build Failed

continuous-integration/kibana-ci/pull-request
Commit: 7e565aa
Pipeline Steps (look for red circles / failed steps)
Interpreting CI Failures

Failed CI Steps

Test Failures

Kibana Pipeline / kibana-xpack-agent / X-Pack Endpoint API Integration Without Ingest Tests.x-pack/test/endpoint_api_integration_no_ingest/apis/metadata·ts.Endpoint plugin test metadata api when ingest manager is not initialized "before all" hook for "metadata api should not return results"

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

[00:00:00]       │
[00:00:00]         └-: Endpoint plugin
[00:00:00]           └-> "before all" hook
[00:00:00]           └-: test metadata api when ingest manager is not initialized
[00:00:00]             └-> "before all" hook
[00:00:00]             └-> "before all" hook
[00:00:00]               │ info [endpoint/metadata/api_feature] Loading "data.json"
[00:00:00]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1593703498332428848] [.ds-metrics-endpoint.metadata-default-000001] creating index, cause [initialize_data_stream], templates [metrics], shards [1]/[1]
[00:00:00]               │ info [o.e.c.m.MetadataCreateDataStreamService] [kibana-ci-immutable-ubuntu-18-tests-xl-1593703498332428848] adding data stream [metrics-endpoint.metadata-default]
[00:00:00]               │ info [o.e.x.i.IndexLifecycleTransition] [kibana-ci-immutable-ubuntu-18-tests-xl-1593703498332428848] moving index [.ds-metrics-endpoint.metadata-default-000001] from [null] to [{"phase":"new","action":"complete","name":"complete"}] in policy [metrics]
[00:00:00]               │ info [o.e.x.i.IndexLifecycleTransition] [kibana-ci-immutable-ubuntu-18-tests-xl-1593703498332428848] moving index [.ds-metrics-endpoint.metadata-default-000001] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"hot","action":"unfollow","name":"wait-for-indexing-complete"}] in policy [metrics]
[00:00:00]               └- ✖ fail: "Endpoint plugin test metadata api when ingest manager is not initialized "before all" hook for "metadata api should not return results""
[00:00:00]               │

Stack Trace

{ Error: [illegal_argument_exception] The provided expression [metrics-endpoint.metadata-default] matches a data stream, specify the corresponding concrete indices instead.
    at respond (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:349:15)
    at checkRespForFailure (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:306:7)
    at HttpConnector.<anonymous> (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
    at IncomingMessage.wrapper (/dev/shm/workspace/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4929:19)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  status: 400,
  displayName: 'BadRequest',
  message:
   '[illegal_argument_exception] The provided expression [metrics-endpoint.metadata-default] matches a data stream, specify the corresponding concrete indices instead.',
  path: '/_bulk',
  query: {},
  body:
   { error:
      { root_cause: [Array],
        type: 'illegal_argument_exception',
        reason:
         'The provided expression [metrics-endpoint.metadata-default] matches a data stream, specify the corresponding concrete indices instead.' },
     status: 400 },
  statusCode: 400,
  response:
   '{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"The provided expression [metrics-endpoint.metadata-default] matches a data stream, specify the corresponding concrete indices instead."}],"type":"illegal_argument_exception","reason":"The provided expression [metrics-endpoint.metadata-default] matches a data stream, specify the corresponding concrete indices instead."},"status":400}',
  toString: [Function],
  toJSON: [Function] }

Kibana Pipeline / kibana-xpack-agent / X-Pack Endpoint API Integration Without Ingest Tests.x-pack/test/endpoint_api_integration_no_ingest/apis/metadata·ts.Endpoint plugin test metadata api when ingest manager is not initialized "before all" hook for "metadata api should not return results"

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 1 times on tracked branches: https://dryrun

[00:00:00]       │
[00:00:00]         └-: Endpoint plugin
[00:00:00]           └-> "before all" hook
[00:00:00]           └-: test metadata api when ingest manager is not initialized
[00:00:00]             └-> "before all" hook
[00:00:00]             └-> "before all" hook
[00:00:00]               │ info [endpoint/metadata/api_feature] Loading "data.json"
[00:00:00]               │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-ubuntu-18-tests-xl-1593703498332428848] [.ds-metrics-endpoint.metadata-default-000001] creating index, cause [initialize_data_stream], templates [metrics], shards [1]/[1]
[00:00:00]               │ info [o.e.c.m.MetadataCreateDataStreamService] [kibana-ci-immutable-ubuntu-18-tests-xl-1593703498332428848] adding data stream [metrics-endpoint.metadata-default]
[00:00:00]               │ info [o.e.x.i.IndexLifecycleTransition] [kibana-ci-immutable-ubuntu-18-tests-xl-1593703498332428848] moving index [.ds-metrics-endpoint.metadata-default-000001] from [null] to [{"phase":"new","action":"complete","name":"complete"}] in policy [metrics]
[00:00:00]               │ info [o.e.x.i.IndexLifecycleTransition] [kibana-ci-immutable-ubuntu-18-tests-xl-1593703498332428848] moving index [.ds-metrics-endpoint.metadata-default-000001] from [{"phase":"new","action":"complete","name":"complete"}] to [{"phase":"hot","action":"unfollow","name":"wait-for-indexing-complete"}] in policy [metrics]
[00:00:00]               └- ✖ fail: "Endpoint plugin test metadata api when ingest manager is not initialized "before all" hook for "metadata api should not return results""
[00:00:00]               │

Stack Trace

{ Error: [illegal_argument_exception] The provided expression [metrics-endpoint.metadata-default] matches a data stream, specify the corresponding concrete indices instead.
    at respond (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:349:15)
    at checkRespForFailure (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/transport.js:306:7)
    at HttpConnector.<anonymous> (/dev/shm/workspace/kibana/node_modules/elasticsearch/src/lib/connectors/http.js:173:7)
    at IncomingMessage.wrapper (/dev/shm/workspace/kibana/node_modules/elasticsearch/node_modules/lodash/lodash.js:4929:19)
    at endReadableNT (_stream_readable.js:1145:12)
    at process._tickCallback (internal/process/next_tick.js:63:19)
  status: 400,
  displayName: 'BadRequest',
  message:
   '[illegal_argument_exception] The provided expression [metrics-endpoint.metadata-default] matches a data stream, specify the corresponding concrete indices instead.',
  path: '/_bulk',
  query: {},
  body:
   { error:
      { root_cause: [Array],
        type: 'illegal_argument_exception',
        reason:
         'The provided expression [metrics-endpoint.metadata-default] matches a data stream, specify the corresponding concrete indices instead.' },
     status: 400 },
  statusCode: 400,
  response:
   '{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"The provided expression [metrics-endpoint.metadata-default] matches a data stream, specify the corresponding concrete indices instead."}],"type":"illegal_argument_exception","reason":"The provided expression [metrics-endpoint.metadata-default] matches a data stream, specify the corresponding concrete indices instead."},"status":400}',
  toString: [Function],
  toJSON: [Function] }

Kibana Pipeline / x-pack-intake-agent / X-Pack Jest Tests.x-pack/plugins/security_solution/public/timelines/store/timeline.epicLocalStorage persist adding / reordering of a column correctly

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has not failed recently on tracked branches

Stack Trace

Error: expect(jest.fn()).toHaveBeenCalled()

Expected number of calls: >= 1
Received number of calls:    0
    at Object.it (/dev/shm/workspace/kibana/x-pack/plugins/security_solution/public/timelines/store/timeline/epic_local_storage.test.tsx:126:38)

Build metrics

✅ unchanged

History

💔 Build #58097 failed fecd11f

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

…st against side effects introduced by Ingest Manager. (#70533) (#70612) * Make Index Management functional and API integration tests robust against side effects introduced by Ingest Manager. (#70533) * Update component templates list to render empty prompt inside of content container. Show detail panel when deep-linked, even if there are no component templates. (#70633) Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com> Co-authored-by: Elastic Machine <elasticmachine@users.noreply.github.com>

Make Index Management functional and API integration tests robust aga…

d63872b

…inst side effects introduced by Ingest Manager.

cjcenizal added chore Feature:Index Management Index and index templates UI v8.0.0 Team:Kibana Management Dev Tools, Index Management, Upgrade Assistant, ILM, Ingest Node Pipelines, and more t// release_note:skip Skip the PR/issue when compiling release notes v7.9.0 labels Jul 1, 2020

cjcenizal requested review from sebelga and spalger July 1, 2020 23:09

cjcenizal added 2 commits July 1, 2020 16:10

Fix wording.

769d55d

Merge branch 'master' into bug/index-management-functional-tests

3a1b467

spalger reviewed Jul 1, 2020

View reviewed changes

x-pack/test/api_integration/apis/management/index_management/templates.js Outdated Show resolved Hide resolved

cjcenizal added 3 commits July 1, 2020 16:13

Unskip tests.

0f19f12

Better error message.

6f064c1

Temporarily change ES snapshot.

fecd11f

spalger approved these changes Jul 2, 2020

View reviewed changes

sebelga approved these changes Jul 2, 2020

View reviewed changes

Revert "Temporarily change ES snapshot."

7e565aa

This reverts commit fecd11f.

cjcenizal merged commit 59ece79 into elastic:master Jul 2, 2020

cjcenizal deleted the bug/index-management-functional-tests branch July 2, 2020 15:24

cjcenizal mentioned this pull request Jul 2, 2020

[7.x] Make Index Management functional and API integration tests robust against side effects introduced by Ingest Manager. (#70533) #70612

Merged

cjcenizal mentioned this pull request Jul 2, 2020

Update component templates list to render empty prompt inside of content container #70633

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Make Index Management functional and API integration tests robust against side effects introduced by Ingest Manager.#70533

Make Index Management functional and API integration tests robust against side effects introduced by Ingest Manager.#70533
cjcenizal merged 7 commits intoelastic:masterfrom
cjcenizal:bug/index-management-functional-tests

cjcenizal commented Jul 1, 2020 •

edited

Loading

Uh oh!

elasticmachine commented Jul 1, 2020

Uh oh!

spalger left a comment

Uh oh!

Uh oh!

spalger left a comment

Uh oh!

sebelga left a comment

Uh oh!

sebelga Jul 2, 2020

Uh oh!

cjcenizal commented Jul 2, 2020

Uh oh!

cjcenizal commented Jul 2, 2020

Uh oh!

kibanamachine commented Jul 2, 2020

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Conversation

cjcenizal commented Jul 1, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

elasticmachine commented Jul 1, 2020

Uh oh!

spalger left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

spalger left a comment

Choose a reason for hiding this comment

Uh oh!

sebelga left a comment

Choose a reason for hiding this comment

Uh oh!

sebelga Jul 2, 2020

Choose a reason for hiding this comment

Uh oh!

cjcenizal commented Jul 2, 2020

Uh oh!

cjcenizal commented Jul 2, 2020

Uh oh!

kibanamachine commented Jul 2, 2020

💔 Build Failed

Failed CI Steps

Test Failures

Standard Out

Stack Trace

Standard Out

Stack Trace

Standard Out

Stack Trace

Build metrics

History

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

cjcenizal commented Jul 1, 2020 •

edited

Loading