Skip to content

[7.12] Discover: Add handling for source column (#91815), enable fields API by default, Fix context view for date_nanos format with custom timestamps (#54089)#91981

Merged
flash1293 merged 7 commits intoelastic:7.12from
flash1293:backport/7.12/pr-91815
Feb 19, 2021

Conversation

@flash1293
Copy link
Copy Markdown
Contributor

@flash1293 flash1293 commented Feb 19, 2021

Backports the following commits to 7.12:

@flash1293 flash1293 added the backport This PR is a backport of another PR label Feb 19, 2021
@flash1293 flash1293 enabled auto-merge (squash) February 19, 2021 08:38
@flash1293
Copy link
Copy Markdown
Contributor Author

This PR failed because fields API isn't default yet on 7.12 - I'm going to change that on a separate PR, then update this one

@flash1293 flash1293 merged commit 0b37d8c into elastic:7.12 Feb 19, 2021
@flash1293 flash1293 changed the title [7.12] Discover: Add handling for source column (#91815) [7.12] Discover: Add handling for source column (#91815), enable fields API by default, Fix context view for date_nanos format with custom timestamps (#54089) Feb 22, 2021
@kibanamachine
Copy link
Copy Markdown
Contributor

💛 Build succeeded, but was flaky


Test Failures

Kibana Pipeline / general / APM API Integration tests (trial).x-pack/test/apm_api_integration/tests/correlations/slow_transactions·ts.APM API tests trial apm_8.0.0 with data and default args returns significant terms

Link to Jenkins

Standard Out

Failed Tests Reporter:
  - Test has failed 4 times on tracked branches: https://github.com/elastic/kibana/issues/91673

[00:00:00]       │
[00:00:00]         │ info [o.e.x.i.IndexLifecycleTransition] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] moving index [.ds-ilm-history-5-2021.02.22-000001] from [{"phase":"hot","action":"unfollow","name":"branch-check-unfollow-prerequisites"}] to [{"phase":"hot","action":"rollover","name":"check-rollover-ready"}] in policy [ilm-history-ilm-policy]
[00:00:00]         └-: APM API tests
[00:00:00]           └-> "before all" hook in "APM API tests"
[00:00:00]           └-: trial
[00:00:00]             └-> "before all" hook in "trial"
[00:00:30]             └-: apm_8.0.0
[00:00:30]               └-> "before all" hook in "apm_8.0.0"
[00:00:30]               └-> "before all" hook: runBefore in "apm_8.0.0"
[00:00:30]                 │ info (0ms) Loading apm_8.0.0
[00:00:30]                 │ info [apm_8.0.0] Loading "mappings.json"
[00:00:30]                 │ info [apm_8.0.0] Loading "data.json.gz"
[00:00:30]                 │ info [o.e.x.m.a.TransportDeleteJobAction] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [apm-production-4da8-high_mean_transaction_duration] running delete by query on [.ml-anomalies-shared]
[00:00:30]                 │ info [o.e.x.m.a.TransportDeleteJobAction] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [apm-staging-56d9-high_mean_transaction_duration] running delete by query on [.ml-anomalies-shared]
[00:00:30]                 │ info [o.e.x.m.a.TransportDeleteJobAction] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] Job [apm-staging-56d9-high_mean_transaction_duration] deleted
[00:00:30]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_001/80NIJcOZQUWlLvCWZq2_tQ] deleting index
[00:00:30]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_task_manager_7.12.0_001/FVIESV0VROq_0r-4z_rbvw] deleting index
[00:00:30]                 │ info [o.e.x.m.a.TransportDeleteJobAction] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] Job [apm-production-4da8-high_mean_transaction_duration] deleted
[00:00:30]                 │ info [apm_8.0.0] Deleted existing index ".kibana_task_manager_7.12.0_001"
[00:00:30]                 │ info [apm_8.0.0] Deleted existing index ".kibana_7.12.0_001"
[00:00:30]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.tasks] creating index, cause [auto(bulk api)], templates [], shards [1]/[1]
[00:00:30]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.tasks]
[00:00:30]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_1] creating index, cause [api], templates [], shards [1]/[1]
[00:00:30]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.kibana_1]
[00:00:31]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.kibana_1][0]]]).
[00:00:31]                 │ info [apm_8.0.0] Created index ".kibana_1"
[00:00:31]                 │ debg [apm_8.0.0] ".kibana_1" settings {"index":{"auto_expand_replicas":"0-1","number_of_replicas":"1","number_of_shards":"1"}}
[00:00:31]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.ml-anomalies-shared/Omv3VZ36RdW2LfsoPUs8kQ] deleting index
[00:00:31]                 │ info [apm_8.0.0] Deleted existing index ".ml-anomalies-shared"
[00:00:31]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.ml-anomalies-shared] creating index, cause [api], templates [.ml-anomalies-], shards [1]/[1]
[00:00:31]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.ml-anomalies-shared]
[00:00:31]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.ml-anomalies-shared][0]]]).
[00:00:31]                 │ info [apm_8.0.0] Created index ".ml-anomalies-shared"
[00:00:31]                 │ debg [apm_8.0.0] ".ml-anomalies-shared" settings {"index":{"auto_expand_replicas":"0-1","hidden":"true","number_of_replicas":"1","number_of_shards":"1","translog":{"durability":"async"}}}
[00:00:31]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.ml-config/Vl6dXoiuTOea87urN2K_og] deleting index
[00:00:31]                 │ info [apm_8.0.0] Deleted existing index ".ml-config"
[00:00:31]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.ml-config] creating index, cause [api], templates [], shards [1]/[1]
[00:00:31]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.ml-config]
[00:00:31]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] Cluster health status changed from [YELLOW] to [GREEN] (reason: [shards started [[.ml-config][0]]]).
[00:00:31]                 │ info [apm_8.0.0] Created index ".ml-config"
[00:00:31]                 │ debg [apm_8.0.0] ".ml-config" settings {"index":{"auto_expand_replicas":"0-1","max_result_window":"10000","number_of_replicas":"1","number_of_shards":"1"}}
[00:00:31]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [apm-8.0.0-error-2020.12.03-000001] creating index, cause [api], templates [], shards [1]/[1]
[00:00:31]                 │ info [apm_8.0.0] Created index "apm-8.0.0-error-2020.12.03-000001"
[00:00:31]                 │ debg [apm_8.0.0] "apm-8.0.0-error-2020.12.03-000001" settings {"index":{"codec":"best_compression","lifecycle":{"name":"apm-rollover-30-days","rollover_alias":"apm-8.0.0-error"},"mapping":{"total_fields":{"limit":"2000"}},"max_docvalue_fields_search":"200","number_of_replicas":"1","number_of_shards":"1","priority":"100","refresh_interval":"5s"}}
[00:00:31]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [apm-8.0.0-metric-2020.12.03-000001] creating index, cause [api], templates [], shards [1]/[1]
[00:00:31]                 │ info [apm_8.0.0] Created index "apm-8.0.0-metric-2020.12.03-000001"
[00:00:31]                 │ debg [apm_8.0.0] "apm-8.0.0-metric-2020.12.03-000001" settings {"index":{"codec":"best_compression","lifecycle":{"name":"apm-rollover-30-days","rollover_alias":"apm-8.0.0-metric"},"mapping":{"total_fields":{"limit":"2000"}},"max_docvalue_fields_search":"200","number_of_replicas":"1","number_of_shards":"1","priority":"100","refresh_interval":"5s"}}
[00:00:31]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [apm-8.0.0-span-2020.12.03-000001] creating index, cause [api], templates [], shards [1]/[1]
[00:00:31]                 │ info [apm_8.0.0] Created index "apm-8.0.0-span-2020.12.03-000001"
[00:00:31]                 │ debg [apm_8.0.0] "apm-8.0.0-span-2020.12.03-000001" settings {"index":{"codec":"best_compression","lifecycle":{"name":"apm-rollover-30-days","rollover_alias":"apm-8.0.0-span"},"mapping":{"total_fields":{"limit":"2000"}},"max_docvalue_fields_search":"200","number_of_replicas":"1","number_of_shards":"1","priority":"100","refresh_interval":"5s"}}
[00:00:31]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [apm-8.0.0-transaction-2020.12.03-000001] creating index, cause [api], templates [], shards [1]/[1]
[00:00:31]                 │ info [apm_8.0.0] Created index "apm-8.0.0-transaction-2020.12.03-000001"
[00:00:31]                 │ debg [apm_8.0.0] "apm-8.0.0-transaction-2020.12.03-000001" settings {"index":{"codec":"best_compression","lifecycle":{"name":"apm-rollover-30-days","rollover_alias":"apm-8.0.0-transaction"},"mapping":{"total_fields":{"limit":"2000"}},"max_docvalue_fields_search":"200","number_of_replicas":"1","number_of_shards":"1","priority":"100","refresh_interval":"5s"}}
[00:00:40]                 │ info [apm_8.0.0] Indexed 20 docs into ".kibana_1"
[00:00:40]                 │ info [apm_8.0.0] Indexed 114 docs into ".ml-anomalies-shared"
[00:00:40]                 │ info [apm_8.0.0] Indexed 4 docs into ".ml-config"
[00:00:40]                 │ info [apm_8.0.0] Indexed 207 docs into "apm-8.0.0-error-2020.12.03-000001"
[00:00:40]                 │ info [apm_8.0.0] Indexed 14970 docs into "apm-8.0.0-metric-2020.12.03-000001"
[00:00:40]                 │ info [apm_8.0.0] Indexed 12781 docs into "apm-8.0.0-span-2020.12.03-000001"
[00:00:40]                 │ info [apm_8.0.0] Indexed 4786 docs into "apm-8.0.0-transaction-2020.12.03-000001"
[00:00:40]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_1/m8sE-aPNSoiwVrQpc2hIfA] update_mapping [_doc]
[00:00:40]                 │ debg Migrating saved objects
[00:00:40]                 │ proc [kibana]   log   [10:11:55.829] [info][savedobjects-service] [.kibana_task_manager] INIT -> CREATE_NEW_TARGET
[00:00:40]                 │ proc [kibana]   log   [10:11:55.837] [info][savedobjects-service] [.kibana] INIT -> SET_SOURCE_WRITE_BLOCK
[00:00:40]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_task_manager_7.12.0_001] creating index, cause [api], templates [], shards [1]/[1]
[00:00:40]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.kibana_task_manager_7.12.0_001]
[00:00:40]                 │ info [o.e.c.m.MetadataIndexStateService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] adding block write to indices [[.kibana_1/m8sE-aPNSoiwVrQpc2hIfA]]
[00:00:40]                 │ info [o.e.c.m.MetadataIndexStateService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] completed adding block write to indices [.kibana_1]
[00:00:40]                 │ proc [kibana]   log   [10:11:55.905] [info][savedobjects-service] [.kibana_task_manager] CREATE_NEW_TARGET -> MARK_VERSION_INDEX_READY
[00:00:40]                 │ proc [kibana]   log   [10:11:55.922] [info][savedobjects-service] [.kibana] SET_SOURCE_WRITE_BLOCK -> CREATE_REINDEX_TEMP
[00:00:40]                 │ proc [kibana]   log   [10:11:55.939] [info][savedobjects-service] [.kibana_task_manager] MARK_VERSION_INDEX_READY -> DONE
[00:00:40]                 │ proc [kibana]   log   [10:11:55.940] [info][savedobjects-service] [.kibana_task_manager] Migration completed after 113ms
[00:00:40]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_reindex_temp] creating index, cause [api], templates [], shards [1]/[1]
[00:00:40]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.kibana_7.12.0_reindex_temp]
[00:00:40]                 │ proc [kibana]   log   [10:11:55.990] [info][savedobjects-service] [.kibana] CREATE_REINDEX_TEMP -> REINDEX_SOURCE_TO_TEMP
[00:00:40]                 │ proc [kibana]   log   [10:11:56.004] [info][savedobjects-service] [.kibana] REINDEX_SOURCE_TO_TEMP -> REINDEX_SOURCE_TO_TEMP_WAIT_FOR_TASK
[00:00:40]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_reindex_temp/B2CwJYQpQ7WjMNBmMSFm-g] update_mapping [_doc]
[00:00:40]                 │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] 5084 finished with response BulkByScrollResponse[took=39.6ms,timed_out=false,sliceId=null,updated=0,created=20,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:00:40]                 │ proc [kibana]   log   [10:11:56.119] [info][savedobjects-service] [.kibana] REINDEX_SOURCE_TO_TEMP_WAIT_FOR_TASK -> SET_TEMP_WRITE_BLOCK
[00:00:40]                 │ info [o.e.c.m.MetadataIndexStateService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] adding block write to indices [[.kibana_7.12.0_reindex_temp/B2CwJYQpQ7WjMNBmMSFm-g]]
[00:00:40]                 │ info [o.e.c.m.MetadataIndexStateService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] completed adding block write to indices [.kibana_7.12.0_reindex_temp]
[00:00:40]                 │ proc [kibana]   log   [10:11:56.155] [info][savedobjects-service] [.kibana] SET_TEMP_WRITE_BLOCK -> CLONE_TEMP_TO_TARGET
[00:00:40]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] applying create index request using existing index [.kibana_7.12.0_reindex_temp] metadata
[00:00:40]                 │ info [o.e.c.m.MetadataCreateIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_001] creating index, cause [clone_index], templates [], shards [1]/[1]
[00:00:40]                 │ info [o.e.c.r.a.AllocationService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] updating number_of_replicas to [0] for indices [.kibana_7.12.0_001]
[00:00:40]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_001/tCfL7v7VQb2iDmJJEUQV1g] create_mapping [_doc]
[00:00:40]                 │ proc [kibana]   log   [10:11:56.268] [info][savedobjects-service] [.kibana] CLONE_TEMP_TO_TARGET -> OUTDATED_DOCUMENTS_SEARCH
[00:00:40]                 │ proc [kibana]   log   [10:11:56.278] [info][savedobjects-service] [.kibana] OUTDATED_DOCUMENTS_SEARCH -> UPDATE_TARGET_MAPPINGS
[00:00:40]                 │ info [o.e.c.m.MetadataMappingService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_001/tCfL7v7VQb2iDmJJEUQV1g] update_mapping [_doc]
[00:00:40]                 │ proc [kibana]   log   [10:11:56.335] [info][savedobjects-service] [.kibana] UPDATE_TARGET_MAPPINGS -> UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK
[00:00:40]                 │ info [o.e.t.LoggingTaskListener] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] 5133 finished with response BulkByScrollResponse[took=19.6ms,timed_out=false,sliceId=null,updated=20,created=0,deleted=0,batches=1,versionConflicts=0,noops=0,retries=0,throttledUntil=0s,bulk_failures=[],search_failures=[]]
[00:00:40]                 │ proc [kibana]   log   [10:11:56.442] [info][savedobjects-service] [.kibana] UPDATE_TARGET_MAPPINGS_WAIT_FOR_TASK -> MARK_VERSION_INDEX_READY
[00:00:40]                 │ info [o.e.c.m.MetadataDeleteIndexService] [kibana-ci-immutable-centos-tests-xxl-1613984638972749827] [.kibana_7.12.0_reindex_temp/B2CwJYQpQ7WjMNBmMSFm-g] deleting index
[00:00:40]                 │ proc [kibana]   log   [10:11:56.484] [info][savedobjects-service] [.kibana] MARK_VERSION_INDEX_READY -> DONE
[00:00:40]                 │ proc [kibana]   log   [10:11:56.484] [info][savedobjects-service] [.kibana] Migration completed after 659ms
[00:00:40]                 │ debg [apm_8.0.0] Migrated Kibana index after loading Kibana data
[00:00:41]                 │ debg [apm_8.0.0] Ensured that default space exists in .kibana
[00:00:41]                 │ info (10217ms) Loaded all archives
[00:00:41]               └-: with data and default args
[00:00:41]                 └-> "before all" hook for "returns successfully"
[00:00:41]                 └-> "before all" hook for "returns successfully"
[00:00:41]                 └-> returns successfully
[00:00:41]                   └-> "before each" hook: global before each for "returns successfully"
[00:00:41]                   └- ✓ pass  (1ms) "APM API tests trial apm_8.0.0 with data and default args returns successfully"
[00:00:41]                 └-> returns significant terms
[00:00:41]                   └-> "before each" hook: global before each for "returns significant terms"
[00:00:41]                   └- ✖ fail: APM API tests trial apm_8.0.0 with data and default args returns significant terms
[00:00:41]                   │      Error: expect(received).toMatchInlineSnapshot(snapshot)
[00:00:41]                   │ 
[00:00:41]                   │ Snapshot name: `APM API tests trial apm_8.0.0 with data and default args returns significant terms 1`
[00:00:41]                   │ 
[00:00:41]                   │ - Snapshot  - 1
[00:00:41]                   │ + Received  + 1
[00:00:41]                   │ 
[00:00:41]                   │ @@ -1,11 +1,11 @@
[00:00:41]                   │   Array [
[00:00:41]                   │     "user_agent.name",
[00:00:41]                   │ -   "url.domain",
[00:00:41]                   │     "host.ip",
[00:00:41]                   │     "service.node.name",
[00:00:41]                   │     "container.id",
[00:00:41]                   │ +   "url.domain",
[00:00:41]                   │     "url.domain",
[00:00:41]                   │     "host.ip",
[00:00:41]                   │     "service.node.name",
[00:00:41]                   │     "container.id",
[00:00:41]                   │     "user_agent.name",
[00:00:41]                   │       at toMatchInline (/dev/shm/workspace/parallel/21/kibana/packages/kbn-test/src/functional_test_runner/lib/snapshots/decorate_snapshot_ui.ts:226:11)
[00:00:41]                   │       at Context.apply (test/apm_api_integration/tests/correlations/slow_transactions.ts:57:61)
[00:00:41]                   │       at Object.apply (/dev/shm/workspace/parallel/21/kibana/packages/kbn-test/src/functional_test_runner/lib/mocha/wrap_function.js:73:30)
[00:00:41]                   │ 
[00:00:41]                   │ 

Stack Trace

Error: expect(received).toMatchInlineSnapshot(snapshot)

Snapshot name: `APM API tests trial apm_8.0.0 with data and default args returns significant terms 1`

- Snapshot  - 1
+ Received  + 1

@@ -1,11 +1,11 @@
  Array [
    "user_agent.name",
-   "url.domain",
    "host.ip",
    "service.node.name",
    "container.id",
+   "url.domain",
    "url.domain",
    "host.ip",
    "service.node.name",
    "container.id",
    "user_agent.name",
    at toMatchInline (/dev/shm/workspace/parallel/21/kibana/packages/kbn-test/src/functional_test_runner/lib/snapshots/decorate_snapshot_ui.ts:226:11)
    at Context.apply (test/apm_api_integration/tests/correlations/slow_transactions.ts:57:61)
    at Object.apply (/dev/shm/workspace/parallel/21/kibana/packages/kbn-test/src/functional_test_runner/lib/mocha/wrap_function.js:73:30)

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id before after diff
discover 348 349 +1

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id before after diff
discover 408.7KB 409.5KB +745.0B

History

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport This PR is a backport of another PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants