Skip to content

Commit 3548954

Browse files
pgayvalletpgayvallet
committed
Merge remote-tracking branch 'upstream/master' into kbn-17298-kibana-banners
2 parents d289b6e + fa18be9 commit 3548954

101 files changed

Lines changed: 1681 additions & 718 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/CODEOWNERS‎

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -223,9 +223,8 @@
223223
/x-pack/plugins/telemetry_collection_xpack/ @elastic/kibana-core
224224
/.telemetryrc.json @elastic/kibana-core
225225
/x-pack/.telemetryrc.json @elastic/kibana-core
226-
src/plugins/telemetry/schema/legacy_oss_plugins.json @elastic/kibana-core
227-
src/plugins/telemetry/schema/oss_plugins.json @elastic/kibana-core
228-
x-pack/plugins/telemetry_collection_xpack/schema/xpack_plugins.json @elastic/kibana-core
226+
src/plugins/telemetry/schema/ @elastic/kibana-core @elastic/kibana-telemetry @elastic/infra-telemetry
227+
x-pack/plugins/telemetry_collection_xpack/schema/ @elastic/kibana-core @elastic/kibana-telemetry @elastic/infra-telemetry
229228

230229
# Kibana Localization
231230
/src/dev/i18n/ @elastic/kibana-localization @elastic/kibana-core

‎rfcs/text/0013_saved_object_migrations.md‎

Lines changed: 26 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -248,45 +248,49 @@ Note:
248248
6. Use the reindexed legacy `.kibana_pre6.5.0_001` as the source for the rest of the migration algorithm.
249249
3. If `.kibana` and `.kibana_7.10.0` both exists and are pointing to the same index this version's migration has already been completed.
250250
1. Because the same version can have plugins enabled at any point in time,
251-
perform the mappings update in step (7) and migrate outdated documents
252-
with step (8).
253-
2. Skip to step (10) to start serving traffic.
251+
migrate outdated documents with step (9) and perform the mappings update in step (10).
252+
2. Skip to step (12) to start serving traffic.
254253
4. Fail the migration if:
255254
1. `.kibana` is pointing to an index that belongs to a later version of Kibana .e.g. `.kibana_7.12.0_001`
256255
2. (Only in 8.x) The source index contains documents that belong to an unknown Saved Object type (from a disabled plugin). Log an error explaining that the plugin that created these documents needs to be enabled again or that these objects should be deleted. See section (4.2.1.4).
257-
5. Mark the source index as read-only and wait for all in-flight operations to drain (requires https://github.com/elastic/elasticsearch/pull/58094). This prevents any further writes from outdated nodes. Assuming this API is similar to the existing `/<index>/_close` API, we expect to receive `"acknowledged" : true` and `"shards_acknowledged" : true`. If all shards don’t acknowledge within the timeout, retry the operation until it succeeds.
258-
6. Clone the source index into a new target index which has writes enabled. All nodes on the same version will use the same fixed index name e.g. `.kibana_7.10.0_001`. The `001` postfix isn't used by Kibana, but allows for re-indexing an index should this be required by an Elasticsearch upgrade. E.g. re-index `.kibana_7.10.0_001` into `.kibana_7.10.0_002` and point the `.kibana_7.10.0` alias to `.kibana_7.10.0_002`.
259-
1. `POST /.kibana_n/_clone/.kibana_7.10.0_001?wait_for_active_shards=all {"settings": {"index.blocks.write": false}}`. Ignore errors if the clone already exists.
260-
2. Wait for the cloning to complete `GET /_cluster/health/.kibana_7.10.0_001?wait_for_status=green&timeout=60s` If cloning doesn’t complete within the 60s timeout, log a warning for visibility and poll again.
261-
7. Update the mappings of the target index
256+
5. Set a write block on the source index. This prevents any further writes from outdated nodes.
257+
6. Create a new temporary index `.kibana_7.10.0_reindex_temp` with `dynamic: false` on the top-level mappings so that any kind of document can be written to the index. This allows us to write untransformed documents to the index which might have fields which have been removed from the latest mappings defined by the plugin. Define minimal mappings for the `migrationVersion` and `type` fields so that we're still able to search for outdated documents that need to be transformed.
258+
1. Ignore errors if the target index already exists.
259+
7. Reindex the source index into the new temporary index.
260+
1. Use `op_type=create` `conflicts=proceed` and `wait_for_completion=false` so that multiple instances can perform the reindex in parallel but only one write per document will succeed.
261+
2. Wait for the reindex task to complete. If reindexing doesn’t complete within the 60s timeout, log a warning for visibility and poll again.
262+
8. Clone the temporary index into the target index `.kibana_7.10.0_001`. Since any further writes will only happen against the cloned target index this prevents a lost delete from occuring where one instance finishes the migration and deletes a document and another instance's reindex operation re-creates the deleted document.
263+
1. Set a write block on the temporary index
264+
2. Clone the temporary index into the target index while specifying that the target index should have writes enabled.
265+
3. If the clone operation fails because the target index already exist, ignore the error and wait for the target index to become green before proceeding.
266+
4. (The `001` postfix in the target index name isn't used by Kibana, but allows for re-indexing an index should this be required by an Elasticsearch upgrade. E.g. re-index `.kibana_7.10.0_001` into `.kibana_7.10.0_002` and point the `.kibana_7.10.0` alias to `.kibana_7.10.0_002`.)
267+
9. Transform documents by reading batches of outdated documents from the target index then transforming and updating them with optimistic concurrency control.
268+
1. Ignore any version conflict errors.
269+
2. If a document transform throws an exception, add the document to a failure list and continue trying to transform all other documents. If any failures occured, log the complete list of documents that failed to transform. Fail the migration.
270+
10. Update the mappings of the target index
262271
1. Retrieve the existing mappings including the `migrationMappingPropertyHashes` metadata.
263-
2. Update the mappings with `PUT /.kibana_7.10.0_001/_mapping`. The API deeply merges any updates so this won't remove the mappings of any plugins that were enabled in a previous version but are now disabled.
272+
2. Update the mappings with `PUT /.kibana_7.10.0_001/_mapping`. The API deeply merges any updates so this won't remove the mappings of any plugins that are disabled on this instance but have been enabled on another instance that also migrated this index.
264273
3. Ensure that fields are correctly indexed using the target index's latest mappings `POST /.kibana_7.10.0_001/_update_by_query?conflicts=proceed`. In the future we could optimize this query by only targeting documents:
265274
1. That belong to a known saved object type.
266-
2. Which don't have outdated migrationVersion numbers since these will be transformed anyway.
267-
3. That belong to a type whose mappings were changed by comparing the `migrationMappingPropertyHashes`. (Metadata, unlike the mappings isn't commutative, so there is a small chance that the metadata hashes do not accurately reflect the latest mappings, however, this will just result in an less efficient query).
268-
8. Transform documents by reading batches of outdated documents from the target index then transforming and updating them with optimistic concurrency control.
269-
1. Ignore any version conflict errors.
270-
2. If a document transform throws an exception, add the document to a failure list and continue trying to transform all other documents. If any failures occured, log the complete list of documents that failed to transform. Fail the migration.
271-
9. Mark the migration as complete. This is done as a single atomic
275+
11. Mark the migration as complete. This is done as a single atomic
272276
operation (requires https://github.com/elastic/elasticsearch/pull/58100)
273-
to guarantees when multiple versions of Kibana are performing the
277+
to guarantee that when multiple versions of Kibana are performing the
274278
migration in parallel, only one version will win. E.g. if 7.11 and 7.12
275279
are started in parallel and migrate from a 7.9 index, either 7.11 or 7.12
276280
should succeed and accept writes, but not both.
277-
3. Checks that `.kibana` alias is still pointing to the source index
278-
4. Points the `.kibana_7.10.0` and `.kibana` aliases to the target index.
279-
5. If this fails with a "required alias [.kibana] does not exist" error fetch `.kibana` again:
281+
1. Check that `.kibana` alias is still pointing to the source index
282+
2. Point the `.kibana_7.10.0` and `.kibana` aliases to the target index.
283+
3. Remove the temporary index `.kibana_7.10.0_reindex_temp`
284+
4. If this fails with a "required alias [.kibana] does not exist" error or "index_not_found_exception" for the temporary index, fetch `.kibana` again:
280285
1. If `.kibana` is _not_ pointing to our target index fail the migration.
281-
2. If `.kibana` is pointing to our target index the migration has succeeded and we can proceed to step (10).
282-
10. Start serving traffic. All saved object reads/writes happen through the
286+
2. If `.kibana` is pointing to our target index the migration has succeeded and we can proceed to step (12).
287+
12. Start serving traffic. All saved object reads/writes happen through the
283288
version-specific alias `.kibana_7.10.0`.
284289
285290
Together with the limitations, this algorithm ensures that migrations are
286291
idempotent. If two nodes are started simultaneously, both of them will start
287292
transforming documents in that version's target index, but because migrations
288293
are idempotent, it doesn’t matter which node’s writes win.
289-
290294
#### Known weaknesses:
291295
(Also present in our existing migration algorithm since v7.4)
292296
When the task manager index gets reindexed a reindex script is applied.

‎src/dev/build/tasks/copy_source_task.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ export const CopySource: Task = {
2929
'!src/cli/dev.js',
3030
'!src/functional_test_runner/**',
3131
'!src/dev/**',
32+
'!src/plugins/telemetry/schema/**', // Skip telemetry schemas
3233
// this is the dev-only entry
3334
'!src/setup_node_env/index.js',
3435
'!**/public/**/*.{js,ts,tsx,json}',

‎src/plugins/discover/public/application/angular/discover.js‎

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -252,6 +252,12 @@ function discoverController($route, $scope, Promise) {
252252
(prop) => !_.isEqual(newStatePartial[prop], oldStatePartial[prop])
253253
);
254254

255+
if (oldStatePartial.hideChart && !newStatePartial.hideChart) {
256+
// in case the histogram is hidden, no data is requested
257+
// so when changing this state data needs to be fetched
258+
changes.push(true);
259+
}
260+
255261
if (changes.length) {
256262
refetch$.next();
257263
}
@@ -313,6 +319,8 @@ function discoverController($route, $scope, Promise) {
313319
setAppState,
314320
data,
315321
stateContainer,
322+
searchSessionManager,
323+
refetch$,
316324
};
317325

318326
const inspectorAdapters = ($scope.opts.inspectorAdapters = {
@@ -412,6 +420,9 @@ function discoverController($route, $scope, Promise) {
412420
if (savedSearch.grid) {
413421
defaultState.grid = savedSearch.grid;
414422
}
423+
if (savedSearch.hideChart) {
424+
defaultState.hideChart = savedSearch.hideChart;
425+
}
415426

416427
return defaultState;
417428
}
@@ -562,13 +573,6 @@ function discoverController($route, $scope, Promise) {
562573
});
563574
};
564575

565-
$scope.handleRefresh = function (_payload, isUpdate) {
566-
if (isUpdate === false) {
567-
searchSessionManager.removeSearchSessionIdFromURL({ replace: false });
568-
refetch$.next();
569-
}
570-
};
571-
572576
function getDimensions(aggs, timeRange) {
573577
const [metric, agg] = aggs;
574578
agg.params.timeRange = timeRange;
@@ -601,7 +605,7 @@ function discoverController($route, $scope, Promise) {
601605
function onResults(resp) {
602606
inspectorRequest.stats(getResponseInspectorStats(resp, $scope.searchSource)).ok({ json: resp });
603607

604-
if (getTimeField()) {
608+
if (getTimeField() && !$scope.state.hideChart) {
605609
const tabifiedData = tabifyAggResponse($scope.opts.chartAggConfigs, resp);
606610
$scope.searchSource.rawResponse = resp;
607611
$scope.histogramData = discoverResponseHandler(
@@ -704,7 +708,7 @@ function discoverController($route, $scope, Promise) {
704708

705709
async function setupVisualization() {
706710
// If no timefield has been specified we don't create a histogram of messages
707-
if (!getTimeField()) return;
711+
if (!getTimeField() || $scope.state.hideChart) return;
708712
const { interval: histogramInterval } = $scope.state;
709713

710714
const visStateAggs = [

‎src/plugins/discover/public/application/angular/discover_legacy.html‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,6 @@
1717
state="state"
1818
time-range="timeRange"
1919
top-nav-menu="topNavMenu"
20-
update-query="handleRefresh"
2120
use-new-fields-api="useNewFieldsApi"
2221
unmapped-fields-config="unmappedFieldsConfig"
2322
>

‎src/plugins/discover/public/application/angular/discover_state.ts‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,10 @@ export interface AppState {
4444
* Data Grid related state
4545
*/
4646
grid?: DiscoverGridSettings;
47+
/**
48+
* Hide chart
49+
*/
50+
hideChart?: boolean;
4751
/**
4852
* id of the used index pattern
4953
*/

‎src/plugins/discover/public/application/components/discover.test.tsx‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,8 @@ import { indexPatternWithTimefieldMock } from '../../__mocks__/index_pattern_wit
2525
import { calcFieldCounts } from '../helpers/calc_field_counts';
2626
import { DiscoverProps } from './types';
2727
import { RequestAdapter } from '../../../../inspector/common';
28+
import { Subject } from 'rxjs';
29+
import { DiscoverSearchSessionManager } from '../angular/discover_search_session';
2830

2931
const mockNavigation = navigationPluginMock.createStartContract();
3032

@@ -73,8 +75,10 @@ function getProps(indexPattern: IndexPattern): DiscoverProps {
7375
indexPatternList: (indexPattern as unknown) as Array<SavedObject<IndexPatternAttributes>>,
7476
inspectorAdapters: { requests: {} as RequestAdapter },
7577
navigateTo: jest.fn(),
78+
refetch$: {} as Subject<undefined>,
7679
sampleSize: 10,
7780
savedSearch: savedSearchMock,
81+
searchSessionManager: {} as DiscoverSearchSessionManager,
7882
setHeaderActionMenu: jest.fn(),
7983
timefield: indexPattern.timeFieldName || '',
8084
setAppState: jest.fn(),
@@ -86,7 +90,6 @@ function getProps(indexPattern: IndexPattern): DiscoverProps {
8690
rows: esHits,
8791
searchSource: searchSourceMock,
8892
state: { columns: [] },
89-
updateQuery: jest.fn(),
9093
};
9194
}
9295

‎src/plugins/discover/public/application/components/discover.tsx‎

Lines changed: 22 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66
* Side Public License, v 1.
77
*/
88
import './discover.scss';
9-
import React, { useCallback, useMemo, useRef, useState } from 'react';
9+
import React, { useState, useRef, useMemo, useCallback } from 'react';
1010
import {
1111
EuiButtonEmpty,
1212
EuiButtonIcon,
@@ -66,7 +66,6 @@ export function Discover({
6666
searchSource,
6767
state,
6868
timeRange,
69-
updateQuery,
7069
unmappedFieldsConfig,
7170
}: DiscoverProps) {
7271
const [expandedDoc, setExpandedDoc] = useState<ElasticSearchHit | undefined>(undefined);
@@ -76,8 +75,11 @@ export function Discover({
7675
// collapse icon isn't displayed in mobile view, use it to detect which view is displayed
7776
return collapseIcon && !collapseIcon.current;
7877
};
79-
80-
const [toggleOn, toggleChart] = useState(true);
78+
const toggleHideChart = useCallback(() => {
79+
const newState = { ...state, hideChart: !state.hideChart };
80+
opts.stateContainer.setAppState(newState);
81+
}, [state, opts]);
82+
const hideChart = useMemo(() => state.hideChart, [state]);
8183
const { savedSearch, indexPatternList, config, services, data, setAppState } = opts;
8284
const { trackUiMetric, capabilities, indexPatterns } = services;
8385
const [isSidebarClosed, setIsSidebarClosed] = useState(false);
@@ -89,6 +91,15 @@ export function Discover({
8991
const contentCentered = resultState === 'uninitialized';
9092
const isLegacy = services.uiSettings.get('doc_table:legacy');
9193
const useNewFieldsApi = !services.uiSettings.get(SEARCH_FIELDS_FROM_SOURCE);
94+
const updateQuery = useCallback(
95+
(_payload, isUpdate?: boolean) => {
96+
if (isUpdate === false) {
97+
opts.searchSessionManager.removeSearchSessionIdFromURL({ replace: false });
98+
opts.refetch$.next();
99+
}
100+
},
101+
[opts]
102+
);
92103

93104
const { onAddColumn, onRemoveColumn, onMoveColumn, onSetColumns } = useMemo(
94105
() =>
@@ -192,7 +203,8 @@ export function Discover({
192203
indexPattern={indexPattern}
193204
opts={opts}
194205
onOpenInspector={onOpenInspector}
195-
state={state}
206+
query={state.query}
207+
savedQuery={state.savedQuery}
196208
updateQuery={updateQuery}
197209
/>
198210
<EuiPageBody className="dscPageBody" aria-describedby="savedSearchTitle">
@@ -277,7 +289,7 @@ export function Discover({
277289
onResetQuery={resetQuery}
278290
/>
279291
</EuiFlexItem>
280-
{toggleOn && (
292+
{!hideChart && (
281293
<EuiFlexItem className="dscResultCount__actions">
282294
<TimechartHeader
283295
dateFormat={opts.config.get('dateFormat')}
@@ -293,13 +305,13 @@ export function Discover({
293305
<EuiFlexItem className="dscResultCount__toggle" grow={false}>
294306
<EuiButtonEmpty
295307
size="xs"
296-
iconType={toggleOn ? 'eyeClosed' : 'eye'}
308+
iconType={!hideChart ? 'eyeClosed' : 'eye'}
297309
onClick={() => {
298-
toggleChart(!toggleOn);
310+
toggleHideChart();
299311
}}
300312
data-test-subj="discoverChartToggle"
301313
>
302-
{toggleOn
314+
{!hideChart
303315
? i18n.translate('discover.hideChart', {
304316
defaultMessage: 'Hide chart',
305317
})
@@ -312,7 +324,7 @@ export function Discover({
312324
</EuiFlexGroup>
313325
{isLegacy && <SkipBottomButton onClick={onSkipBottomButtonClick} />}
314326
</EuiFlexItem>
315-
{toggleOn && opts.timefield && (
327+
{!hideChart && opts.timefield && (
316328
<EuiFlexItem grow={false}>
317329
<section
318330
aria-label={i18n.translate(

‎src/plugins/discover/public/application/components/discover_topnav.test.tsx‎

Lines changed: 8 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ import React from 'react';
1010
import { mountWithIntl } from '@kbn/test/jest';
1111
import { indexPatternMock } from '../../__mocks__/index_pattern';
1212
import { DiscoverServices } from '../../build_services';
13-
import { AppState, GetStateReturn } from '../angular/discover_state';
13+
import { GetStateReturn } from '../angular/discover_state';
1414
import { savedSearchMock } from '../../__mocks__/saved_search';
1515
import { dataPluginMock } from '../../../../data/public/mocks';
1616
import { createFilterManagerMock } from '../../../../data/public/query/filter_manager/filter_manager.mock';
@@ -20,9 +20,11 @@ import { SavedObject } from '../../../../../core/types';
2020
import { DiscoverTopNav, DiscoverTopNavProps } from './discover_topnav';
2121
import { RequestAdapter } from '../../../../inspector/common/adapters/request';
2222
import { TopNavMenu } from '../../../../navigation/public';
23+
import { Query } from '../../../../data/common';
24+
import { DiscoverSearchSessionManager } from '../angular/discover_search_session';
25+
import { Subject } from 'rxjs';
2326

2427
function getProps(): DiscoverTopNavProps {
25-
const state = ({} as unknown) as AppState;
2628
const services = ({
2729
navigation: {
2830
ui: { TopNavMenu },
@@ -45,15 +47,18 @@ function getProps(): DiscoverTopNavProps {
4547
indexPatternList: (indexPattern as unknown) as Array<SavedObject<IndexPatternAttributes>>,
4648
inspectorAdapters: { requests: {} as RequestAdapter },
4749
navigateTo: jest.fn(),
50+
refetch$: {} as Subject<undefined>,
4851
sampleSize: 10,
4952
savedSearch: savedSearchMock,
53+
searchSessionManager: {} as DiscoverSearchSessionManager,
5054
services,
5155
setAppState: jest.fn(),
5256
setHeaderActionMenu: jest.fn(),
5357
stateContainer: {} as GetStateReturn,
5458
timefield: indexPattern.timeFieldName || '',
5559
},
56-
state,
60+
query: {} as Query,
61+
savedQuery: '',
5762
updateQuery: jest.fn(),
5863
onOpenInspector: jest.fn(),
5964
};

0 commit comments

Comments
 (0)