Skip to content

Commit 0600828

Browse files
jbudzjbudz
committed
Merge branch 'master' into os/restart-on-upgrade
2 parents ecb18c4 + 19dd9b4 commit 0600828

285 files changed

Lines changed: 2662 additions & 1309 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/developer/contributing/development-ci-metrics.asciidoc‎

Lines changed: 77 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -3,6 +3,10 @@
33

44
In addition to running our tests, CI collects metrics about the Kibana build. These metrics are sent to an external service to track changes over time, and to provide PR authors insights into the impact of their changes.
55

6+
* <<ci-metric-types>>
7+
* <<ci-metric-resolving-overages>>
8+
* <<ci-metric-validating-limits>>
9+
610

711
[[ci-metric-types]]
812
=== Metric types
@@ -16,7 +20,7 @@ These metrics help contributors know how they are impacting the size of the bund
1620
[[ci-metric-page-load-bundle-size]] `page load bundle size` ::
1721
The size of the entry file produced for each bundle/plugin. This file is always loaded on every page load, so it should be as small as possible. To reduce this metric you can put any code that isn't necessary on every page load behind an https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import#Dynamic_Imports[`async import()`].
1822
+
19-
Code that is shared statically with other plugins will contribute to the `page load bundle size` of that plugin. This includes exports from the `public/index.ts` file and any file referenced by the `extraPublicDirs` manifest property.
23+
Code that is shared statically with other plugins will contribute to the `page load bundle size` of that plugin. This includes exports from the `public/index.ts` file and any file referenced by the `extraPublicDirs` manifest property.
2024

2125
[[ci-metric-async-chunks-size]] `async chunks size` ::
2226
An "async chunk" is created for the files imported by each https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Statements/import#Dynamic_Imports[`async import()`] statement. This metric tracks the sum size of these chunks, in bytes, broken down by plugin/bundle id. You can think of this as the amount of code users will have to download if they access all the components/applications within a bundle.
@@ -44,7 +48,7 @@ The number of files included in the default distributable.
4448
The number of files included in the OSS distributable.
4549

4650
[[ci-metric-distributable-size]] `distributable size` ::
47-
The size, in bytes, of the default distributable. _(not reported on PRs)_
51+
The size, in bytes, of the default distributable. _(not reported on PRs)_
4852

4953
[[ci-metric-oss-distributable-size]] `oss distributable size` ::
5054
The size, in bytes, of the OSS distributable. _(not reported on PRs)_
@@ -62,4 +66,74 @@ The number of saved object fields broken down by saved object type.
6266
[[ci-metric-adding-new-metrics]]
6367
=== Adding new metrics
6468

65-
You can report new metrics by using the `CiStatsReporter` class provided by the `@kbn/dev-utils` package. This class is automatically configured on CI and its methods noop when running outside of CI. For more details checkout the {kib-repo}blob/{branch}/packages/kbn-dev-utils/src/ci_stats_reporter[`CiStatsReporter` readme].
69+
You can report new metrics by using the `CiStatsReporter` class provided by the `@kbn/dev-utils` package. This class is automatically configured on CI and its methods noop when running outside of CI. For more details checkout the {kib-repo}blob/{branch}/packages/kbn-dev-utils/src/ci_stats_reporter[`CiStatsReporter` readme].
70+
71+
[[ci-metric-resolving-overages]]
72+
=== Resolving `page load bundle size` overages
73+
74+
In order to prevent the page load bundles from growing unexpectedly large we limit the `page load asset size` metric for each plugin. When a PR increases this metric beyond the limit defined for that plugin in {kib-repo}blob/{branch}/packages/kbn-optimizer/limits.yml[`limits.yml`] a failed commit status is set and the PR author needs to decide how to resolve this issue before the PR can be merged.
75+
76+
In most cases the limit should be high enough that PRs shouldn't trigger overages, but when they do make sure it's clear what is cuasing the overage by trying the following:
77+
78+
1. Run the optimizer locally with the `--profile` flag to produce webpack `stats.json` files for bundles which can be inspected using a number of different online tools. Focus on the chunk named `{pluginId}.plugin.js`; the `*.chunk.js` chunks make up the `async chunks size` metric which is currently unlimited and is the main way that we {kib-repo}blob/{branch}/src/core/MIGRATION.md#keep-kibana-fast[reduce the size of page load chunks].
79+
+
80+
[source,shell]
81+
-----------
82+
node scripts/build_kibana_platform_plugins --focus {pluginid} --profile
83+
# builds and creates {pluginDir}target/public/stats.json files for {pluginId} and any plugin it depends on
84+
-----------
85+
86+
- Official Webpack tool: http://webpack.github.io/analyse/
87+
- Webpack visualizer: https://chrisbateman.github.io/webpack-visualizer/
88+
89+
2. You might want to create stats for the upstream branch of your PR as well and then compare them side by side in Webpack visualizer to spot where the size difference is (using two browser tabs).
90+
91+
3. For relatively small changes you might be able to better understand the problem by sticking stats.json files from two different branches into https://www.scootersoftware.com/download.php[Beyond Compare]
92+
93+
4. If the number of changes in https://www.scootersoftware.com/download.php[Beyond Compare] are too large, you can reduce the stats.json file down to just a sorted list of module ids using https://github.com/stedolan/jq[jq]:
94+
+
95+
[source,shell]
96+
-----------
97+
jq -r .modules[].id {pluginDir}/target/public/stats.json | sort - > moduleids.txt
98+
-----------
99+
+
100+
Produce a moduleids.txt file for both your branch and master and then pop them into Beyond Compare to get a very specific view of what's new.
101+
102+
5. As a last resort you might want to try comparing the bundle source directly. It's usually best to do this using the production source so that you're inspecting the actual change in bytes that CI is seeing. After building the distributable version of your bundle run it through prettier and then dropping it into Beyond Compare along with the chunk from upstream:
103+
+
104+
[source,shell]
105+
-----------
106+
node scripts/build_kibana_platform_plugins --focus {pluginId} --dist
107+
npm install -g prettier
108+
prettier -w {pluginDir}/target/public/{pluginId}.plugin.js
109+
# repeat these steps for upstream and then compare the two {pluginId}.plugin.js files in Beyond Compare
110+
-----------
111+
112+
6. If all else fails reach out to Operations for help.
113+
114+
Once you've identified the files which were added to the build you likely just need to stick them behind an async import as described in {kib-repo}blob/{branch}/src/core/MIGRATION.md#keep-kibana-fast[the MIGRATION.md docs].
115+
116+
In the case that the bundle size is not being bloated by anything obvious, but it's still larger than the limit, you can raise the limit in your PR. Do this either by editting the {kib-repo}blob/{branch}/packages/kbn-optimizer/limits.yml[`limits.yml` file] manually or by running the following to have the limit updated to the current size + 15kb
117+
118+
[source,shell]
119+
-----------
120+
node scripts/build_kibana_platform_plugins --focus {pluginId} --update-limits
121+
-----------
122+
123+
This command has to run the optimizer in distributable mode so it will take a lot longer and spawn one worker for each CPU on your machine.
124+
125+
Changes to the {kib-repo}blob/{branch}/packages/kbn-optimizer/limits.yml[`limits.yml` file] will trigger review from the Operations team, who will attempt to verify that the size increase is justified. If you have findings you can share from the steps above that would be very helpful!
126+
127+
[[ci-metric-validating-limits]]
128+
=== Validating `page load bundle size` limits
129+
130+
Once you've fixed any issues discovered while diagnosing overages you probably should just push the changes to your PR and let CI validate them.
131+
132+
If you have a pretty powerful dev machine, or the necessary patience/determination, you can validate the limits locally by running the following command:
133+
134+
[source,shell]
135+
-----------
136+
node scripts/build_kibana_platform_plugins --validate-limits
137+
-----------
138+
139+
This command needs to apply production optimizations to get the right sizes, which means that the optimizer will take significantly longer to run and on most developmer machines will consume all of your machines resources for 20 minutes or more. If you'd like to multi-task while this is running you might need to limit the number of workers using the `--max-workers` flag.

‎docs/user/alerting/action-types/index.asciidoc‎

Lines changed: 5 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -77,13 +77,11 @@ PUT test
7777
"number_of_shards" : 1
7878
},
7979
"mappings" : {
80-
"_doc" : {
81-
"properties" : {
82-
"alert_id" : { "type" : "text" },
83-
"alert_name" : { "type" : "text" },
84-
"alert_instance_id" : { "type" : "text" },
85-
"context_message": { "type" : "text" }
86-
}
80+
"properties" : {
81+
"alert_id" : { "type" : "text" },
82+
"alert_name" : { "type" : "text" },
83+
"alert_instance_id" : { "type" : "text" },
84+
"context_message": { "type" : "text" }
8785
}
8886
}
8987
}

‎package.json‎

Lines changed: 20 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -77,6 +77,10 @@
7777
"url": "https://github.com/elastic/kibana.git"
7878
},
7979
"resolutions": {
80+
"**/@hapi/iron": "^5.1.4",
81+
"**/@types/hapi__boom": "^7.4.1",
82+
"**/@types/hapi__hapi": "^18.2.6",
83+
"**/@types/hapi__mimos": "4.1.0",
8084
"**/@types/node": ">=10.17.17 <10.20.0",
8185
"**/cross-fetch/node-fetch": "^2.6.1",
8286
"**/deepmerge": "^4.2.2",
@@ -124,7 +128,16 @@
124128
"@elastic/numeral": "^2.5.0",
125129
"@elastic/request-crypto": "1.1.4",
126130
"@elastic/safer-lodash-set": "0.0.0",
131+
"@hapi/boom": "^7.4.11",
132+
"@hapi/cookie": "^10.1.2",
127133
"@hapi/good-squeeze": "5.2.1",
134+
"@hapi/h2o2": "^8.3.2",
135+
"@hapi/hapi": "^18.4.1",
136+
"@hapi/hoek": "^8.5.1",
137+
"@hapi/inert": "^5.2.2",
138+
"@hapi/podium": "^3.4.3",
139+
"@hapi/statehood": "^6.1.2",
140+
"@hapi/vision": "^5.5.4",
128141
"@hapi/wreck": "^15.0.2",
129142
"@kbn/ace": "1.0.0",
130143
"@kbn/analytics": "1.0.0",
@@ -147,7 +160,6 @@
147160
"angular-elastic": "^2.5.1",
148161
"angular-sanitize": "^1.8.0",
149162
"bluebird": "3.5.5",
150-
"boom": "^7.2.0",
151163
"chalk": "^4.1.0",
152164
"check-disk-space": "^2.1.0",
153165
"chokidar": "^3.4.2",
@@ -167,15 +179,10 @@
167179
"glob": "^7.1.2",
168180
"glob-all": "^3.2.1",
169181
"globby": "^8.0.1",
170-
"h2o2": "^8.1.2",
171182
"handlebars": "4.7.6",
172-
"hapi": "^17.5.3",
173-
"hapi-auth-cookie": "^9.0.0",
174183
"hjson": "3.2.1",
175-
"hoek": "^5.0.4",
176184
"http-proxy-agent": "^2.1.0",
177185
"https-proxy-agent": "^5.0.0",
178-
"inert": "^5.1.0",
179186
"inline-style": "^2.0.0",
180187
"joi": "^13.5.2",
181188
"js-yaml": "^3.14.0",
@@ -220,7 +227,6 @@
220227
"tslib": "^2.0.0",
221228
"type-detect": "^4.0.8",
222229
"uuid": "3.3.2",
223-
"vision": "^5.3.3",
224230
"whatwg-fetch": "^3.0.0",
225231
"yauzl": "^2.10.0"
226232
},
@@ -274,7 +280,6 @@
274280
"@types/archiver": "^3.1.0",
275281
"@types/babel__core": "^7.1.10",
276282
"@types/bluebird": "^3.1.1",
277-
"@types/boom": "^7.2.0",
278283
"@types/chance": "^1.0.0",
279284
"@types/cheerio": "^0.22.10",
280285
"@types/chromedriver": "^81.0.0",
@@ -294,14 +299,16 @@
294299
"@types/glob": "^7.1.2",
295300
"@types/globby": "^8.0.0",
296301
"@types/graphql": "^0.13.2",
297-
"@types/h2o2": "^8.1.1",
298-
"@types/hapi": "^17.0.18",
299-
"@types/hapi-auth-cookie": "^9.1.0",
302+
"@types/hapi__boom": "^7.4.1",
303+
"@types/hapi__cookie": "^10.1.1",
304+
"@types/hapi__h2o2": "8.3.0",
305+
"@types/hapi__hapi": "^18.2.6",
306+
"@types/hapi__hoek": "^6.2.0",
307+
"@types/hapi__inert": "^5.2.1",
308+
"@types/hapi__podium": "^3.4.1",
300309
"@types/has-ansi": "^3.0.0",
301310
"@types/history": "^4.7.3",
302311
"@types/hjson": "^2.4.2",
303-
"@types/hoek": "^4.1.3",
304-
"@types/inert": "^5.1.2",
305312
"@types/jest": "^26.0.14",
306313
"@types/jest-when": "^2.7.1",
307314
"@types/joi": "^13.4.2",
@@ -326,7 +333,6 @@
326333
"@types/opn": "^5.1.0",
327334
"@types/pegjs": "^0.10.1",
328335
"@types/pngjs": "^3.4.0",
329-
"@types/podium": "^1.0.0",
330336
"@types/prop-types": "^15.7.3",
331337
"@types/reach__router": "^1.2.6",
332338
"@types/react": "^16.9.36",

‎src/core/public/public.api.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,7 @@
66

77
import { Action } from 'history';
88
import { ApiResponse } from '@elastic/elasticsearch/lib/Transport';
9-
import Boom from 'boom';
9+
import Boom from '@hapi/boom';
1010
import { ConfigPath } from '@kbn/config';
1111
import { EnvironmentMode } from '@kbn/config';
1212
import { EuiBreadcrumb } from '@elastic/eui';

‎src/core/server/elasticsearch/legacy/errors.test.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
* under the License.
1818
*/
1919

20-
import Boom from 'boom';
20+
import Boom from '@hapi/boom';
2121

2222
import { LegacyElasticsearchErrorHelpers } from './errors';
2323

‎src/core/server/elasticsearch/legacy/errors.ts‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@
1717
* under the License.
1818
*/
1919

20-
import Boom from 'boom';
20+
import Boom from '@hapi/boom';
2121
import { get } from 'lodash';
2222

2323
const code = Symbol('ElasticsearchError');

‎src/core/server/http/base_path_proxy_server.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,8 +22,8 @@ import { Agent as HttpsAgent, ServerOptions as TlsOptions } from 'https';
2222

2323
import apm from 'elastic-apm-node';
2424
import { ByteSizeValue } from '@kbn/config-schema';
25-
import { Server, Request } from 'hapi';
26-
import HapiProxy from 'h2o2';
25+
import { Server, Request } from '@hapi/hapi';
26+
import HapiProxy from '@hapi/h2o2';
2727
import { sampleSize } from 'lodash';
2828
import * as Rx from 'rxjs';
2929
import { take } from 'rxjs/operators';

‎src/core/server/http/cookie_session_storage.ts‎

Lines changed: 14 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -17,10 +17,10 @@
1717
* under the License.
1818
*/
1919

20-
import { Request, Server } from 'hapi';
21-
import hapiAuthCookie from 'hapi-auth-cookie';
20+
import { Request, Server } from '@hapi/hapi';
21+
import hapiAuthCookie from '@hapi/cookie';
2222
// @ts-expect-error no TS definitions
23-
import Statehood from 'statehood';
23+
import Statehood from '@hapi/statehood';
2424

2525
import { KibanaRequest, ensureRawRequest } from './router';
2626
import { SessionStorageFactory, SessionStorage } from './session_storage';
@@ -80,7 +80,7 @@ class ScopedCookieSessionStorage<T extends Record<string, any>> implements Sessi
8080
const session = await this.server.auth.test('security-cookie', this.request);
8181
// A browser can send several cookies, if it's not an array, just return the session value
8282
if (!Array.isArray(session)) {
83-
return session as T;
83+
return session.credentials as T;
8484
}
8585

8686
// If we have an array with one value, we're good also
@@ -141,20 +141,22 @@ export async function createCookieSessionStorageFactory<T>(
141141
await server.register({ plugin: hapiAuthCookie });
142142

143143
server.auth.strategy('security-cookie', 'cookie', {
144-
cookie: cookieOptions.name,
145-
password: cookieOptions.encryptionKey,
146-
validateFunc: async (req, session: T | T[]) => {
144+
cookie: {
145+
name: cookieOptions.name,
146+
password: cookieOptions.encryptionKey,
147+
isSecure: cookieOptions.isSecure,
148+
path: basePath === undefined ? '/' : basePath,
149+
clearInvalid: false,
150+
isHttpOnly: true,
151+
isSameSite: cookieOptions.sameSite === 'None' ? false : cookieOptions.sameSite ?? false,
152+
},
153+
validateFunc: async (req: Request, session: T | T[]) => {
147154
const result = cookieOptions.validate(session);
148155
if (!result.isValid) {
149156
clearInvalidCookie(req, result.path);
150157
}
151158
return { valid: result.isValid };
152159
},
153-
isSecure: cookieOptions.isSecure,
154-
path: basePath,
155-
clearInvalid: false,
156-
isHttpOnly: true,
157-
isSameSite: cookieOptions.sameSite === 'None' ? false : cookieOptions.sameSite ?? false,
158160
});
159161

160162
// A hack to support SameSite: 'None'.

‎src/core/server/http/http_server.mocks.ts‎

Lines changed: 10 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,8 @@
1616
* specific language governing permissions and limitations
1717
* under the License.
1818
*/
19-
import { parse as parseUrl } from 'url';
20-
import { Request } from 'hapi';
19+
import { URL, format as formatUrl } from 'url';
20+
import { Request } from '@hapi/hapi';
2121
import { merge } from 'lodash';
2222
import { Socket } from 'net';
2323
import { stringify } from 'query-string';
@@ -73,7 +73,7 @@ function createKibanaRequestMock<P = any, Q = any, B = any>({
7373
auth = { isAuthenticated: true },
7474
}: RequestFixtureOptions<P, Q, B> = {}) {
7575
const queryString = stringify(query, { sort: false });
76-
const url = parseUrl(`${path}${queryString ? `?${queryString}` : ''}`);
76+
const url = new URL(`${path}${queryString ? `?${queryString}` : ''}`, 'http://localhost');
7777

7878
return KibanaRequest.from<P, Q, B>(
7979
createRawRequestMock({
@@ -87,6 +87,9 @@ function createKibanaRequestMock<P = any, Q = any, B = any>({
8787
method,
8888
url,
8989
route: {
90+
// @ts-expect-error According to types/hapi__hapi the following settings-fields have problems:
91+
// - `auth` can't be a boolean, but it can according to the @hapi/hapi source (https://github.com/hapijs/hapi/blob/v18.4.2/lib/route.js#L139)
92+
// - `app` isn't a valid property, but it is and this was fixed in the types in v19.0.1 (https://github.com/DefinitelyTyped/DefinitelyTyped/pull/41968)
9093
settings: { tags: routeTags, auth: routeAuthRequired, app: kibanaRouteOptions },
9194
},
9295
raw: {
@@ -120,9 +123,11 @@ type DeepPartialObject<T> = { [P in keyof T]+?: DeepPartial<T[P]> };
120123
function createRawRequestMock(customization: DeepPartial<Request> = {}) {
121124
const pathname = customization.url?.pathname || '/';
122125
const path = `${pathname}${customization.url?.search || ''}`;
123-
const url = Object.assign({ pathname, path, href: path }, customization.url);
126+
const url = new URL(
127+
formatUrl(Object.assign({ pathname, path, href: path }, customization.url)),
128+
'http://localhost'
129+
);
124130

125-
// @ts-expect-error _core isn't supposed to be accessed - remove once we upgrade to hapi v18
126131
return merge(
127132
{},
128133
{
@@ -140,12 +145,6 @@ function createRawRequestMock(customization: DeepPartial<Request> = {}) {
140145
socket: {},
141146
},
142147
},
143-
// TODO: Remove once we upgrade to hapi v18
144-
_core: {
145-
info: {
146-
uri: 'http://localhost',
147-
},
148-
},
149148
},
150149
customization
151150
) as Request;

‎src/core/server/http/http_server.ts‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -16,8 +16,8 @@
1616
* specific language governing permissions and limitations
1717
* under the License.
1818
*/
19-
import { Server } from 'hapi';
20-
import HapiStaticFiles from 'inert';
19+
import { Server } from '@hapi/hapi';
20+
import HapiStaticFiles from '@hapi/inert';
2121
import url from 'url';
2222
import uuid from 'uuid';
2323

@@ -245,8 +245,11 @@ export class HttpServer {
245245
return;
246246
}
247247

248-
this.log.debug('stopping http server');
249-
await this.server.stop();
248+
const hasStarted = this.server.info.started > 0;
249+
if (hasStarted) {
250+
this.log.debug('stopping http server');
251+
await this.server.stop();
252+
}
250253
}
251254

252255
private getAuthOption(

0 commit comments

Comments
 (0)