Skip to content

Commit 90347cb

Browse files
madireymadirey
committed
Merge branch 'master' of github.com:elastic/kibana into threshold-exceptions
2 parents c8388f1 + 1bc42b9 commit 90347cb

55 files changed

Lines changed: 8081 additions & 5638 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.ci/teamcity/tests/xpack_list_cyclic_dependency.sh‎

Lines changed: 0 additions & 8 deletions
This file was deleted.

‎.ci/teamcity/tests/xpack_siem_cyclic_dependency.sh‎

Lines changed: 0 additions & 8 deletions
This file was deleted.

‎.teamcity/src/builds/test/QuickTests.kt‎

Lines changed: 0 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,6 @@ object QuickTests : BuildType({
1313

1414
val testScripts = mapOf(
1515
"Test Hardening" to ".ci/teamcity/tests/test_hardening.sh",
16-
"X-Pack List cyclic dependency" to ".ci/teamcity/tests/xpack_list_cyclic_dependency.sh",
17-
"X-Pack SIEM cyclic dependency" to ".ci/teamcity/tests/xpack_siem_cyclic_dependency.sh",
1816
"Test Projects" to ".ci/teamcity/tests/test_projects.sh",
1917
"Mocha Tests" to ".ci/teamcity/tests/mocha.sh"
2018
)

‎docs/developer/getting-started/debugging.asciidoc‎

Lines changed: 80 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -15,7 +15,17 @@ For information about how to debug unit tests, refer to <<debugging-unit-tests>>
1515
https://github.com/elastic/apm-agent-nodejs[Elastic APM Node.js Agent]
1616
built-in for debugging purposes.
1717

18-
Its default configuration is meant to be used by core {kib} developers
18+
With an application as varied and complex as Kibana has become, it's not practical or scalable to craft all possible performance measurements by hand ahead of time. As such, we need to rely on tooling to help us catch things we may otherwise have missed.
19+
20+
For example, say you implement a brand new feature, plugin or service but don't quite know how it will impact Kibana's performance as a whole. APM allows us to not only spot that something is slow, but also hints at why it might be performing slowly. For example, if a function is slow on specific types of inputs, we can see where the time is spent by viewing the trace for that function call in the APM UI.
21+
22+
image::images/apm_example_trace.png[]
23+
24+
The net of metrics captured by APM are both a wide and deep because the entire application is instrumented at runtime and we simply take a sample of these metrics. This means that we don't have to know what we need to measure ahead of time, we'll instead just get (most) of the data we're likely going to need by default.
25+
26+
This type of data can help us identify unknown bottlenecks, spot when a performance regression may have been introduced, and inform how the performance of Kibana is changing between releases. Using APM allows us to be proactive in getting ahead of potential performance regressions before they are released.
27+
28+
The default APM configuration is meant to be used by core {kib} developers
1929
only, but it can easily be re-configured to your needs. In its default
2030
configuration it’s disabled and will, once enabled, send APM data to a
2131
centrally managed {es} cluster accessible only to Elastic
@@ -27,11 +37,8 @@ APM config option. To activate the APM agent, use the
2737
https://www.elastic.co/guide/en/apm/agent/nodejs/current/configuration.html#active[`active`]
2838
APM config option.
2939

30-
All config options can be set either via environment variables, or by
31-
creating an appropriate config file under `config/apm.dev.js`. For
32-
more information about configuring the APM agent, please refer to
33-
https://www.elastic.co/guide/en/apm/agent/nodejs/current/configuring-the-agent.html[the
34-
documentation].
40+
All config options can be set by
41+
creating an appropriate config file under `config/apm.dev.js`.
3542

3643
Example `config/apm.dev.js` file:
3744

@@ -56,4 +63,70 @@ ELASTIC_APM_ACTIVE=true yarn start
5663
Once the agent is active, it will trace all incoming HTTP requests to
5764
{kib}, monitor for errors, and collect process-level metrics. The
5865
collected data will be sent to the APM Server and is viewable in the APM
59-
UI in {kib}.
66+
UI in {kib}.
67+
68+
[discrete]
69+
=== Running Kibana with the APM Agent Locally
70+
71+
The easiest and recommended way of running Kibana with the APM agent locally is to use the solution provided by the https://github.com/elastic/apm-integration-testing[apm-integration-testing] repo. You’ll need https://www.docker.com/community-edition[Docker] and https://docs.docker.com/compose/install/[Docker Compose] to use the tool.
72+
73+
[discrete]
74+
==== Quick start guide
75+
76+
. Clone the https://github.com/elastic/apm-integration-testing[elastic/apm-integration-testing] repo.
77+
. Change into the apm-integration-testing repo:
78+
+
79+
[source,bash]
80+
----
81+
cd apm-integration-testing
82+
----
83+
84+
. Run {es} and the APM servers without running Kibana:
85+
+
86+
[source,bash]
87+
----
88+
./scripts/compose.py start master --no-kibana
89+
----
90+
91+
. Change into the {kib} repo:
92+
+
93+
[source,bash]
94+
----
95+
cd ../kibana
96+
----
97+
98+
. Change the elasticsearch credentials in your `kibana.yml` configuration file to match those needed by elasticsearch and the APM server (see the apm-integration-testing repo's https://github.com/elastic/apm-integration-testing#logging-in[README] for users provided to test different scenarios).
99+
. Make sure that the APM agent is active and points to the local APM server by adding the following configuration settings to to a config file under `config/apm.dev.js`:
100+
+
101+
Example `config/apm.dev.js` file:
102+
+
103+
[source,js]
104+
----
105+
module.exports = {
106+
active: true,
107+
serverUrl: 'http://127.0.0.1:8200', // supports `http://localhost:8200`
108+
centralConfig: false,
109+
breakdownMetrics: false,
110+
transactionSampleRate: 0.1,
111+
metricsInterval: '120s'
112+
};
113+
----
114+
115+
. Start Kibana with APM active using:
116+
+
117+
[source,bash]
118+
----
119+
yarn start
120+
----
121+
122+
. After Kibana starts up, navigate to the APM app, where you should see some transactions.
123+
124+
image::images/apm_ui_transactions.png[]
125+
126+
You can now continue doing what you want to in Kibana (e.g. install sample data sets, issue queries in dashboards, build new visualizations etc).
127+
Once you're finished, you can stop Kibana normally, then stop the {es} and APM servers in the apm-integration-testing clone with the following script:
128+
129+
[source,bash]
130+
----
131+
./scripts/compose.py stop
132+
----

‎docs/developer/images/apm_example_trace.png‎

557 KB
Loading

‎docs/developer/images/apm_ui_transactions.png‎

192 KB
Loading

‎docs/fleet/fleet.asciidoc‎

Lines changed: 1 addition & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,13 +17,11 @@ Standalone mode requires you to manually configure and manage the agent locally.
1717
* An overview of the data ingest in your {es} cluster.
1818
* Multiple integrations to collect and transform data.
1919

20-
//TODO: Redo screen capture.
21-
2220
[role="screenshot"]
2321
image::fleet/images/fleet-start.png[{fleet} app in {kib}]
2422

2523
[float]
2624
== Get started
2725

2826
To get started with {fleet}, refer to the
29-
{ingest-guide}/index.html[Ingest Management Guide].
27+
{ingest-guide}/index.html[{fleet}] docs.

‎docs/settings/fleet-settings.asciidoc‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ experimental[]
1010
You can configure `xpack.fleet` settings in your `kibana.yml`.
1111
By default, {fleet} is enabled. To use {fleet}, you also need to configure {kib} and {es} hosts.
1212

13-
See the {ingest-guide}/index.html[Ingest Management] docs for more information.
13+
See the {ingest-guide}/index.html[{fleet}] docs for more information.
1414

1515
[[general-fleet-settings-kb]]
1616
==== General {fleet} settings

‎package.json‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -731,7 +731,6 @@
731731
"loader-utils": "^1.2.3",
732732
"log-symbols": "^2.2.0",
733733
"lz-string": "^1.4.4",
734-
"madge": "3.4.4",
735734
"mapbox-gl": "^1.12.0",
736735
"mapbox-gl-draw-rectangle-mode": "^1.0.4",
737736
"marge": "^1.0.1",

‎packages/kbn-es-archiver/src/cli.ts‎

Lines changed: 47 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -26,8 +26,9 @@
2626
import Path from 'path';
2727
import Url from 'url';
2828
import readline from 'readline';
29+
import Fs from 'fs';
2930

30-
import { RunWithCommands, createFlagError } from '@kbn/dev-utils';
31+
import { RunWithCommands, createFlagError, KbnClient, CA_CERT_PATH } from '@kbn/dev-utils';
3132
import { readConfigFile } from '@kbn/test';
3233
import legacyElasticsearch from 'elasticsearch';
3334

@@ -40,13 +41,15 @@ export function runCli() {
4041
new RunWithCommands({
4142
description: 'CLI to manage archiving/restoring data in elasticsearch',
4243
globalFlags: {
43-
string: ['es-url', 'kibana-url', 'dir', 'config'],
44+
string: ['es-url', 'kibana-url', 'dir', 'config', 'es-ca', 'kibana-ca'],
4445
help: `
4546
--config path to an FTR config file that sets --es-url, --kibana-url, and --dir
4647
default: ${defaultConfigPath}
4748
--es-url url for Elasticsearch, prefer the --config flag
4849
--kibana-url url for Kibana, prefer the --config flag
4950
--dir where arechives are stored, prefer the --config flag
51+
--kibana-ca if Kibana url points to https://localhost we default to the CA from @kbn/dev-utils, customize the CA with this flag
52+
--es-ca if Elasticsearch url points to https://localhost we default to the CA from @kbn/dev-utils, customize the CA with this flag
5053
`,
5154
},
5255
async extendContext({ log, flags, addCleanupTask }) {
@@ -78,6 +81,40 @@ export function runCli() {
7881
throw createFlagError('--kibana-url or --config must be defined');
7982
}
8083

84+
const kibanaCaPath = flags['kibana-ca'];
85+
if (kibanaCaPath && typeof kibanaCaPath !== 'string') {
86+
throw createFlagError('--kibana-ca must be a string');
87+
}
88+
89+
let kibanaCa;
90+
if (config.get('servers.kibana.certificateAuthorities') && !kibanaCaPath) {
91+
kibanaCa = config.get('servers.kibana.certificateAuthorities');
92+
} else if (kibanaCaPath) {
93+
kibanaCa = Fs.readFileSync(kibanaCaPath);
94+
} else {
95+
const { protocol, hostname } = Url.parse(kibanaUrl);
96+
if (protocol === 'https:' && hostname === 'localhost') {
97+
kibanaCa = Fs.readFileSync(CA_CERT_PATH);
98+
}
99+
}
100+
101+
const esCaPath = flags['es-ca'];
102+
if (esCaPath && typeof esCaPath !== 'string') {
103+
throw createFlagError('--es-ca must be a string');
104+
}
105+
106+
let esCa;
107+
if (config.get('servers.elasticsearch.certificateAuthorities') && !esCaPath) {
108+
esCa = config.get('servers.elasticsearch.certificateAuthorities');
109+
} else if (esCaPath) {
110+
esCa = Fs.readFileSync(esCaPath);
111+
} else {
112+
const { protocol, hostname } = Url.parse(kibanaUrl);
113+
if (protocol === 'https:' && hostname === 'localhost') {
114+
esCa = Fs.readFileSync(CA_CERT_PATH);
115+
}
116+
}
117+
81118
let dir = flags.dir;
82119
if (dir && typeof dir !== 'string') {
83120
throw createFlagError('--dir must be a string');
@@ -91,15 +128,22 @@ export function runCli() {
91128

92129
const client = new legacyElasticsearch.Client({
93130
host: esUrl,
131+
ssl: esCa ? { ca: esCa } : undefined,
94132
log: flags.verbose ? 'trace' : [],
95133
});
96134
addCleanupTask(() => client.close());
97135

136+
const kbnClient = new KbnClient({
137+
log,
138+
url: kibanaUrl,
139+
certificateAuthorities: kibanaCa ? [kibanaCa] : undefined,
140+
});
141+
98142
const esArchiver = new EsArchiver({
99143
log,
100144
client,
101145
dataDir: dir,
102-
kibanaUrl,
146+
kbnClient,
103147
});
104148

105149
return {

0 commit comments

Comments
 (0)