Skip to content

Commit b8e0449

Browse files
efd6efd6
authored
packetbeat,winlogbeat,x-pack/filebeat: add metrics documentation (#34889)
This adds documentation for execution metrics collection by filebeat inputs packetbeat protocols, and winlogbeat event processing.
1 parent 5c0aae2 commit b8e0449

10 files changed

Lines changed: 102 additions & 20 deletions

‎CHANGELOG.next.asciidoc‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -236,6 +236,7 @@ automatic splitting at root level, if root level element is an array. {pull}3415
236236
- Add string manipulation extensions to CEL input {issue}34610[34610] {pull}34689[34689]
237237
- Add unix socket log parsing for nginx ingress_controller {pull}34732[34732]
238238
- Improve CEL input documentation {pull}34831[34831]
239+
- Add metrics documentation for CEL and AWS CloudWatch inputs. {issue}34887[34887] {pull}34889[34889]
239240

240241
*Auditbeat*
241242

@@ -272,6 +273,7 @@ automatic splitting at root level, if root level element is an array. {pull}3415
272273
- Reduce logging level for ENOENT to WARN when mapping sockets to processes. {issue}33793[33793] {pull}33854[33854]
273274
- Add metrics for TCP and UDP packet processing. {pull}33833[33833] {pull}34353[34353]
274275
- Allow user to prevent Npcap library installation on Windows. {issue}34420[34420] {pull}34428[34428]
276+
- Add metrics documentation for TCP and UDP protocols. {issue}34887[34887] {pull}34889[34889]
275277

276278
*Packetbeat*
277279

@@ -282,6 +284,7 @@ automatic splitting at root level, if root level element is an array. {pull}3415
282284
*Winlogbeat*
283285

284286
- Add metrics for log event processing. {pull}33922[33922]
287+
- Add metrics documentation for event processing. {issue}34887[34887] {pull}34889[34889]
285288

286289
*Elastic Log Driver*
287290

‎filebeat/docs/inputs/input-tcp.asciidoc‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ include::../inputs/input-common-tcp-options.asciidoc[]
3131
=== Metrics
3232

3333
This input exposes metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
34-
These metrics are exposed under the `/dataset` path. They can be used to
34+
These metrics are exposed under the `/inputs` path. They can be used to
3535
observe the activity of the input.
3636

3737
[options="header"]

‎filebeat/docs/inputs/input-udp.asciidoc‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -31,7 +31,7 @@ include::../inputs/input-common-udp-options.asciidoc[]
3131
=== Metrics
3232

3333
This input exposes metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
34-
These metrics are exposed under the `/dataset` path. They can be used to
34+
These metrics are exposed under the `/inputs` path. They can be used to
3535
observe the activity of the input.
3636

3737
[options="header"]

‎filebeat/docs/inputs/input-unix.asciidoc‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ include::../inputs/input-common-unix-options.asciidoc[]
3333
=== Metrics
3434

3535
This input exposes metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
36-
These metrics are exposed under the `/dataset` path. They can be used to
36+
These metrics are exposed under the `/inputs` path. They can be used to
3737
observe the activity of the input.
3838

3939
[options="header"]

‎packetbeat/docs/configuring-howto.asciidoc‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -57,6 +57,8 @@ include::{libbeat-dir}/loggingconfig.asciidoc[]
5757

5858
include::{libbeat-dir}/http-endpoint.asciidoc[]
5959

60+
include::./protocol-metrics-packetbeat.asciidoc[]
61+
6062
include::{libbeat-dir}/shared-instrumentation.asciidoc[]
6163

6264
include::{libbeat-dir}/shared-feature-flags.asciidoc[]

‎packetbeat/docs/protocol-metrics-packetbeat.asciidoc‎

Lines changed: 36 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,36 @@
1+
[[protocol-metrics-packetbeat]]
2+
=== Protocol-Specific Metrics
3+
4+
Packetbeat exposes per-protocol metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
5+
These metrics are exposed under the `/inputs` path. They can be used to
6+
observe the activity of Packetbeat for the monitored protocol.
7+
8+
[float]
9+
==== TCP Metrics
10+
11+
[options="header"]
12+
|=======
13+
| Metric | Description
14+
| `device` | Name of the device being monitored.
15+
| `received_events_total` | Number of packets processed.
16+
| `received_bytes_total` | Number of bytes processed.
17+
| `tcp_overlaps` | Number of packets shrunk due to overlap.
18+
| `tcp.dropped_because_of_gaps` | Number of packets dropped because of gaps.
19+
| `arrival_period` | Histogram of the elapsed time between packet arrivals.
20+
| `processing_time` | Histogram of the elapsed time between packet receipt and publication.
21+
|=======
22+
23+
24+
[float]
25+
==== UDP Metrics
26+
27+
[options="header"]
28+
|=======
29+
| Metric | Description
30+
| `device` | Name of the device being monitored.
31+
| `received_events_total` | Number of packets processed.
32+
| `received_bytes_total` | Number of bytes processed.
33+
| `arrival_period` | Histogram of the elapsed time between packet arrivals.
34+
| `processing_time` | Histogram of the elapsed time between packet receipt and publication.
35+
|=======
36+

‎winlogbeat/docs/configuring-howto.asciidoc‎

Lines changed: 1 addition & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -53,23 +53,7 @@ include::{libbeat-dir}/loggingconfig.asciidoc[]
5353

5454
include::{libbeat-dir}/http-endpoint.asciidoc[]
5555

56-
[float]
57-
=== Metrics
58-
59-
{beatname_uc} exposes the following additional metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
60-
These metrics are exposed under the `/dataset` path.
61-
62-
[options="header"]
63-
|=======
64-
| Metric | Description
65-
| `provider` | The name of the provider being read.
66-
| `received_events_total` | Total number of events read by the input.
67-
| `discarded_events_total` | Total number of events dropped by the input.
68-
| `errors_total` | Total number of errors encountered by the input.
69-
| `batch_read_period` | A histogram of intervals between non-empty event batch reads.
70-
| `received_events_count` | A histogram of the number of events read in each batch.
71-
| `source_lag_time` | The difference between the timestamp recorded in each event and the time when it was read.
72-
|=======
56+
include::./metrics-winlogbeat.asciidoc[]
7357

7458
include::{libbeat-dir}/shared-instrumentation.asciidoc[]
7559

‎winlogbeat/docs/metrics-winlogbeat.asciidoc‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
[[metrics-winlogbeat]]
2+
=== Event Processing Metrics
3+
4+
Winlogbeat exposes metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
5+
These metrics are exposed under the `/inputs` path. They can be used to
6+
observe the event log processing activity of Winlogbeat.
7+
8+
[float]
9+
==== Winlog Metrics
10+
11+
[options="header"]
12+
|=======
13+
| Metric | Description
14+
| `provider` | Name of the provider being read.
15+
| `received_events_total` | Total number of events received.
16+
| `discarded_events_total` | Total number of discarded events.
17+
| `errors_total` | Total number of errors.
18+
| `received_events_count` | Histogram of the number of events in each non-zero batch.
19+
| `source_lag_time` | Histogram of the difference between timestamped event's creation and reading.
20+
| `batch_read_period` | Histogram of the elapsed time between non-zero batch reads.
21+
|=======

‎x-pack/filebeat/docs/inputs/input-aws-cloudwatch.asciidoc‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,6 +130,22 @@ cloudwatchlogs:DescribeLogGroups
130130
logs:FilterLogEvents
131131
----
132132

133+
[float]
134+
=== Metrics
135+
136+
This input exposes metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
137+
These metrics are exposed under the `/inputs` path. They can be used to
138+
observe the activity of the input.
139+
140+
[options="header"]
141+
|=======
142+
| Metric | Description
143+
| `log_events_received_total` | Number of CloudWatch log events received.
144+
| `log_groups_total` | Logs collected from number of CloudWatch log groups.
145+
| `cloudwatch_events_created_total` | Number of events created from processing logs from CloudWatch.
146+
| `api_calls_total` | Number of API calls made total.
147+
|=======
148+
133149
[id="{beatname_lc}-input-{type}-common-options"]
134150
include::../../../../filebeat/docs/inputs/input-common-options.asciidoc[]
135151

‎x-pack/filebeat/docs/inputs/input-cel.asciidoc‎

Lines changed: 20 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -601,6 +601,26 @@ This specifies fields in the `state` to be redacted prior to debug logging. Fiel
601601

602602
This specifies whether fields should be replaced with a `*` or deleted entirely from messages sent to debug logs. If delete is `true`, fields will be deleted rather than replaced.
603603

604+
[float]
605+
=== Metrics
606+
607+
This input exposes metrics under the <<http-endpoint, HTTP monitoring endpoint>>.
608+
These metrics are exposed under the `/inputs` path. They can be used to
609+
observe the activity of the input.
610+
611+
[options="header"]
612+
|=======
613+
| Metric | Description
614+
| `resource` | URL or path of the input resource.
615+
| `cel_executions` | Number times the CEL program has been executed.
616+
| `batches_received_total` | Number of event arrays received.
617+
| `events_received_total` | Number of events received.
618+
| `batches_published_total` | Number of event arrays published.
619+
| `events_published_total` | Number of events published.
620+
| `cel_processing_time` | Histogram of the elapsed successful CEL program processing times in nanoseconds.
621+
| `batch_processing_time` | Histogram of the elapsed successful batch processing times in nanoseconds (time of receipt to time of ACK for non-empty batches).
622+
|=======
623+
604624
[id="{beatname_lc}-input-{type}-common-options"]
605625
include::../../../../filebeat/docs/inputs/input-common-options.asciidoc[]
606626

0 commit comments

Comments
 (0)