Enabling agent.monitoring.metrics in an agent policy will spawn a Metricbeat instance to collect metrics from processes supervised by the agent. This Metricbeat instance needs to have a module enabled to capture the shipper's CPU utilization, memory usage, and application specific metrics. The monitoring configuration for this Metricbeat instance is defined in the the v1_monitor.go file.
The agent currently knows how to start both the beat/metrics module for monitoring beats and the http/metrics module for monitoring the agent itself:
https://github.com/elastic/elastic-agent/blob/f27f8d01bcb57f21903c1bc4927d7bf62ba46595/internal/pkg/agent/application/monitoring/v1_monitor.go#L786-L796
inputs := []interface{}{
map[string]interface{}{
idKey: "metrics-monitoring-beats",
"name": "metrics-monitoring-beats",
"type": "beat/metrics",
useOutputKey: monitoringOutput,
"data_stream": map[string]interface{}{
"namespace": monitoringNamespace,
},
"streams": beatsStreams,
},
map[string]interface{}{
idKey: "metrics-monitoring-agent",
"name": "metrics-monitoring-agent",
"type": "http/metrics",
Likely the easiest path to support the shipper is to use the http/metrics input to collect the shipper's metrics. A recent example doing this for custom Filebeat metrics can be found in elastic/elastic-agent#2171, with the corresponding updates to the mappings in the Elastic Agent integration in elastic/integrations#5077.
The shipper metrics will need to be written to a dedicated data stream if elastic/elastic-agent#1814 is not completed before this issue is completed. This will require updating the list of possible monitoring datastreams in the Fleet UI as described in that issue.
Acceptance Criteria:
- Enabling
agent.monitoring.metrics causes the agent to start a Metricbeat monitoring input capable of collecting the shipper's metrics and sending them to Fleet.
- A demo can be performed proving that shipper CPU, memory, queue, output, and other application specific metrics are shipped to the expected data stream in Fleet and can be queried and inspected.
The work to update the Elastic Agent monitoring dashboard to account for the new metrics is captured into #54
Enabling
agent.monitoring.metricsin an agent policy will spawn a Metricbeat instance to collect metrics from processes supervised by the agent. This Metricbeat instance needs to have a module enabled to capture the shipper's CPU utilization, memory usage, and application specific metrics. The monitoring configuration for this Metricbeat instance is defined in the the v1_monitor.go file.The agent currently knows how to start both the
beat/metricsmodule for monitoring beats and thehttp/metricsmodule for monitoring the agent itself:https://github.com/elastic/elastic-agent/blob/f27f8d01bcb57f21903c1bc4927d7bf62ba46595/internal/pkg/agent/application/monitoring/v1_monitor.go#L786-L796
Likely the easiest path to support the shipper is to use the
http/metricsinput to collect the shipper's metrics. A recent example doing this for custom Filebeat metrics can be found in elastic/elastic-agent#2171, with the corresponding updates to the mappings in the Elastic Agent integration in elastic/integrations#5077.The shipper metrics will need to be written to a dedicated data stream if elastic/elastic-agent#1814 is not completed before this issue is completed. This will require updating the list of possible monitoring datastreams in the Fleet UI as described in that issue.
Acceptance Criteria:
agent.monitoring.metricscauses the agent to start a Metricbeat monitoring input capable of collecting the shipper's metrics and sending them to Fleet.The work to update the Elastic Agent monitoring dashboard to account for the new metrics is captured into #54