Currently, beats in kubernetes are thought to be run as a daemon set where each beat collects information from the pods running in the same node, and from the node itself.
Autodiscover is built on this premise and it only discovers pods running in the same node.
Running the beat in the same node as the observed pods is necessary for example in filebeat because it needs access to local files, but it doesn't need to be necessary in metricbeat modules, that could be connecting to network endpoints.
(The kubernetes module is an exception to that, it collects information from the API and don't need to be associated to any node, and is actually recommended to be run from a single Deployment per cluster)
The proposal is to add an option to remove this dependency between autodiscover and the node where the beat is running, so, when possible, pods are discovered from a different node. This would allow different deployment architectures in special cases:
- Monitoring services from an endpoint external to the cluster, this can be quite interesting for heartbeat for example.
- Collect metrics from a single metricbeat pod. For example to collect prometheus metrics from all the cluster from a single metricbeat pod.
Currently, beats in kubernetes are thought to be run as a daemon set where each beat collects information from the pods running in the same node, and from the node itself.
Autodiscover is built on this premise and it only discovers pods running in the same node.
Running the beat in the same node as the observed pods is necessary for example in filebeat because it needs access to local files, but it doesn't need to be necessary in metricbeat modules, that could be connecting to network endpoints.
(The kubernetes module is an exception to that, it collects information from the API and don't need to be associated to any node, and is actually recommended to be run from a single Deployment per cluster)
The proposal is to add an option to remove this dependency between autodiscover and the node where the beat is running, so, when possible, pods are discovered from a different node. This would allow different deployment architectures in special cases: