Describe the enhancement:
It's possible to use autodiscover templating + hints based autodiscover together in a single Filebeat configuration. I don't think this is very clear in the docs, and it might be helpful to document an example while explaining the expected behavior. Just a rough idea:
Container 1
docker run -d --publish-all \
--label "type=default" \
-e "ES_JAVA_OPTS=-Xms1g -Xmx1g" \
-e "discovery.type=single-node" \
-e "cluster.name=cluster-1" \
docker.elastic.co/elasticsearch/elasticsearch:7.13.2
Container 2
docker run -d --publish-all \
--label "co.elastic.logs/module=elasticsearch" \
--label "type=module" \
-e "ES_JAVA_OPTS=-Xms1g -Xmx1g" \
-e "discovery.type=single-node" \
-e "cluster.name=cluster-2" \
docker.elastic.co/elasticsearch/elasticsearch:7.13.2
Container 3
docker run -d --publish-all \
--label "type=pipeline" \
-e "ES_JAVA_OPTS=-Xms1g -Xmx1g" \
-e "discovery.type=single-node" \
-e "cluster.name=cluster-3" \
docker.elastic.co/elasticsearch/elasticsearch:7.13.2
filebeat.autodiscover.providers:
- type: docker
hints.enabled: true
hints.default_config:
type: container
paths:
- /var/lib/docker/containers/${data.container.id}/*.log
templates:
- condition:
equals:
docker.container.labels.type: "pipeline"
config:
- type: container
paths:
- "/var/lib/docker/containers/${data.docker.container.id}/*.log"
pipeline: my-pipelineThe above Filebeat config results in 3x separate configurations:
- Container 1 - will be read in using
hints.default_config.
- Container 2 - will be read in using the Elasticsearch module.
- Container 3 - will be read in using a generic container input but also leverage an ingest pipeline called
my-pipeline.
Describe the enhancement:
It's possible to use autodiscover templating + hints based autodiscover together in a single Filebeat configuration. I don't think this is very clear in the docs, and it might be helpful to document an example while explaining the expected behavior. Just a rough idea:
Container 1
Container 2
Container 3
The above Filebeat config results in 3x separate configurations:
hints.default_config.my-pipeline.