Filebeat Version : 6.4.3-1
Openshift version : 3.10
Since Openshift version 3.10, docker has been replaced by cri-o. Log format changed from json to cri format and we need to use input docker to parse file as describe in this issue #7418
Nethertheless, the path of docker logs in Openshift 3.10 is /var/log/containers/<pod_name><container_name>-<container_id>.log. Docker input consider this is this /var/lib/docker/containers/<container_id>/*.log.
I used this workaround for all docker
filebeat:
inputs:
- type: docker
containers:
path: "/var/log/containers"
ids:
- ''
symlinks: true
multiline.pattern: '^\['
multiline.match: after
multiline.negate: true
fields:
type: openshift-pods
fields_under_root: true
exclude_lines: ["DEBUG","TRACE"]
exclude_files: ["rabbitmq","nginx","redis"]
It will considered this path -> /var/log/containers//*.log and it's ok if I wanted to process all log on the same way. In the other side, I need to process rabbitmq, nginx and redis pods on another ways but I'm blocked;
For example
filebeat:
inputs:
- type: docker
containers:
path: "/var/log/containers"
ids:
- ''
symlinks: true
multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
multiline.match: after
multiline.negate: true
fields:
type: rabbit-pods
fields_under_root: true
## Negative pattern not supported
exclude_lines: [".*(?!rabbitmq).*"]
##
exclude_lines: ["DEBUG","TRACE"]
This example failed. I can't use exclude_line with negative pattern. Include line is not supported too. I think it should be good to be able to modify the "/*.log" automatically added by filebeat with input docker.
PS : directory /var/log/pods should be the good directory for input but it contain only ID of dockers and I'll lost all <pod_name><container_name>-<container_id> I mentionned before.
Filebeat Version : 6.4.3-1
Openshift version : 3.10
Since Openshift version 3.10, docker has been replaced by cri-o. Log format changed from json to cri format and we need to use input docker to parse file as describe in this issue #7418
Nethertheless, the path of docker logs in Openshift 3.10 is /var/log/containers/<pod_name><container_name>-<container_id>.log. Docker input consider this is this /var/lib/docker/containers/<container_id>/*.log.
I used this workaround for all docker
filebeat: inputs: - type: docker containers: path: "/var/log/containers" ids: - '' symlinks: true multiline.pattern: '^\[' multiline.match: after multiline.negate: true fields: type: openshift-pods fields_under_root: true exclude_lines: ["DEBUG","TRACE"] exclude_files: ["rabbitmq","nginx","redis"]It will considered this path -> /var/log/containers//*.log and it's ok if I wanted to process all log on the same way. In the other side, I need to process rabbitmq, nginx and redis pods on another ways but I'm blocked;
For example
filebeat: inputs: - type: docker containers: path: "/var/log/containers" ids: - '' symlinks: true multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}' multiline.match: after multiline.negate: true fields: type: rabbit-pods fields_under_root: true ## Negative pattern not supported exclude_lines: [".*(?!rabbitmq).*"] ## exclude_lines: ["DEBUG","TRACE"]This example failed. I can't use exclude_line with negative pattern. Include line is not supported too. I think it should be good to be able to modify the "/*.log" automatically added by filebeat with input docker.
PS : directory /var/log/pods should be the good directory for input but it contain only ID of dockers and I'll lost all <pod_name><container_name>-<container_id> I mentionned before.