Skip to content

Input docker behaviour with Openshift 3.10 #8988

@rbonvalot

Description

@rbonvalot

Filebeat Version : 6.4.3-1
Openshift version : 3.10

Since Openshift version 3.10, docker has been replaced by cri-o. Log format changed from json to cri format and we need to use input docker to parse file as describe in this issue #7418

Nethertheless, the path of docker logs in Openshift 3.10 is /var/log/containers/<pod_name><container_name>-<container_id>.log. Docker input consider this is this /var/lib/docker/containers/<container_id>/*.log.

I used this workaround for all docker

filebeat:
   inputs:
   - type: docker
     containers:
       path: "/var/log/containers"
       ids:
         - ''
     symlinks: true
     multiline.pattern: '^\['
     multiline.match: after
     multiline.negate: true
     fields:
       type: openshift-pods
     fields_under_root: true
     exclude_lines: ["DEBUG","TRACE"]
     exclude_files: ["rabbitmq","nginx","redis"]

It will considered this path -> /var/log/containers//*.log and it's ok if I wanted to process all log on the same way. In the other side, I need to process rabbitmq, nginx and redis pods on another ways but I'm blocked;

For example

filebeat:
   inputs:
   - type: docker
     containers:
       path: "/var/log/containers"
       ids:
         - ''
     symlinks: true
     multiline.pattern: '^[0-9]{4}-[0-9]{2}-[0-9]{2}'
     multiline.match: after
     multiline.negate: true
     fields:
       type: rabbit-pods
     fields_under_root: true
     ## Negative pattern not supported
     exclude_lines: [".*(?!rabbitmq).*"]
     ##
     exclude_lines: ["DEBUG","TRACE"]

This example failed. I can't use exclude_line with negative pattern. Include line is not supported too. I think it should be good to be able to modify the "/*.log" automatically added by filebeat with input docker.

PS : directory /var/log/pods should be the good directory for input but it contain only ID of dockers and I'll lost all <pod_name><container_name>-<container_id> I mentionned before.

Metadata

Metadata

Assignees

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions