CRI wrong partial log line #4813
Description
Description
When the last line of a container log doesn't container a newline at the end, cri containerd will log a partial log which will confuse log forwarding tools like fluentd.
Steps to reproduce the issue:
- Create this pod in Kubernetes:
apiVersion: v1
kind: Pod
metadata:
name: myapp-pod
labels:
app: myapp
spec:
containers:
- name: myapp-container
image: busybox
command: ['sh', '-c', 'echo The app is running! && sleep 3600']
initContainers:
- name: init-myservice
image: busybox
command: ['sh', '-c', 'printf "Hello from init container!";']
Describe the results you received:
The log of the init container will contain a partial (P) log line:
2020-12-07T15:32:38.994574754Z stdout P Hello from init container!
Describe the results you expected:
The log should contain a full (F) log line instead:
2020-12-07T15:32:38.994574754Z stdout F Hello from init container!
Output of containerd --version:
containerd github.com/containerd/containerd v1.4.1 c623d1b36f09f8ef6536a057bd658b3aa8632828
Any other relevant information:
This confuses the concat plugin of fluentd and it times out:
2020-12-07 15:33:39 +0000 [warn]: #0 dump an error event: error_class=Fluent::Plugin::ConcatFilter::TimeoutError error="Timeout flush: except.kubernetes.var.log.pods.default_myapp-pod_bce902ae-33e0-43a5-b5b4-f3cbbbabd56a.init-myservice.0.log:default" location=nil tag="except.kubernetes.var.log.pods.default_myapp-pod_bce902ae-33e0-43a5-b5b4-f3cbbbabd56a.init-myservice.0.log" time=2020-12-07 15:33:39.042533262 +0000 record={"logtag"=>"P", "log"=>"Hello from init container!", "tailed_path"=>"/var/log/pods/default_myapp-pod_bce902ae-33e0-43a5-b5b4-f3cbbbabd56a/init-myservice/0.log"}