containerd-shim-runc-v2 exit abnormally #6402
Description
Description
We cause an strange phenomenon that the container's containerd-shim-runc-v2 and process inside the container both exit, but docker ps can still list this container with Up state.
The containerd's log about it is:
containerd[3905483]: time="2022-01-02T21:53:01.309515034+08:00" level=info msg="shim disconnected" id=6ce9d405aaae9dfae615d0810ba46b4726c60607dd940233fe824b651158372e
containerd[3905483]: time="2022-01-02T21:53:01.312121940+08:00" level=warning msg="cleaning up after shim disconnected" id=6ce9d405aaae9dfae615d0810ba46b4726c60607dd940233fe824b651158372e namespace=moby
And we can sure that the dockerd not stop or kill the container initiatively. It seems like that the shim exit first, but no more information about how it works abnormal.
Steps to reproduce the issue
Not find the accurate way to reproduce this issue. Kill containerd-shim-runc-v2 can product the same phenomenon .
- docker run -itd busybox:latest
- kill -s 9 ${containerd-shim-runc-v2_pid}
- docker ps
If use containerd-shim, there is no this problem.
Describe the results you received and expected
First, if the containerd-shim-runc-v2 process exit about the container, I think the containerd need to publish TaskExit and TaskDelete event to dockerd to clean this dead container.
What problems may cause containerd-shim-runc-v2 exit anomaly ? May be this is a system problem, because all the containers on the host had this issue at the same time.
What version of containerd are you using?
1.4.3
Any other relevant information
runc --version
runc version 1.0.1
uname -r
4.18.0-2.3.8
Show configuration if it is related to CRI plugin.
# Copyright 2018-2020 Docker Inc.
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
# http://www.apache.org/licenses/LICENSE-2.0
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
disabled_plugins = ["cri"]
#root = "/var/lib/containerd"
#state = "/run/containerd"
#subreaper = true
#oom_score = 0
#[grpc]
# address = "/run/containerd/containerd.sock"
# uid = 0
# gid = 0
#[debug]
# address = "/run/containerd/debug.sock"
# uid = 0
# gid = 0
# level = "info"