Skip to content

shim v2 hangs on reboot/shutdown #5502

@zhangyoufu

Description

@zhangyoufu

Description

Containerd is shipping with KillMode=process in systemd unit, so that shims won't be killed if containerd stops.
Systemd broadcast SIGTERM during the final stage of shutdown, this is the only chance for shim to terminate gracefully.
It seems that shim v2 does not handle SIGTERM/SIGINT at all, and hangs machine reboot/shutdown for 90s, got killed at last.

Shim v1 handles SIGINT/SIGTERM

case unix.SIGTERM, unix.SIGINT:
go termOnce.Do(func() {
ctx := context.TODO()
if err := server.Shutdown(ctx); err != nil {
logger.WithError(err).Error("failed to shutdown server")
}
// Ensure our child is dead if any
sv.Kill(ctx, &shimapi.KillRequest{
Signal: uint32(syscall.SIGKILL),
All: true,
})
sv.Delete(context.Background(), &ptypes.Empty{})
close(done)
})

Shim v2 registers and ignores SIGINT/SIGTERM

switch s {
case unix.SIGCHLD:
if err := reaper.Reap(); err != nil {
logger.WithError(err).Error("reap exit status")
}
case unix.SIGPIPE:
}

See also:

Steps to reproduce the issue:

  1. install docker-ce 20.10.x, enable live-restore
  2. docker run -d k8s.gcr.io/pause
  3. sudo reboot

Describe the results you received:
The shutdown/reboot process stuck for 90s, due to containerd-shim.

[  OK  ] Reached target Shutdown.
[  OK  ] Reached target Final Step.
[  OK  ] Finished Reboot.
[  OK  ] Reached target Reboot.
[  214.337805] systemd-shutdown[1]: Waiting for process: containerd-shim

Describe the results you expected:
containerd-shim should not interfere with shutdown/reboot.

What version of containerd are you using:

$ containerd --version
containerd containerd.io 1.4.4 05f951a3781f4f2c1911b05e61c160e9c30eaa8e

Metadata

Metadata

Labels

Type

No type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions