Skip to content

Update command causes 'Operation not permitted' #2204

@mYmNeo

Description

@mYmNeo

It's first seen in a kubernetes cluster with docker as container runtime. Our users reported that in some situation their bash script failed with message can't create /dev/null: Operation not permitted. But /dev/null is default device with permission rwm, After digging some logs, we found that it can be reproduced in runc by following steps.

  1. Run a runc container like "busybox". Suppose this container is called A
  2. run while true;do echo >/dev/null;done in container
  3. runc update --cpu-share 1024 A
  4. You will see sh: can't create /dev/null: Operation not permitted

The problem is located at https://github.com/opencontainers/runc/blob/master/libcontainer/cgroups/fs/devices.go#L28. By default, if runc wants to allow some devices, it'll first deny all devices then allow specific devices. It's OK when launching a new container, but if you update the resource of a container, it'll make a change that allowed devices becomes denied first, then allowed, during this period, user app will get xxx Operation not permitted error message.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions