support cgroup2 by AkihiroSuda · Pull Request #40174 · moby/moby

AkihiroSuda · 2019-11-05T09:22:55Z

- What I did
support cgroup2 (unified mode)

Fix #16238

- How I did it

daemon & libcontainerd: Use shim v2 when running with cgroup2 (support cgroup2 containerd/containerd#3799)
pkg/sysinfo: shut up v1 warnings (We should add v2 warnings later)
daemon: Use cgroupns=private by default (cgroupns=private should be enabled by default for unified mode? containers/podman#4363)

- How to verify it

Install Ubuntu 19.10 (kernel 5.3). Fedora 31 and other recent distros with kernel >= 5.2 should work as well.
Add systemd.unified_cgroup_hierarchy=1 to kernel cmdline and reboot. This step is not needed for Fedora 31.
Install containerd from master
Install crun v0.10.4 https://github.com/containers/crun/tree/0.10.4 , or runc with cgroup2: split fs2 from fs opencontainers/runc#2169
docker run -it --rm --runtime=crun --security-opt seccomp=unconfined alpine

Limitations:

Metrics are unimplemented yet
crun requires --security-opt seccomp=unconfined unless using master version of libseccomp ( invalid seccomp syscall 'io_uring_enter': unknown. containers/crun#156, RFE: new release? seccomp/libseccomp#177 )
Doesn't work with runc yet (cgroup2: split fs2 from fs opencontainers/runc#2169)
Rootless mode still doesn't support resource limitation even with --exec-opt native.cgroupdriver=systemd.

- Description for the changelog

support cgroup2

- A picture of a cute animal (not mandatory but encouraged)
🐧

AkihiroSuda · 2019-12-13T06:45:01Z

containerd/containerd#3799 is merged, but we still need a couple of days to stabilize the interface

thaJeztah · 2019-12-13T11:53:08Z

We should look at adding a stage to the Jenkinsfile that runs on a machine with cgroup2 enabled

AkihiroSuda · 2019-12-17T06:26:46Z

Maybe we can merge this PR as-is and work on metrics in separate PRs

thaJeztah · 2019-12-17T09:18:13Z

Maybe we can merge this PR as-is and work on metrics in separate PRs

Do we need a more recent version of containerd and run to actually use this? (with containerd/containerd#3799 in it?)

We should look at adding a stage to the Jenkinsfile that runs on a machine with cgroup2 enabled

^ what would be needed for that? Probably at least a host that runs a more recent version of Ubuntu/Debian/Fedora, but given that we run the tests DIND, do we need to update the base-image or a separate stage for that?

thaJeztah · 2019-12-17T09:18:29Z

@dmcgowan @crosbymichael @cpuguy83 ptal

AkihiroSuda · 2019-12-17T09:31:32Z

Do we need a more recent version of containerd and run to actually use this? (with containerd/containerd#3799 in it?)

We need containerd binary built from master, but no need to vendor containerd library at this moment.
Probably we need to revendor containerd when we implement metrics. (Blocked due to vendor hell moby/buildkit#1293)

^ what would be needed for that? Probably at least a host that runs a more recent version of Ubuntu/Debian/Fedora, but given that we run the tests DIND, do we need to update the base-image or a separate stage for that?

Maybe we should wait for adding cgroup v2 CI until we finish basic v2 support.

Or we can use Podman instead, but not sure it is acceptable 😛 https://github.com/opencontainers/runc/blob/201b06374548b64212f4ceb1529688d435e42899/.travis.yml#L29

* Requires containerd binaries from containerd/containerd#3799 . Metrics are unimplemented yet. * Works with crun v0.10.4, but `--security-opt seccomp=unconfined` is needed unless using master version of libseccomp ( containers/crun#156, seccomp/libseccomp#177 ) * Doesn't work with master runc yet * Resource limitations are unimplemented Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>

enable resource limitation by disabling cgroup v1 warnings resource limitation still doesn't work with rootless mode (even with systemd mode) Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>

For cgroup v1, we were unable to change the default because of compatibility issue. For cgroup v2, we should change the default right now because switching to cgroup v2 is already breaking change. See also containers/podman#4363 containers/podman#4374 Privileged containers also use cgroupns=private by default. containers/podman#4374 (comment) Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>

AkihiroSuda · 2020-01-09T02:46:29Z

@thaJeztah PTAL?

In cgroup v1 container implementations, cgroupns is not used by default because it was not available in the kernel until kernel 4.6 (May 2016), and the default behavior will not change on cgroup v1 environments, because changing the default will break compatibility and surprise users. For cgroup v2, implementations are going to unshare cgroupns by default so as to hide /sys/fs/cgroup from containers. * Discussion: containers/podman#4363 * Podman PR (merged): containers/podman#4374 * Moby PR: moby/moby#40174 This PR enables cgroupns for containers, but pod sandboxes are untouched because probably there is no need to do. Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>

joemccall86 · 2020-01-09T14:25:03Z

@AkihiroSuda I think it means, "Please Take A Look."

AkihiroSuda · 2020-01-09T15:14:30Z

Yes, so, I'm asking thaJeztah to PTAL whether we can merge this PR without including metrics stuff 😅

cpuguy83

LGTM

Tested with Fedora 31 and a master build of containerd and runc.

thaJeztah

LGTM

we should create a tracking issue for what's needed to run CI for this

AkihiroSuda · 2020-01-10T03:08:45Z

tracking issue: #40360

…6654) ### What does this PR do? Adds `cgroup: "host"` to the host-profiler service in the dev docker-compose, so the profiler gets the host cgroup namespace instead of a private one. ### Motivation On cgroup v2, Docker keeps private cgroup namespaces even for privileged containers ([moby#40174](moby/moby#40174)). This causes `/proc/self/cgroup` to return `0::/`, which breaks self container ID detection. Profiles of the profiler itself end up missing container tags. ### Describe how you validated your changes Run `docker compose up --build` from `cmd/host-profiler/` and verify the profiler resolves its own container ID. ### Additional Notes This only affects the dev docker-compose. For customer-facing Docker deployments, we should add `--cgroupns=host` to the recommended run configuration in the docs. Co-authored-by: nayef.ghattas <nayef.ghattas@datadoghq.com>

AkihiroSuda requested a review from cpuguy83 as a code owner November 5, 2019 09:22

This was referenced Nov 5, 2019

Docker does not run with unified cgroup hierarchy #16238

Closed

cgroup2: procHooks: failed to load program: operation not permitted opencontainers/runc#2167

Closed

AkihiroSuda changed the title ~~support cgroup2~~ [DNM] support cgroup2 Nov 5, 2019

AkihiroSuda force-pushed the cgroup2 branch 3 times, most recently from 038a308 to c424f8f Compare November 6, 2019 05:36

AkihiroSuda mentioned this pull request Nov 6, 2019

Please provide repo for docker-ce on Fedora 31 docker/for-linux#665

Open

3 tasks

AkihiroSuda force-pushed the cgroup2 branch from c424f8f to 576fcfc Compare November 6, 2019 06:34

AkihiroSuda mentioned this pull request Dec 12, 2019

Add support for unified cgroups (cgroup2) #25868

Closed

thaJeztah added area/runtime Runtime impact/changelog status/2-code-review labels Dec 13, 2019

AkihiroSuda force-pushed the cgroup2 branch from 576fcfc to 7e8f5bf Compare December 17, 2019 06:23

AkihiroSuda changed the title ~~[DNM] support cgroup2~~ support cgroup2 Dec 17, 2019

AkihiroSuda mentioned this pull request Dec 17, 2019

Rootless Docker/containerd leaks file descriptors #40258

Closed

AkihiroSuda force-pushed the cgroup2 branch from 7e8f5bf to 14e0db2 Compare December 31, 2019 07:23

AkihiroSuda added 4 commits January 1, 2020 02:58

cgroup2: enable resource limitation

409bbdc

enable resource limitation by disabling cgroup v1 warnings resource limitation still doesn't work with rootless mode (even with systemd mode) Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>

cgroup2: mark cpu-rt-{period,runtime} unimplemented

491531c

Signed-off-by: Akihiro Suda <akihiro.suda.cz@hco.ntt.co.jp>

AkihiroSuda force-pushed the cgroup2 branch from 14e0db2 to 491531c Compare December 31, 2019 17:58

AkihiroSuda mentioned this pull request Jan 9, 2020

cgroup2: unshare cgroup namespace for containers containerd/cri#1371

Merged

cpuguy83 approved these changes Jan 9, 2020

View reviewed changes

thaJeztah approved these changes Jan 9, 2020

View reviewed changes

thaJeztah merged commit e6c1820 into moby:master Jan 9, 2020

AkihiroSuda mentioned this pull request Jan 10, 2020

cgroup2 check list #40360

Closed

8 tasks

abbra mentioned this pull request Jan 14, 2020

Failed to set devices.allow freeipa/freeipa-container#306

Closed

thaJeztah added this to the 20.03.0 milestone Apr 2, 2020

junaruga mentioned this pull request Jul 30, 2020

Change outdated *make* syntax FNNDSC/ChRIS_ultron_backEnd#213

Merged

AkihiroSuda added the area/cgroup2 cgroup v2 label Jan 23, 2021

xruins mentioned this pull request May 20, 2022

Missing some fields to support Stats API on Docker host uses cgroup2 fsouza/go-dockerclient#919

Closed

paologallinaharbur mentioned this pull request May 1, 2024

[ECS] [request] Support cgroupns Docker flag aws/containers-roadmap#2346

Open

Gandem mentioned this pull request Feb 18, 2026

host-profiler: set cgroup namespace to host in dev docker-compose DataDog/datadog-agent#46654

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

support cgroup2#40174

support cgroup2#40174
thaJeztah merged 4 commits intomoby:masterfrom
AkihiroSuda:cgroup2

AkihiroSuda commented Nov 5, 2019 •

edited

Loading

Uh oh!

AkihiroSuda commented Dec 13, 2019

Uh oh!

thaJeztah commented Dec 13, 2019

Uh oh!

AkihiroSuda commented Dec 17, 2019

Uh oh!

thaJeztah commented Dec 17, 2019

Uh oh!

thaJeztah commented Dec 17, 2019

Uh oh!

AkihiroSuda commented Dec 17, 2019

Uh oh!

AkihiroSuda commented Jan 9, 2020

Uh oh!

joemccall86 commented Jan 9, 2020

Uh oh!

AkihiroSuda commented Jan 9, 2020

Uh oh!

cpuguy83 left a comment

Uh oh!

thaJeztah left a comment

Uh oh!

AkihiroSuda commented Jan 10, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

AkihiroSuda commented Nov 5, 2019 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AkihiroSuda commented Dec 13, 2019

Uh oh!

thaJeztah commented Dec 13, 2019

Uh oh!

AkihiroSuda commented Dec 17, 2019

Uh oh!

thaJeztah commented Dec 17, 2019

Uh oh!

thaJeztah commented Dec 17, 2019

Uh oh!

AkihiroSuda commented Dec 17, 2019

Uh oh!

AkihiroSuda commented Jan 9, 2020

Uh oh!

joemccall86 commented Jan 9, 2020

Uh oh!

AkihiroSuda commented Jan 9, 2020

Uh oh!

cpuguy83 left a comment

Choose a reason for hiding this comment

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

AkihiroSuda commented Jan 10, 2020

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

AkihiroSuda commented Nov 5, 2019 •

edited

Loading