Enable Writable cgroups for unprivileged containers #11131

Divya063 · 2024-12-10T12:16:06Z

Currently, cgroupfs is only writable when containers are running in privileged mode. For unprivileged containers with cgroup v2 enabled, the cgroup interface (/sys/fs/cgroup) is mounted read-only by default, preventing containers from managing their own cgroup hierarchies. This PR enables the support for writable cgroups in unprivileged containers via CgroupWritable field.

Fixes #10924

k8s-ci-robot · 2024-12-10T12:16:17Z

Hi @Divya063. Thanks for your PR.

I'm waiting for a containerd member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

internal/cri/config/config.go

internal/cri/opts/spec_linux_opts.go

internal/cri/server/container_create_linux_test.go

dentiny

Curious, what's the interface for the writable cgroup feature?
For the cri-o implementation PR, the author exposes the functionality via annotation.

internal/cri/config/config.go

Currently, cgroupfs is only writable when containers are running in privileged mode. For unprivileged containers with cgroup v2 enabled, the cgroup interface (/sys/fs/cgroup) is mounted read-only by default, preventing containers from managing their own cgroup hierarchies. This PR enables the support for writable cgroups in unprivileged containers via CgroupWritable field. Signed-off-by: Divya <ranidivya063@gmail.com>

Divya063 · 2024-12-17T08:29:20Z

Curious, what's the interface for the writable cgroup feature? For the cri-o implementation PR, the author exposes the functionality via annotation.

@dentiny We are exposing through the containerd config using the field CgroupWritable- discussion here

Divya063 · 2024-12-19T11:42:58Z

/retest

k8s-ci-robot · 2024-12-19T11:43:13Z

@Divya063: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

Details

In response to this:

/retest

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

Divya063 · 2024-12-19T12:13:51Z

The code looks good, but needs an integration test

Thanks @AkihiroSuda! I have added the integration test.

akhilerm · 2024-12-19T12:21:55Z

/ok-to-test

Divya063 · 2024-12-19T12:46:14Z

/retest

Divya063 · 2024-12-19T13:17:06Z

/retest

djdongjin

LGTM, thanks

AkihiroSuda

Thanks

…eded for user-namespaced systemd in Kubernetes containers. Implemented via containerd/containerd/pull/11131.

which added support for writable cgroups via containerd/containerd#11131.

containerd 2.1.0 Welcome to the v2.1.0 release of containerd! The first minor release of containerd 2.x focuses on continued stability alongside new features and improvements. This is the first time-based released for containerd. Most the feature set and core functionality has long been stable and harderened in production environments, so now we transition to a balance of timely delivery of new functionality with the same high confidence in stability and performance. * Add no_sync option to boost boltDB performance on ephemeral environments ([containerd#10745](containerd#10745)) * Add content create event ([containerd#11006](containerd#11006)) * Erofs snapshotter and differ ([containerd#10705](containerd#10705)) * Update CRI to use transfer service for image pull by default ([containerd#8515](containerd#8515)) * Support multiple cni plugin bin dirs ([containerd#11311](containerd#11311)) * Support container restore through CRI/Kubernetes ([containerd#10365](containerd#10365)) * Add OCI/Image Volume Source support ([containerd#10579](containerd#10579)) * Enable Writable cgroups for unprivileged containers ([containerd#11131](containerd#11131)) * Fix recursive RLock() mutex acquisition ([containerd/go-cni#126](containerd/go-cni#126)) * Support CNI STATUS Verb ([containerd/go-cni#123](containerd/go-cni#123)) * Retry last registry host on 50x responses ([containerd#11484](containerd#11484)) * Multipart layer fetch ([containerd#10177](containerd#10177)) * Enable HTTP debug and trace for transfer based puller ([containerd#10762](containerd#10762)) * Add support for unpacking custom media types ([containerd#11744](containerd#11744)) * Add dial timeout field to hosts toml configuration ([containerd#11106](containerd#11106)) * Expose Pod assigned IPs to NRI plugins ([containerd#10921](containerd#10921)) * Support multiple uid/gid mappings ([containerd#10722](containerd#10722)) * Fix race between serve and immediate shutdown on the server ([containerd/ttrpc#175](containerd/ttrpc#175)) * Update FreeBSD defaults and re-organize platform defaults ([containerd#11017](containerd#11017)) * Postpone cri config deprecations to v2.2 ([containerd#11684](containerd#11684)) * Remove deprecated dynamic library plugins ([containerd#11683](containerd#11683)) * Remove the support for Schema 1 images ([containerd#11681](containerd#11681)) Please try out the release binaries and report any issues at https://github.com/containerd/containerd/issues. * Derek McGowan * Phil Estes * Akihiro Suda * Maksym Pavlenko * Jin Dong * Wei Fu * Sebastiaan van Stijn * Samuel Karp * Mike Brown * Adrien Delorme * Austin Vazquez * Akhil Mohan * Kazuyoshi Kato * Henry Wang * Gao Xiang * ningmingxiao * Krisztian Litkey * Yang Yang * Archit Kulkarni * Chris Henzie * Iceber Gu * Alexey Lunev * Antonio Ojea * Davanum Srinivas * Marat Radchenko * Michael Zappa * Paweł Gronowski * Rodrigo Campos * Alberto Garcia Hierro * Amit Barve * Andrey Smirnov * Divya * Etienne Champetier * Kirtana Ashok * Philip Laine * QiPing Wan * fengwei0328 * zounengren * Adrian Reber * Alfred Wingate * Amal Thundiyil * Athos Ribeiro * Brian Goff * Cesar Talledo * ChengyuZhu6 * Chongyi Zheng * Craig Ingram * Danny Canter * David Son * Fupan Li * HirazawaUi * Jing Xu * Jonathan A. Sternberg * Jose Fernandez * Kaita Nakamura * Kohei Tokunaga * Lei Liu * Marco Visin * Mike Baynton * Qiyuan Liang * Sameer * Shiming Zhang * Swagat Bora * Teresaliu * Tony Fang * Tõnis Tiigi * Vered Rosen * Vinayak Goyal * bo.jiang * chriskery * luchenhan * mahmut * zhaixiaojuan * **github.com/Microsoft/hcsshim** v0.12.9 -> v0.13.0-rc.3 * **github.com/cilium/ebpf** v0.11.0 -> v0.16.0 * **github.com/containerd/cgroups/v3** v3.0.3 -> v3.0.5 * **github.com/containerd/containerd/api** v1.8.0 -> v1.9.0 * **github.com/containerd/continuity** v0.4.4 -> v0.4.5 * **github.com/containerd/go-cni** v1.1.10 -> v1.1.12 * **github.com/containerd/imgcrypt/v2** v2.0.0-rc.1 -> v2.0.1 * **github.com/containerd/otelttrpc** ea5083fda723 -> v0.1.0 * **github.com/containerd/platforms** v1.0.0-rc.0 -> v1.0.0-rc.1 * **github.com/containerd/ttrpc** v1.2.6 -> v1.2.7 * **github.com/containerd/typeurl/v2** v2.2.2 -> v2.2.3 * **github.com/containernetworking/cni** v1.2.3 -> v1.3.0 * **github.com/containernetworking/plugins** v1.5.1 -> v1.7.1 * **github.com/containers/ocicrypt** v1.2.0 -> v1.2.1 * **github.com/davecgh/go-spew** d8f796af33cc -> v1.1.1 * **github.com/fsnotify/fsnotify** v1.7.0 -> v1.9.0 * **github.com/go-jose/go-jose/v4** v4.0.4 -> v4.0.5 * **github.com/google/go-cmp** v0.6.0 -> v0.7.0 * **github.com/grpc-ecosystem/grpc-gateway/v2** v2.22.0 -> v2.26.1 * **github.com/klauspost/compress** v1.17.11 -> v1.18.0 * **github.com/mdlayher/socket** v0.4.1 -> v0.5.1 * **github.com/moby/spdystream** v0.4.0 -> v0.5.0 * **github.com/moby/sys/user** v0.3.0 -> v0.4.0 * **github.com/opencontainers/image-spec** v1.1.0 -> v1.1.1 * **github.com/opencontainers/runtime-spec** v1.2.0 -> v1.2.1 * **github.com/opencontainers/selinux** v1.11.1 -> v1.12.0 * **github.com/pelletier/go-toml/v2** v2.2.3 -> v2.2.4 * **github.com/petermattis/goid** 4fcff4a6cae7 **_new_** * **github.com/pmezard/go-difflib** 5d4384ee4fb2 -> v1.0.0 * **github.com/prometheus/client_golang** v1.20.5 -> v1.22.0 * **github.com/prometheus/common** v0.55.0 -> v0.62.0 * **github.com/sasha-s/go-deadlock** v0.3.5 **_new_** * **github.com/smallstep/pkcs7** v0.1.1 **_new_** * **github.com/stretchr/testify** v1.9.0 -> v1.10.0 * **github.com/tchap/go-patricia/v2** v2.3.1 -> v2.3.2 * **github.com/urfave/cli/v2** v2.27.5 -> v2.27.6 * **github.com/vishvananda/netlink** v1.3.0 -> 0e7078ed04c8 * **github.com/vishvananda/netns** v0.0.4 -> v0.0.5 * **go.etcd.io/bbolt** v1.3.11 -> v1.4.0 * **go.opentelemetry.io/auto/sdk** v1.1.0 **_new_** * **go.opentelemetry.io/contrib/instrumentation/google.golang.org/grpc/otelgrpc** v0.56.0 -> v0.60.0 * **go.opentelemetry.io/contrib/instrumentation/net/http/otelhttp** v0.56.0 -> v0.60.0 * **go.opentelemetry.io/otel** v1.31.0 -> v1.35.0 * **go.opentelemetry.io/otel/exporters/otlp/otlptrace** v1.31.0 -> v1.35.0 * **go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracegrpc** v1.31.0 -> v1.35.0 * **go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp** v1.31.0 -> v1.35.0 * **go.opentelemetry.io/otel/metric** v1.31.0 -> v1.35.0 * **go.opentelemetry.io/otel/sdk** v1.31.0 -> v1.35.0 * **go.opentelemetry.io/otel/trace** v1.31.0 -> v1.35.0 * **go.opentelemetry.io/proto/otlp** v1.3.1 -> v1.5.0 * **golang.org/x/crypto** v0.28.0 -> v0.36.0 * **golang.org/x/exp** aacd6d4b4611 -> 2d47ceb2692f * **golang.org/x/mod** v0.21.0 -> v0.24.0 * **golang.org/x/net** v0.30.0 -> v0.38.0 * **golang.org/x/oauth2** v0.22.0 -> v0.27.0 * **golang.org/x/sync** v0.8.0 -> v0.14.0 * **golang.org/x/sys** v0.26.0 -> v0.33.0 * **golang.org/x/term** v0.25.0 -> v0.30.0 * **golang.org/x/text** v0.19.0 -> v0.23.0 * **golang.org/x/time** v0.3.0 -> v0.7.0 * **google.golang.org/genproto/googleapis/api** 5fefd90f89a9 -> 56aae31c358a * **google.golang.org/genproto/googleapis/rpc** 324edc3d5d38 -> 56aae31c358a * **google.golang.org/grpc** v1.67.1 -> v1.72.0 * **google.golang.org/protobuf** v1.35.1 -> v1.36.6 * **k8s.io/api** v0.31.2 -> v0.32.3 * **k8s.io/apimachinery** v0.31.2 -> v0.32.3 * **k8s.io/apiserver** v0.31.2 -> v0.32.3 * **k8s.io/client-go** v0.31.2 -> v0.32.3 * **k8s.io/cri-api** v0.31.2 -> v0.32.3 * **k8s.io/kubelet** v0.31.2 -> v0.32.3 * **k8s.io/utils** 18e509b52bc8 -> 3ea5e8cea738 * **sigs.k8s.io/json** bc3834ca7abd -> 9aa6b5e7a4b3 * **sigs.k8s.io/structured-merge-diff/v4** v4.4.1 -> v4.4.2 * **tags.cncf.io/container-device-interface** v0.8.0 -> v1.0.1 * **tags.cncf.io/container-device-interface/specs-go** v0.8.0 -> v1.0.0 Previous release can be found at [v2.0.0](https://github.com/containerd/containerd/releases/tag/v2.0.0) * `containerd-<VERSION>-<OS>-<ARCH>.tar.gz`: ✅Recommended. Dynamically linked with glibc 2.35 (Ubuntu 22.04). * `containerd-static-<VERSION>-<OS>-<ARCH>.tar.gz`: Statically linked. Expected to be used on Linux distributions that do not use glibc >= 2.35. Not position-independent. In addition to containerd, typically you will have to install [runc](https://github.com/opencontainers/runc/releases) and [CNI plugins](https://github.com/containernetworking/plugins/releases) from their official sites too. See also the [Getting Started](https://github.com/containerd/containerd/blob/main/docs/getting-started.md) documentation.

k8s-ci-robot added do-not-merge/work-in-progress needs-ok-to-test size/S labels Dec 10, 2024

AkihiroSuda reviewed Dec 10, 2024

View reviewed changes

internal/cri/config/config.go Show resolved Hide resolved

AkihiroSuda reviewed Dec 10, 2024

View reviewed changes

internal/cri/config/config.go Show resolved Hide resolved

Divya063 force-pushed the rw-cgroups branch from cfcf5a1 to 9df5356 Compare December 12, 2024 18:06

k8s-ci-robot added size/M and removed size/S labels Dec 12, 2024

Divya063 force-pushed the rw-cgroups branch from 9df5356 to 4c3e57f Compare December 12, 2024 18:52

k8s-ci-robot added size/S and removed size/M labels Dec 12, 2024

Divya063 force-pushed the rw-cgroups branch 2 times, most recently from d98974c to 5d3afcf Compare December 16, 2024 03:56

Divya063 marked this pull request as ready for review December 16, 2024 04:11

k8s-ci-robot removed the do-not-merge/work-in-progress label Dec 16, 2024

dosubot bot added area/cri Container Runtime Interface (CRI) kind/feature labels Dec 16, 2024

Divya063 force-pushed the rw-cgroups branch from 5d3afcf to 0aef239 Compare December 16, 2024 05:17

k8s-ci-robot added size/M and removed size/S labels Dec 16, 2024

Divya063 requested review from AkihiroSuda and djdongjin December 16, 2024 05:51

djdongjin reviewed Dec 16, 2024

View reviewed changes

internal/cri/opts/spec_linux_opts.go Outdated Show resolved Hide resolved

internal/cri/opts/spec_linux_opts.go Show resolved Hide resolved

dentiny reviewed Dec 16, 2024

View reviewed changes

internal/cri/server/container_create_linux_test.go Outdated Show resolved Hide resolved

dentiny reviewed Dec 16, 2024

View reviewed changes

internal/cri/config/config.go Show resolved Hide resolved

Divya063 force-pushed the rw-cgroups branch from 0aef239 to dda7020 Compare December 17, 2024 08:25

k8s-ci-robot added ok-to-test and removed needs-ok-to-test labels Dec 19, 2024

djdongjin approved these changes Dec 19, 2024

View reviewed changes

AkihiroSuda approved these changes Jan 2, 2025

View reviewed changes

AkihiroSuda requested a review from dmcgowan January 2, 2025 18:10

dmcgowan approved these changes Jan 8, 2025

View reviewed changes

dmcgowan added this pull request to the merge queue Jan 8, 2025

AkihiroSuda added this to the 2.1 milestone Jan 8, 2025

AkihiroSuda added the impact/changelog label Jan 8, 2025

Merged via the queue into containerd:main with commit 7c380b9 Jan 8, 2025
59 checks passed

AkihiroSuda mentioned this pull request Feb 26, 2025

rootless: update docs and examples moby/buildkit#5765

Merged

adelton mentioned this pull request Feb 26, 2025

[fedora-41-4.12.2] slapd not listens on 636 freeipa/freeipa-container#657

Closed

dmcgowan mentioned this pull request Mar 8, 2025

Prepare release notes for v2.1.0-beta.0 #11510

Merged

dmcgowan mentioned this pull request Apr 23, 2025

Add release notes for v2.1.0-rc.0 #11752

Merged

dmcgowan mentioned this pull request May 7, 2025

Prepare release notes for v2.1.0 #11819

Merged

adelton added a commit to adelton/freeipa-container that referenced this pull request May 13, 2025

Use containerd 2.1 as it contains the support for cgroup_writable, ne…

d3560d7

…eded for user-namespaced systemd in Kubernetes containers. Implemented via containerd/containerd/pull/11131.

adelton added a commit to adelton/freeipa-container that referenced this pull request May 14, 2025

User-namespaced systemd containers are possible with containerd 2.1

0728a38

which added support for writable cgroups via containerd/containerd#11131.

adelton added a commit to adelton/freeipa-container that referenced this pull request May 14, 2025

User-namespaced systemd containers are possible with containerd 2.1

6af0065

which added support for writable cgroups via containerd/containerd#11131.

rata mentioned this pull request Aug 7, 2025

Writable cgroups doesn't work with userns #12182

Closed

Divya063 mentioned this pull request Aug 21, 2025

Add documentation for cgroup_writable field #12227

Merged

KCSesh mentioned this pull request Aug 21, 2025

Add package: containerd-2.1 bottlerocket-os/bottlerocket-core-kit#621

Merged

z0rc mentioned this pull request Oct 21, 2025

Expose cgroup_writable in API bottlerocket-os/bottlerocket#4666

Open

Enable Writable cgroups for unprivileged containers #11131

Enable Writable cgroups for unprivileged containers #11131

Uh oh!

Conversation

Divya063 commented Dec 10, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

k8s-ci-robot commented Dec 10, 2024

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

dentiny left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Divya063 commented Dec 17, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Divya063 commented Dec 19, 2024

Uh oh!

k8s-ci-robot commented Dec 19, 2024

Uh oh!

Divya063 commented Dec 19, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

akhilerm commented Dec 19, 2024

Uh oh!

Divya063 commented Dec 19, 2024

Uh oh!

Divya063 commented Dec 19, 2024

Uh oh!

djdongjin left a comment

Choose a reason for hiding this comment

Uh oh!

AkihiroSuda left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

Divya063 commented Dec 10, 2024 •

edited

Loading

Divya063 commented Dec 17, 2024 •

edited

Loading

Divya063 commented Dec 19, 2024 •

edited

Loading