[26.1 backport] apparmor: Allow confined runc to kill containers#47829
Merged
[26.1 backport] apparmor: Allow confined runc to kill containers#47829
Conversation
/usr/sbin/runc is confined with "runc" profile[1] introduced in AppArmor
v4.0.0. This change breaks stopping of containers, because the profile
assigned to containers doesn't accept signals from the "runc" peer.
AppArmor >= v4.0.0 is currently part of Ubuntu Mantic (23.10) and later.
In the case of Docker, this regression is hidden by the fact that
dockerd itself sends SIGKILL to the running container after runc fails
to stop it. It is still a regression, because graceful shutdowns of
containers via "docker stop" are no longer possible, as SIGTERM from
runc is not delivered to them. This can be seen in logs from dockerd
when run with debug logging enabled and also from tracing signals with
killsnoop utility from bcc[2] (in bpfcc-tools package in Debian/Ubuntu):
Test commands:
root@cloudimg:~# docker run -d --name test redis
ba04c137827df8468358c274bc719bf7fc291b1ed9acf4aaa128ccc52816fe46
root@cloudimg:~# docker stop test
Relevant syslog messages (with wrapped long lines):
Apr 23 20:45:26 cloudimg kernel: audit:
type=1400 audit(1713905126.444:253): apparmor="DENIED"
operation="signal" class="signal" profile="docker-default" pid=9289
comm="runc" requested_mask="receive" denied_mask="receive"
signal=kill peer="runc"
Apr 23 20:45:36 cloudimg dockerd[9030]:
time="2024-04-23T20:45:36.447016467Z"
level=warning msg="Container failed to exit within 10s of kill - trying direct SIGKILL"
container=ba04c137827df8468358c274bc719bf7fc291b1ed9acf4aaa128ccc52816fe46
error="context deadline exceeded"
Killsnoop output after "docker stop ...":
root@cloudimg:~# killsnoop-bpfcc
TIME PID COMM SIG TPID RESULT
20:51:00 9631 runc 3 9581 -13
20:51:02 9637 runc 9 9581 -13
20:51:12 9030 dockerd 9 9581 0
This change extends the docker-default profile with rules that allow
receiving signals from processes that run confined with either runc or
crun profile (crun[4] is an alternative OCI runtime that's also confined
in AppArmor >= v4.0.0, see [1]). It is backward compatible because the
peer value is a regular expression (AARE) so the referenced profile
doesn't have to exist for this profile to successfully compile and load.
Note that the runc profile has an attachment to /usr/sbin/runc. This is
the path where the runc package in Debian/Ubuntu puts the binary. When
the docker-ce package is installed from the upstream repository[3], runc
is installed as part of the containerd.io package at /usr/bin/runc.
Therefore it's still running unconfined and has no issues sending
signals to containers.
[1] https://gitlab.com/apparmor/apparmor/-/commit/2594d936
[2] https://github.com/iovisor/bcc/blob/master/tools/killsnoop.py
[3] https://download.docker.com/linux/ubuntu
[4] https://github.com/containers/crun
Signed-off-by: Tomáš Virtus <nechtom@gmail.com>
(cherry picked from commit 5ebe2c0)
Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>
renovate bot
added a commit
to earthly/dind
that referenced
this pull request
May 27, 2024
[](https://renovatebot.com) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [docker/docker](https://togithub.com/docker/docker) | patch | `26.1.2` -> `26.1.3` | --- ### Release Notes <details> <summary>docker/docker (docker/docker)</summary> ### [`v26.1.3`](https://togithub.com/moby/moby/releases/tag/v26.1.3) [Compare Source](https://togithub.com/docker/docker/compare/v26.1.2...v26.1.3) #### 26.1.3 For a full list of pull requests and changes in this release, refer to the relevant GitHub milestones: - [docker/cli, 26.1.3 milestone](https://togithub.com/docker/cli/issues?q=is%3Aclosed+milestone%3A26.1.3) - [moby/moby, 26.1.3 milestone](https://togithub.com/moby/moby/issues?q=is%3Aclosed+milestone%3A26.1.3) - Deprecated and removed features, see [Deprecated Features](https://togithub.com/docker/cli/blob/v26.1.3/docs/deprecated.md). - Changes to the Engine API, see [API version history](https://togithub.com/moby/moby/blob/v26.1.3/docs/api/version-history.md). ##### Bug fixes and enhancements - Fix a regression that prevented the use of DNS servers within a `--internal` network. [moby/moby#47832](https://togithub.com/moby/moby/pull/47832) - When the internal DNS server's own address is supplied as an external server address, ignore it to avoid unproductive recursion. [moby/moby#47833](https://togithub.com/moby/moby/pull/47833) ##### Packaging updates - Allow runc to kill containers when confined to the runc profile in AppArmor version 4.0.0 and later. [moby/moby#47829](https://togithub.com/moby/moby/pull/47829) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6am on monday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/earthly/dind). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zNzcuOCIsInVwZGF0ZWRJblZlciI6IjM3LjM3Ny44IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZSJdfQ==--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
renovate bot
added a commit
to earthly/dind
that referenced
this pull request
May 27, 2024
[](https://renovatebot.com) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [docker/docker](https://togithub.com/docker/docker) | patch | `26.1.2` -> `26.1.3` | --- ### Release Notes <details> <summary>docker/docker (docker/docker)</summary> ### [`v26.1.3`](https://togithub.com/moby/moby/releases/tag/v26.1.3) [Compare Source](https://togithub.com/docker/docker/compare/v26.1.2...v26.1.3) #### 26.1.3 For a full list of pull requests and changes in this release, refer to the relevant GitHub milestones: - [docker/cli, 26.1.3 milestone](https://togithub.com/docker/cli/issues?q=is%3Aclosed+milestone%3A26.1.3) - [moby/moby, 26.1.3 milestone](https://togithub.com/moby/moby/issues?q=is%3Aclosed+milestone%3A26.1.3) - Deprecated and removed features, see [Deprecated Features](https://togithub.com/docker/cli/blob/v26.1.3/docs/deprecated.md). - Changes to the Engine API, see [API version history](https://togithub.com/moby/moby/blob/v26.1.3/docs/api/version-history.md). ##### Bug fixes and enhancements - Fix a regression that prevented the use of DNS servers within a `--internal` network. [moby/moby#47832](https://togithub.com/moby/moby/pull/47832) - When the internal DNS server's own address is supplied as an external server address, ignore it to avoid unproductive recursion. [moby/moby#47833](https://togithub.com/moby/moby/pull/47833) ##### Packaging updates - Allow runc to kill containers when confined to the runc profile in AppArmor version 4.0.0 and later. [moby/moby#47829](https://togithub.com/moby/moby/pull/47829) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6am on monday" (UTC), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Mend Renovate](https://www.mend.io/free-developer-tools/renovate/). View repository job log [here](https://developer.mend.io/github/earthly/dind). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiIzNy4zNzcuOCIsInVwZGF0ZWRJblZlciI6IjM3LjM3Ny44IiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJyZW5vdmF0ZSJdfQ==--> Co-authored-by: renovate[bot] <29139614+renovate[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
/usr/sbin/runc is confined with "runc" profile[1] introduced in AppArmor
v4.0.0. This change breaks stopping of containers, because the profile
assigned to containers doesn't accept signals from the "runc" peer.
AppArmor >= v4.0.0 is currently part of Ubuntu Mantic (23.10) and later.
In the case of Docker, this regression is hidden by the fact that
dockerd itself sends SIGKILL to the running container after runc fails
to stop it. It is still a regression, because graceful shutdowns of
containers via "docker stop" are no longer possible, as SIGTERM from
runc is not delivered to them. This can be seen in logs from dockerd
when run with debug logging enabled and also from tracing signals with
killsnoop utility from bcc[2] (in bpfcc-tools package in Debian/Ubuntu):
Test commands:
Relevant syslog messages (with wrapped long lines):
Killsnoop output after "docker stop ...":
This change extends the docker-default profile with rules that allow
receiving signals from processes that run confined with either runc or
crun profile (crun[4] is an alternative OCI runtime that's also confined
in AppArmor >= v4.0.0, see [1]). It is backward compatible because the
peer value is a regular expression (AARE) so the referenced profile
doesn't have to exist for this profile to successfully compile and load.
Note that the runc profile has an attachment to /usr/sbin/runc. This is
the path where the runc package in Debian/Ubuntu puts the binary. When
the docker-ce package is installed from the upstream repository[3], runc
is installed as part of the containerd.io package at /usr/bin/runc.
Therefore it's still running unconfined and has no issues sending
signals to containers.
[1] https://gitlab.com/apparmor/apparmor/-/commit/2594d936
[2] https://github.com/iovisor/bcc/blob/master/tools/killsnoop.py
[3] https://download.docker.com/linux/ubuntu
[4] https://github.com/containers/crun
Signed-off-by: Tomáš Virtus nechtom@gmail.com
- Description for the changelog