Add default sysctls to allow ping sockets and privileged ports with no capabilities #41030

justincormack · 2020-05-26T15:07:29Z

Currently default capability CAP_NET_RAW allows users to open ICMP echo
sockets, and CAP_NET_BIND_SERVICE allows binding to ports under 1024.
Both of these are safe operations, and Linux now provides ways that
these can be set, per container, to be allowed without any capabilties
for non root users. Enable these by default. Users can revert to the
previous behaviour by overriding the sysctl values explicitly.

Signed-off-by: Justin Cormack justin.cormack@docker.com

(picture by https://twitter.com/CountrysideBen)

justincormack · 2020-05-26T16:14:47Z

Seems to be erroring in user namespace, need to exclude this case apparently.

thaJeztah · 2020-05-26T17:58:46Z

added "fixes #8460" to the description

@tianon this one is for you! 🤗

thaJeztah · 2020-05-26T18:01:09Z

daemon/oci_linux.go

+		s.Linux.Sysctl["net.ipv4.ping_group_range"] = "0 2147483647"
+		// allow opening any port less than 1024 without CAP_NET_BIND_SERVICE
+		s.Linux.Sysctl["net.ipv4.ip_unprivileged_port_start"] = "0"


Are custom values already set at this point? If so, should we skip these if a custom value was set? (not sure if someone would have any reason to)

@yosifkit looking over my shoulder while I'm cheering for this glorious PR also asked if one or both of these should be specifically excluded when using --network=host? (it's not clear from the code if this is already the case 🙈)

Good point! 👍 (not sure what it does when manually specifying --sysctl 🤔

Looks like it "works" just fine? 😅

$ docker run --init -it --rm --network host --sysctl net.ipv4.ip_unprivileged_port_start=0 --user 1000:1000 tianon/network-toolbox nc -vvv -l -p 123 Listening on [0.0.0.0] (family 2, port 123)

... and persistently adjusts the value for my host: 🥳

$ cat /proc/sys/net/ipv4/ip_unprivileged_port_start 0

did it change your host's configuration? 😬

LOL; race condition in comments.

Looks like that's a .... "feature"?

Ah yes, better exclude net=host too...

As a follow-up on this thread, @kolyshkin separately fixed the "--sysctl=net.* + --net=host changes the host's settings" part of this discussion in runc itself via opencontainers/runc#2706 (issue opencontainers/runc#2693). 😄 🎉

$ docker run --init -it --rm --network host --sysctl net.ipv4.ip_unprivileged_port_start=0 --user 1000:1000 tianon/network-toolbox nc -vvv -l -p 123 docker: Error response from daemon: OCI runtime create failed: sysctl "net.ipv4.ip_unprivileged_port_start" not allowed in host network namespace: unknown.

thaJeztah · 2020-05-28T07:45:46Z

Looks like userns is still failing;


[2020-05-27T19:14:03.024Z] === FAIL: amd64.integration-cli TestDockerDaemonSuite/TestDaemonUserNamespaceRootSetting (1.70s)
[2020-05-27T19:14:03.024Z]     --- FAIL: TestDockerDaemonSuite/TestDaemonUserNamespaceRootSetting (1.70s)
[2020-05-27T19:14:03.024Z]         docker_cli_userns_test.go:50: assertion failed: error is not nil: exit status 125: Output: df409cf62ef426ddc2fc30b44eb45c79203b4a2c20ab46b9e587389f9cb5095d
[2020-05-27T19:14:03.024Z]             /usr/local/cli/docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"write sysctl key net.ipv4.ping_group_range: write /proc/sys/net/ipv4/ping_group_range: invalid argument\"": unknown.

Could that be because of the order in which options are applied (both WithSysctls and WithNamespaces are applied after this)?

moby/daemon/oci_linux.go

Lines 983 to 999 in 07e6b84

    
           opts = append(opts, 
        
           	WithCommonOptions(daemon, c), 
        
           	WithCgroups(daemon, c), 
        
           	WithResources(c), 
        
           	WithSysctls(c), 
        
           	WithDevices(daemon, c), 
        
           	WithUser(c), 
        
           	WithRlimits(daemon, c), 
        
           	WithNamespaces(daemon, c), 
        
           	WithCapabilities(c), 
        
           	WithSeccomp(daemon, c), 
        
           	WithMounts(daemon, c), 
        
           	WithLibnetwork(daemon, c), 
        
           	WithApparmor(c), 
        
           	WithSelinux(c), 
        
           	WithOOMScore(&c.HostConfig.OomScoreAdj), 
        
           )

Or is it something in how those tests are setup?

thaJeztah · 2020-05-28T07:47:49Z

Also wondering if we should set the defaults inside WithSysctls 🤔 (might possibly be clearer, although there's something to be said for having the defaults in a central place)

justincormack · 2020-05-28T09:47:52Z

I think WithSysctls is designed to be a generic method so we should not set defaults there. There was a typo in the userns maybe this will work. I also need to check rootless mode, and a couple of other tweaks (need to support old kernels without the unpriv support I think).

thaJeztah · 2020-05-28T11:07:55Z

I think WithSysctls is designed to be a generic method so we should not set defaults there.

Yes, agreed.

Should we make WithSysctls return an error if the container is using host networking, and Linux.Sysctl is non-empty? (I'd have to check the flow; unfortunately our "validation" is handled in various places, so not sure if that would work / be the best place; also may be out of scope for this PR)

justincormack · 2020-05-28T13:23:00Z

For host networking, it would be all sysctls starting with "net.*" that should be blocked. Thats independent of this PR but I think it does make sense. Maybe it should happen in runc though, which is where non namespaced sysctls are blocked now?

thaJeztah · 2020-06-04T11:52:01Z

Looks like there's a panic on ppc64le;


[2020-06-04T09:04:38.364Z] === FAIL: ppc64le.integration.build TestBuildMultiStageOnBuild (22.32s)
[2020-06-04T09:04:38.364Z]     build_test.go:377: assertion failed: string "{\"stream\":\"Step 1/7 : FROM busybox AS stage1\"}\r\n{\"stream\":\"\\n\"}\r\n{\"stream\":\" ---\\u003e f4279da41337\\n\"}\r\n{\"stream\":\"Step 2/7 : ONBUILD RUN echo 'foo' \\u003esomefile\"}\r\n{\"stream\":\"\\n\"}\r\n{\"stream\":\" ---\\u003e Running in 2dd830333d56\\n\"}\r\n{\"stream\":\"Removing intermediate container 2dd830333d56\\n\"}\r\n{\"stream\":\" ---\\u003e ed9596ea0ee0\\n\"}\r\n{\"stream\":\"Step 3/7 : ONBUILD ENV bar=baz\"}\r\n{\"stream\":\"\\n\"}\r\n{\"stream\":\" ---\\u003e Running in a65eb56bb730\\n\"}\r\n{\"stream\":\"Removing intermediate container a65eb56bb730\\n\"}\r\n{\"stream\":\" ---\\u003e 72c5aefcdca5\\n\"}\r\n{\"aux\":{\"ID\":\"sha256:72c5aefcdca5c616afb139e3bd6a90d5e68c22124645b9ea51fe358fe5038154\"}}\r\n{\"stream\":\"Step 4/7 : FROM stage1\"}\r\n{\"stream\":\"\\n\"}\r\n{\"stream\":\"# Executing 2 build trigger\"}\r\n{\"stream\":\"s\"}\r\n{\"stream\":\"\\n\"}\r\n{\"stream\":\" ---\\u003e Running in 670fc6466a09\\n\"}\r\n{\"stream\":\"Removing intermediate container 670fc6466a09\\n\"}\r\n{\"errorDetail\":{\"message\":\"OCI runtime create failed: container_linux.go:349: starting container process caused \\\"process_linux.go:449: container init caused \\\\\\\"write sysctl key net.ipv4.ip_unprivileged_port_start: open /proc/sys/net/ipv4/ip_unprivileged_port_start: no such file or directory\\\\\\\"\\\": unknown\"},\"error\":\"OCI runtime create failed: container_linux.go:349: starting container process caused \\\"process_linux.go:449: container init caused \\\\\\\"write sysctl key net.ipv4.ip_unprivileged_port_start: open /proc/sys/net/ipv4/ip_unprivileged_port_start: no such file or directory\\\\\\\"\\\": unknown\"}\r\n" does not contain "Successfully built"
[2020-06-04T09:04:38.364Z]     build_test.go:381: assertion failed: 3 (int) != 1 (int)
[2020-06-04T09:04:38.364Z] panic: runtime error: index out of range [2] with length 1 [recovered]
[2020-06-04T09:04:38.364Z] 	panic: runtime error: index out of range [2] with length 1

write sysctl key net.ipv4.ip_unprivileged_port_start: open /proc/sys/net/ipv4/ip_unprivileged_port_start: no such file or directory

thaJeztah · 2020-06-04T11:54:48Z

That error seems to come from runc; @kolyshkin @AkihiroSuda

justincormack · 2020-06-04T13:38:50Z

I am just adding checks for whether the sysctl node exists, which should work around this, although it was mainly designed around old kernels. However it shouldn't panic.

…o capabilities Currently default capability CAP_NET_RAW allows users to open ICMP echo sockets, and CAP_NET_BIND_SERVICE allows binding to ports under 1024. Both of these are safe operations, and Linux now provides ways that these can be set, per container, to be allowed without any capabilties for non root users. Enable these by default. Users can revert to the previous behaviour by overriding the sysctl values explicitly. Signed-off-by: Justin Cormack <justin.cormack@docker.com>

kolyshkin · 2020-06-04T18:17:48Z

The panic is not from runc, but from the test case itself, here:

moby/integration/build/build_test.go

Lines 379 to 383 in 0a82354

    
           imageIDs, err := getImageIDsFromBuild(out.Bytes()) 
        
           assert.NilError(t, err) 
        
           assert.Check(t, is.Equal(3, len(imageIDs))) 
        
           image, _, err := apiclient.ImageInspectWithRaw(context.Background(), imageIDs[2])

It panics right there, line 383. The panic fix is

diff --git a/integration/build/build_test.go b/integration/build/build_test.go
index edbfb37935..cd682a5e58 100644
--- a/integration/build/build_test.go
+++ b/integration/build/build_test.go
@@ -378,7 +378,7 @@ RUN cat somefile`
 
        imageIDs, err := getImageIDsFromBuild(out.Bytes())
        assert.NilError(t, err)
-       assert.Check(t, is.Equal(3, len(imageIDs)))
+       assert.Assert(t, is.Equal(3, len(imageIDs)))
 
        image, _, err := apiclient.ImageInspectWithRaw(context.Background(), imageIDs[2])
        assert.NilError(t, err)

thaJeztah · 2020-06-04T20:19:42Z

oh, thanks! I must've read the output wrong 😅

kolyshkin

LGTM

thaJeztah

LGTM, thanks!

(asked @kolyshkin to double-check the approach for sysctlExists() 🤗)

@dfangl

As @dfangl pointed out: Binding port 53 might require root permissions for binding port < 1024 depending on the Docker version moby/moby#41030

@dfangl

As @dfangl pointed out: Binding port 53 might require root permissions for binding port < 1024 depending on the Docker version moby/moby#41030

In this PR grafana/agent#6817 the setcap was added for the alloy/agent binary for some reason unknown to me. Since the docker engine allows binding to unpriviliged ports already for a long time via moby/moby#41030, which was implemented in the Docker engine v20.10.x and also other container runtimes. This should solve the following issues. Fixes grafana#117 Fixes grafana#303

In this PR grafana/agent#6817 the setcap was added for the alloy/agent binary for some reason unknown to me. Since the docker engine allows binding to unpriviliged ports already for a long time via moby/moby#41030, which was implemented in the Docker engine v20.10.x and also other container runtimes. This should solve the following issues. Fixes #177 Fixes #303

In this PR grafana/agent#6817 the setcap was added for the alloy/agent binary for some reason unknown to me. Since the docker engine allows binding to unpriviliged ports already for a long time via moby/moby#41030, which was implemented in the Docker engine v20.10.x and also other container runtimes. This should solve the following issues. Fixes grafana#177 Fixes grafana#303

lukanoluwatobi · 2024-06-27T08:23:11Z

For docker compose file use this instead

...
sysctls:
      net.ipv4.ip_unprivileged_port_start: 0
    user: "${UID}:${GID}"
...

thaJeztah · 2024-06-27T10:44:22Z

@lukanoluwatobi this option is set by default on current versions of the engine (20.10 and up), so I don't think there should be a need to manually pass that option. Was there a reason you had to set it manually?

thaJeztah added impact/changelog status/2-code-review labels May 26, 2020

thaJeztah reviewed May 26, 2020

View reviewed changes

justincormack force-pushed the default-sysctls branch from d733554 to 241fbd8 Compare May 27, 2020 18:14

justincormack force-pushed the default-sysctls branch from 241fbd8 to 2e2d3e8 Compare May 28, 2020 09:47

justincormack force-pushed the default-sysctls branch from 2e2d3e8 to fc7b8d7 Compare May 28, 2020 13:21

justincormack force-pushed the default-sysctls branch 5 times, most recently from d3e1917 to bd4bd48 Compare June 2, 2020 11:41

justincormack force-pushed the default-sysctls branch 2 times, most recently from 65bcfa9 to ed291d9 Compare June 4, 2020 13:41

justincormack force-pushed the default-sysctls branch from ed291d9 to dae652e Compare June 4, 2020 17:11

kolyshkin mentioned this pull request Jun 4, 2020

int/build/TestBuildWithEmptyLayers: prevent panic #41070

Merged

kolyshkin approved these changes Jun 4, 2020

View reviewed changes

thaJeztah approved these changes Jun 4, 2020

View reviewed changes

This was referenced Feb 22, 2023

Correct the condition based on the previous commit as the current one allows any network on cloud to ping without adhering to any security protocols #45053

Open

AppArmor policy to deny network is not working #44984

Open

dfangl mentioned this pull request Mar 8, 2023

Switch user and drop privileges localstack/lambda-runtime-init#13

Merged

udf2457 mentioned this pull request Mar 25, 2023

[bitnami/mysql] Needs to expose podSecurityContext/containerSecurityContext for metrics bitnami/charts#15552

Closed

thw0rted mentioned this pull request Apr 6, 2023

Official container doesn't run in non-Docker runtime on default port pgadmin-org/pgadmin4#6101

Closed

thaJeztah mentioned this pull request Apr 11, 2023

Container does not have the cap_net_raw+p capability or setuid? capabilities. nicolaka/netshoot#137

Closed

dfangl mentioned this pull request Aug 24, 2023

Bind hypercorn to privileged ports if root localstack/localstack#8948

Merged

mhofstetter mentioned this pull request Mar 13, 2024

ingress/gateway-api: expose listeners on host network cilium/cilium#30840

Merged

prezha mentioned this pull request Mar 17, 2024

add default sysctls to allow privileged ports with no capabilities kubernetes/minikube#18421

Merged

tianon mentioned this pull request Mar 18, 2024

HAproxy 2.4 cannot bind socket (Permission denied) [0.0.0.0:80] docker-library/haproxy#170

Closed

thielj mentioned this pull request Apr 17, 2024

Rootless postfix? bokysan/docker-postfix#195

Closed

BlackDex mentioned this pull request May 1, 2024

Remove set bind permissions grafana/alloy#726

Merged

4 tasks

tianon mentioned this pull request May 21, 2024

[nats] Release v2.10.16 docker-library/official-images#16797

Merged

thaJeztah added the area/networking Networking label Jun 27, 2024

lixmal mentioned this pull request Aug 27, 2024

[relay] Feature/relay integration docker netbirdio/netbird#2484

Merged

6 tasks

ferferga mentioned this pull request Sep 27, 2024

Docker bind() to 0.0.0.0:80 failed (13: Permission denied) jellyfin/jellyfin-vue#2236

Closed

This comment was marked as spam.

Sign in to view

t3chguy mentioned this pull request Jan 20, 2025

Docker container crash after update 1.11.90 element-hq/element-web#29030

Closed

gwenya mentioned this pull request Feb 28, 2025

Allow non-root users in OCI to listen to low port numbers lxc/incus#1704

Closed

pas256 mentioned this pull request Mar 20, 2025

Guidance on running on AWS ECS basecamp/thruster#53

Open

tianon mentioned this pull request Jul 28, 2025

Update YOURLS docker-library/official-images#19553

Merged

tianon mentioned this pull request Sep 2, 2025

Caddy: Added NET_BIND_SERVICE note docker-library/docs#2620

Open

Add default sysctls to allow ping sockets and privileged ports with no capabilities #41030

Add default sysctls to allow ping sockets and privileged ports with no capabilities #41030

Uh oh!

Conversation

justincormack commented May 26, 2020 • edited by thaJeztah Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

justincormack commented May 26, 2020

Uh oh!

thaJeztah commented May 26, 2020

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tianon Jun 4, 2021 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented May 28, 2020

Uh oh!

thaJeztah commented May 28, 2020

Uh oh!

justincormack commented May 28, 2020

Uh oh!

thaJeztah commented May 28, 2020

Uh oh!

justincormack commented May 28, 2020

Uh oh!

thaJeztah commented Jun 4, 2020 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

thaJeztah commented Jun 4, 2020

Uh oh!

justincormack commented Jun 4, 2020

Uh oh!

kolyshkin commented Jun 4, 2020

Uh oh!

thaJeztah commented Jun 4, 2020

Uh oh!

kolyshkin left a comment

Choose a reason for hiding this comment

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

lukanoluwatobi commented Jun 27, 2024

Uh oh!

thaJeztah commented Jun 27, 2024

Uh oh!

This comment was marked as spam.

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

justincormack commented May 26, 2020 •

edited by thaJeztah

Loading

tianon Jun 4, 2021 •

edited

Loading

thaJeztah commented Jun 4, 2020 •

edited

Loading