Link containers with --net=host #5835
Description
I was wondering why linking a container using the bridge networking to a container using the host networking doesn't work.
The use case is simple: I have a first container running a service on a port and a second container accessing this service and internet. The work done by the second container is limited by the speed of the network, so I would like to switch to host networking to see if it makes a difference.
I reproduced my tests using phusion/baseimage. In these examples, the service provided by the first container will be ssh.
I would like to know if you had similar issues and how you approached the problem.
Initial configuration
This is the configuration with the bridge networking. As expected, the second container receive the right environment variables.
First container
#!/bin/bash -ex
sudo docker stop test_a || true
sudo docker rm test_a || true
sudo docker run -t --name test_a --publish=22 phusion/baseimage:0.9.10Second container
#!/bin/bash -ex
sudo docker stop test_b || true
sudo docker rm test_b || true
sudo docker run -t --link=test_a:test --name=test_b phusion/baseimage:0.9.10 envOutput
HOME=/root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=4762d6f3de3f
TERM=xterm
TEST_PORT=tcp://172.17.0.4:22
TEST_PORT_22_TCP=tcp://172.17.0.4:22
TEST_PORT_22_TCP_ADDR=172.17.0.4
TEST_PORT_22_TCP_PORT=22
TEST_PORT_22_TCP_PROTO=tcp
TEST_NAME=/test_b/test
Host networking
If I switch the second container to host networking, it doesn't start.
First container
#!/bin/bash -ex
sudo docker stop test_c || true
sudo docker rm test_c || true
sudo docker run -t --name test_c --publish=22 phusion/baseimage:0.9.10Second container
#!/bin/bash -ex
sudo docker stop test_d || true
sudo docker rm test_d || true
sudo docker run -t --net=host --link=test_c:test --name=test_d phusion/baseimage:0.9.10 envOutput
2014/05/16 09:15:57 Error: Cannot start container 52febf2b6bc944fd13a8e4ef2a13e3eaf2e365b8b908b2b382ac69ce0bafe8ae: (exit status 2)
Host networking without publish
If I don't publish the port on the first container, the link works (the second container receives the environment variable TEST_NAME=/test_f/test) but as expected the second container doesn't receive the ip of the first one.
First container
#!/bin/bash -ex
sudo docker stop test_e || true
sudo docker rm test_e || true
sudo docker run -t --name test_e phusion/baseimage:0.9.10Second container
#!/bin/bash -ex
sudo docker stop test_f || true
sudo docker rm test_f || true
sudo docker run -t --net=host --link=test_e:test --name=test_f phusion/baseimage:0.9.10 envOutput
HOME=/root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=kazoup-nicolas
TERM=xterm
TEST_NAME=/test_f/test
Workaround 1 : no link + inspect
The documentation says that "Publishing ports and linking to other containers will not work when sharing the host's network stack." but also that "With the networking mode set to host a container will share the host's network stack and all interfaces from the host will be available to the container."
If all the interfaces are available, it also means that the second container will have access to the docker0 interface and have access to the first container using the bridge networking. This can be verified with the following scripts. I published the port on the first container and removed the link.
First container
#!/bin/bash -ex
sudo docker stop test_g || true
sudo docker rm test_g || true
sudo docker run -t --name test_g --publish=22 phusion/baseimage:0.9.10 /sbin/my_init --enable-insecure-keySecond container
#!/bin/bash -ex
sudo docker stop test_h || true
sudo docker rm test_h || true
IP=$(sudo docker inspect --format '{{ .NetworkSettings.IPAddress }}' test_g)
sudo docker run -i -t --net=host --name=test_h phusion/baseimage:0.9.10 \
bash -c "ping ${IP} -c 5 \
&& curl --insecure -o insecure_key -fSL https://github.com/phusion/baseimage-docker/raw/master/image/insecure_key \
&& chmod 600 insecure_key \
&& ssh -i insecure_key -o StrictHostKeyChecking=no ${IP} -t ifconfig" Output
PING 172.17.0.7 (172.17.0.7) 56(84) bytes of data.
64 bytes from 172.17.0.7: icmp_seq=1 ttl=64 time=0.035 ms
64 bytes from 172.17.0.7: icmp_seq=2 ttl=64 time=0.078 ms
64 bytes from 172.17.0.7: icmp_seq=3 ttl=64 time=0.074 ms
64 bytes from 172.17.0.7: icmp_seq=4 ttl=64 time=0.073 ms
64 bytes from 172.17.0.7: icmp_seq=5 ttl=64 time=0.038 ms
--- 172.17.0.7 ping statistics ---
5 packets transmitted, 5 received, 0% packet loss, time 3997ms
rtt min/avg/max/mdev = 0.035/0.059/0.078/0.020 ms
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 150 100 150 0 0 25 0 0:00:06 0:00:05 0:00:01 34
100 1679 100 1679 0 0 145 0 0:00:11 0:00:11 --:--:-- 405
Warning: Permanently added '172.17.0.7' (ECDSA) to the list of known hosts.
eth0 Link encap:Ethernet HWaddr fe:67:37:c8:03:bb
inet addr:172.17.0.7 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::fc67:37ff:fec8:3bb/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:12232 errors:0 dropped:70906 overruns:0 frame:0
TX packets:12238 errors:0 dropped:2 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1203310 (1.2 MB) TX bytes:1202990 (1.2 MB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Connection to 172.17.0.7 closed.
Workaround 2 : no link + publish
It also works when the port of the first container is published on the host machine. The second container can then access it using localhost
#!/bin/bash -ex
sudo docker stop test_i || true
sudo docker rm test_i || true
sudo docker run -t --name test_i --publish=2299:22 phusion/baseimage:0.9.10 /sbin/my_init --enable-insecure-key#!/bin/bash -ex
sudo docker stop test_j || true
sudo docker rm test_j || true
sudo docker run -i -t --net=host --name=test_j phusion/baseimage:0.9.10 \
bash -c "curl --insecure -o insecure_key -fSL https://github.com/phusion/baseimage-docker/raw/master/image/insecure_key \
&& chmod 600 insecure_key \
&& ssh -i insecure_key -o StrictHostKeyChecking=no localhost -p 2299 -t ifconfig" % Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 150 100 150 0 0 25 0 0:00:06 0:00:05 0:00:01 44
100 1679 100 1679 0 0 146 0 0:00:11 0:00:11 --:--:-- 411
Warning: Permanently added '[localhost]:2299' (ECDSA) to the list of known hosts.
eth0 Link encap:Ethernet HWaddr de:fa:cc:7c:8a:82
inet addr:172.17.0.8 Bcast:0.0.0.0 Mask:255.255.0.0
inet6 addr: fe80::dcfa:ccff:fe7c:8a82/64 Scope:Link
UP BROADCAST RUNNING MTU:1500 Metric:1
RX packets:22 errors:0 dropped:2 overruns:0 frame:0
TX packets:23 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:4879 (4.8 KB) TX bytes:3889 (3.8 KB)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Connection to localhost closed.
Environment
uname -a
Linux kazoup-nicolas 3.11-2-amd64 #1 SMP Debian 3.11.8-1 (2013-11-13) x86_64 GNU/Linux
docker version
Client version: 0.11.1
Client API version: 1.11
Go version (client): go1.2.1
Git commit (client): fb99f99
Server version: 0.11.1
Server API version: 1.11
Git commit (server): fb99f99
Go version (server): go1.2.1
Last stable version: 0.11.1
docker info
Containers: 18
Images: 60
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Dirs: 96
Execution Driver: native-0.2
Kernel Version: 3.11.0-2-amd64
WARNING: No memory limit support
WARNING: No swap limit support