-
Notifications
You must be signed in to change notification settings - Fork 5.7k
Description
Description
Description
This is a critical bug for Linux containers that use multiple ethernet ports. The issue is each time Docker Compose is invoked, it assigns the IP addresses to the lexically ordered interface names in the associated container in a haphazard order and not in accordance with the YAML file. For example, it can assign eth1's IP to eth2, eth2's IP to eth4 etc. This also means directly connected containers in the YAML file can not ping between interfaces that are supposed to be connected correctly per the YAML file.
Contacted Docker Support, they confirmed the issue and asked me to open a case on Git Hub, so here is the info. Including a simple Docker Compose YAML file that shows the issue with a single generic alpine container. In both cases (working and not), using the following docker compose version so it is not the docker compose that is at fault here:
Docker Compose version v2.35.1
Steps to Reproduce
1. Environment
Any 28.x Docker version I have tried has this issue. You can go to the latest 28.x Docker Engine as well and you will see the issue.
docker version
Client: Docker Engine - Community
Version: 28.0.4
API version: 1.48
Go version: go1.23.7
Git commit: b8034c0
Built: Tue Mar 25 15:07:16 2025
OS/Arch: linux/amd64
Context: default
Server: Docker Engine - Community
Engine:
Version: 28.0.4
API version: 1.48 (minimum version 1.24)
Go version: go1.23.7
Git commit: 6430e49
Built: Tue Mar 25 15:07:16 2025
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.7.27
GitCommit: 05044ec0a9a75232cad458027ca83437aae3f4da
runc:
Version: 1.2.5
GitCommit: v1.2.5-0-g59923ef
docker-init:
Version: 0.19.0
GitCommit: de40ad0
The issue happens on both Ubuntu 24.04 and 22.04 LTS versions
2. Configuration
This simplified Docker Compose YAML can be used to see the issue, showing a single container here for simplicity:
services:
cAlpine1:
image: 'alpine:latest'
container_name: alpine1
hostname: alpine1
command: ["/bin/sh", "-c", "while true; do echo 'Infinite loop!'; sleep 5; done"]
privileged: true
networks:
eth0:
ipv4_address: 10.0.1.2
eth1:
ipv4_address: 10.1.1.2
eth2:
ipv4_address: 10.2.1.2
eth3:
ipv4_address: 10.3.1.2
networks:
eth0:
name: eth0
driver: bridge
ipam:
driver: default
config:
- subnet: 10.0.1.0/24
eth1:
name: eth1
driver: bridge
ipam:
driver: default
config:
- subnet: 10.1.1.0/24
eth2:
name: eth2
driver: bridge
ipam:
driver: default
config:
- subnet: 10.2.1.0/24
eth3:
name: eth3
driver: bridge
ipam:
driver: default
config:
- subnet: 10.3.1.0/24
3. Command
docker compose -f single-alpine-compose.yaml up -d
docker container ls
docker network ls
docker exec -ti alpine1 ash
ip address
4. Result/Error
You may have to try this a couple of times, but the problem is easily reproducable. Have not seen it take more than 3 times.
Below shows that eth0 got the IP address for eth1, eth1 got the IP address for eth2, eth2 got the IP address for eth3, eth3 got the IP address for eth0. There is no predictability here, on different runs, different out of order assignments are done. The docker network ls always shows the networks in correct lexical order (see later below)
docker exec -ti alpine1 ash
/ # ip address
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
valid_lft forever preferred_lft forever
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0@if23: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
link/ether ca:00:9f:52:47:a3 brd ff:ff:ff:ff:ff:ff
inet 10.1.1.2/24 brd 10.1.1.255 scope global eth0
valid_lft forever preferred_lft forever
3: eth1@if24: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
link/ether 3a:ad:30:86:56:84 brd ff:ff:ff:ff:ff:ff
inet 10.2.1.2/24 brd 10.2.1.255 scope global eth1
valid_lft forever preferred_lft forever
4: eth2@if25: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
link/ether 06:15:e4:ed:b0:b8 brd ff:ff:ff:ff:ff:ff
inet 10.3.1.2/24 brd 10.3.1.255 scope global eth2
valid_lft forever preferred_lft forever
5: eth3@if26: <BROADCAST,MULTICAST,UP,LOWER_UP,M-DOWN> mtu 1500 qdisc noqueue state UP
link/ether 92:09:2a:a8:ea:8e brd ff:ff:ff:ff:ff:ff
inet 10.0.1.2/24 brd 10.0.1.255 scope global eth3
valid_lft forever preferred_lft forever
/ #
# docker network ls
NETWORK ID NAME DRIVER SCOPE
fab4d00f70ab bridge bridge local
c8685fe7287c eth0 bridge local
d2dcffbce587 eth1 bridge local
b7bff14d2235 eth2 bridge local
156428999cc7 eth3 bridge local
5ad3a22e4929 host host local
295b8976dbd1 none null local
Steps To Reproduce
1- Use the docker compose up command shown to create the alpine1 container:
docker compose -f single-alpine-compose.yaml up -d
2- Enter the alpine container:
docker exec -ti alpine1 ash
3- Run the following command inside alpine1 to see the IP address assigned to each of eth0 - eth3 and see that they do not match the YAML file.
/ # ip address
Compose Version
# docker compose version
Docker Compose version v2.34.0
Docker Environment
**This issue also happened on 28.1.1-1, 28.1.0**
# docker info
Client: Docker Engine - Community
Version: 28.0.4
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.22.0
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.34.0
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 2
Running: 1
Paused: 0
Stopped: 1
Images: 5
Server Version: 28.0.4
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 05044ec0a9a75232cad458027ca83437aae3f4da
runc version: v1.2.5-0-g59923ef
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.8.0-57-generic
Operating System: Ubuntu 24.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 56
Total Memory: 251.8GiB
Name: q-jlab-05a
ID: c67cc920-990c-47dd-8ada-8f5b259a2259
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
::1/128
127.0.0.0/8
Live Restore Enabled: false
Anything else?
No response