[BUG] Include.env_file content can't be used for secrets from environment variable #12632
Description
Description
Setup: A parent compose project includes a child compose compose project which has secrets from environment defined.
When including an env file in the include directive, the content is not considered for the secret.
The error is then environment variable "MY_SECRET" required by secret "parent_my-secret" is not set
It however works if the env file is included directly via the --env-file flag.
I wonder if this is intentional behavior, at least I was surprised by this.
And there is no restriction regarding secrets mentioned in the include directive docs: https://docs.docker.com/reference/compose-file/include/#env_file
Steps To Reproduce
Create a file child/compose.yaml with content (service prints env var and secret, which will be both defined in env-vars)
secrets:
my-secret:
environment: 'MY_SECRET'
services:
my-service:
image: alpine
secrets:
- my-secret
environment:
- MY_ENV
command: "/bin/sh -c 'echo env-var: $MY_ENV && cat /run/secrets/my-secret'"Then create the parent compose file at compose.yaml with content
name: parent
include:
- path: child/compose.yaml
env_file:
- secret.envAnd the env file secret.env with content
MY_SECRET='super-secret'
MY_ENV='foo'So the resulting file tree is
├── child
│ └── compose.yaml
├── compose.yaml
└── secret.env
Then execute in the root folder of this tree docker compose up, it will output
[+] Running 1/2
✔ Network parent_default Created
⠋ Container parent-my-service-1 Creating
environment variable "MY_SECRET" required by secret "parent_my-secret" is not setRepeating the command docker compose up will start successfully (seems like the secret env var is only checked on container creation, which is a bit confusing):
[+] Running 1/1
✔ Container d9402f3d9545_parent-my-service-1 Created 0.0s
Attaching to d9402f3d9545_parent-my-service-1
d9402f3d9545_parent-my-service-1 | env-var: foo
d9402f3d9545_parent-my-service-1 | cat: can't open '/run/secrets/my-secret': No such file or directory
d9402f3d9545_parent-my-service-1 exited with code 1Clean up with docker compose down -v and execute docker compose --env-file secret.env up will give the desired result:
[+] Running 2/2
✔ Network parent_default Created
✔ Container parent-my-service-1 Created
Attaching to my-service-1
my-service-1 | env-var: foo
my-service-1 | super-secret
my-service-1 exited with code 0Compose Version
Docker Compose version v2.33.1
Docker Environment
Client: Docker Engine - Community
Version: 28.0.1
Context: default
Debug Mode: false
Plugins:
buildx: Docker Buildx (Docker Inc.)
Version: v0.21.1
Path: /usr/libexec/docker/cli-plugins/docker-buildx
compose: Docker Compose (Docker Inc.)
Version: v2.33.1
Path: /usr/libexec/docker/cli-plugins/docker-compose
Server:
Containers: 23
Running: 11
Paused: 0
Stopped: 12
Images: 41
Server Version: 28.0.1
Storage Driver: overlay2
Backing Filesystem: extfs
Supports d_type: true
Using metacopy: false
Native Overlay Diff: true
userxattr: false
Logging Driver: json-file
Cgroup Driver: systemd
Cgroup Version: 2
Plugins:
Volume: local
Network: bridge host ipvlan macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
Swarm: inactive
Runtimes: io.containerd.runc.v2 runc
Default Runtime: runc
Init Binary: docker-init
containerd version: bcc810d6b9066471b0b6fa75f557a15a1cbf31bb
runc version: v1.2.4-0-g6c52b3f
init version: de40ad0
Security Options:
apparmor
seccomp
Profile: builtin
cgroupns
Kernel Version: 6.8.0-52-generic
Operating System: Ubuntu 22.04.5 LTS
OSType: linux
Architecture: x86_64
CPUs: 12
Total Memory: 30.15GiB
Name: GB-HO-50
ID: b7121279-c046-406f-85a0-dade9f473530
Docker Root Dir: /var/lib/docker
Debug Mode: false
Experimental: false
Insecure Registries:
::1/128
127.0.0.0/8
Live Restore Enabled: false
Anything else?
No response