Skip to content

[BUG] commit 163cdfd31 breaks interpolation in extended files #11994

@jolmg

Description

@jolmg

Description

Current behavior is the following error, with a stacktrace (included in reproduction steps):

panic: interface conversion: interface {} is string, not map[string]interface {}

I also get a different error if I remove a mount option next to the interpolation. Both configs and errors included in reproduction steps.

Expected containers to come up when I did docker compose up. Error(s) also happens with docker compose config, and it's got something to do with combining interpolation and extended files.

It seems the bug was introduced in 163cdfd, which adds a test and updates the compose-go dependency.

Checking out compose-go, with compose set to 163cdfd31~, it seems the bug was introduced there in compose-spec/compose-go@65600ce.

Adding report here rather than there because I'm observing and replicating the error from compose.

Steps To Reproduce

Script to reproduce:

d=/tmp/docker-compose-interpolation-test-extended-with-ro

set -e
mkdir "$d"
cd "$d"

cat > docker-compose.yml <<EOF
services:
  foo:
    extends: { file: foo.yml, service: foo }
EOF

cat > foo.yml <<EOF
services:
  foo:
    image: bash
    volumes:
      - \${FOO:-/dev/null}:/tmp/foo:ro
EOF

docker compose config

Result:

panic: interface conversion: interface {} is string, not map[string]interface {}

goroutine 1 [running]:
github.com/compose-spec/compose-go/v2/paths.(*relativePathsResolver).absVolumeMount(0x2?, {0x5ea29c98d9c0?, 0xc0007137a0?})
    github.com/compose-spec/compose-go/v2@v2.1.4/paths/resolve.go:123 +0x1be
github.com/compose-spec/compose-go/v2/paths.(*relativePathsResolver).resolveRelativePaths(0xc00047e6c0, {0x5ea29c98d9c0, 0xc0007137a0}, {0xc00041f770, 0x17})
    github.com/compose-spec/compose-go/v2@v2.1.4/paths/resolve.go:74 +0xf0
github.com/compose-spec/compose-go/v2/paths.(*relativePathsResolver).resolveRelativePaths(0xc00047e6c0, {0x5ea29c959c80, 0xc00073aa98}, {0xc00041f758, 0x14})
    github.com/compose-spec/compose-go/v2@v2.1.4/paths/resolve.go:88 +0x339
github.com/compose-spec/compose-go/v2/paths.(*relativePathsResolver).resolveRelativePaths(0xc00047e6c0, {0x5ea29caa51c0, 0xc00035d590}, {0xc000723da0, 0xc})
    github.com/compose-spec/compose-go/v2@v2.1.4/paths/resolve.go:80 +0x22e
github.com/compose-spec/compose-go/v2/paths.(*relativePathsResolver).resolveRelativePaths(0xc00047e6c0, {0x5ea29caa51c0, 0xc00035d560}, {0xc000723c98, 0x8})
    github.com/compose-spec/compose-go/v2@v2.1.4/paths/resolve.go:80 +0x22e
github.com/compose-spec/compose-go/v2/paths.(*relativePathsResolver).resolveRelativePaths(0xc00047e6c0, {0x5ea29caa51c0, 0xc00035d380}, {0x0, 0x0})
    github.com/compose-spec/compose-go/v2@v2.1.4/paths/resolve.go:80 +0x22e
github.com/compose-spec/compose-go/v2/paths.ResolveRelativePaths(0xc00035d380, {0x5ea29c81ad28, 0x1}, {0xc0007137c0, 0x2, 0x2})
    github.com/compose-spec/compose-go/v2@v2.1.4/paths/resolve.go:50 +0x827
github.com/compose-spec/compose-go/v2/loader.getExtendsBaseFromFile({0x5ea29ce88128, 0xc00035c810}, {0xc000723bd8, 0x3}, {0xc000723c78, 0x3}, {0xc000781c80, 0x3e}, {0xc000723c50, 0x9}, ...)
    github.com/compose-spec/compose-go/v2@v2.1.4/loader/extends.go:203 +0x985
github.com/compose-spec/compose-go/v2/loader.applyServiceExtends({0x5ea29ce88128, 0xc00035c810}, {0xc000723bd8, 0x3}, 0xc00035c9c0, 0xc00054fcb0, 0xc0005a0c20, {0xc0005a0900, 0x1, 0x1})
    github.com/compose-spec/compose-go/v2@v2.1.4/loader/extends.go:101 +0x4da
github.com/compose-spec/compose-go/v2/loader.ApplyExtends({0x5ea29ce88128, 0xc00035c810}, 0xc00035c990, 0xc00054fcb0, 0xc0005a0c20, {0xc0005a0900, 0x1, 0x1})
    github.com/compose-spec/compose-go/v2@v2.1.4/loader/extends.go:43 +0x174
github.com/compose-spec/compose-go/v2/loader.loadYamlFile.func1({0x5ea29caa51c0?, 0xc00035c990?}, {0xc0005a0900, 0x1, 0x1})
    github.com/compose-spec/compose-go/v2@v2.1.4/loader/loader.go:438 +0x11d
github.com/compose-spec/compose-go/v2/loader.loadYamlFile({0x5ea29ce88160, 0xc0003eec30}, {{0xc000781c80, 0x3e}, {0xc000536200, 0x40, 0x200}, 0x0}, 0xc00054fcb0, {0xc000781c80, ...}, ...)
    github.com/compose-spec/compose-go/v2@v2.1.4/loader/loader.go:490 +0x564
github.com/compose-spec/compose-go/v2/loader.loadYamlModel({0x5ea29ce88160, 0xc0003eec30}, {{0x0, 0x0}, {0xc000781c80, 0x2b}, {0xc00035c0f0, 0x1, 0x1}, 0xc00057dd70}, ...)
    github.com/compose-spec/compose-go/v2@v2.1.4/loader/loader.go:364 +0x1a5
github.com/compose-spec/compose-go/v2/loader.load({0x5ea29ce88160, 0xc0003eec30}, {{0x0, 0x0}, {0xc000781c80, 0x2b}, {0xc00035c0f0, 0x1, 0x1}, 0xc00057dd70}, ...)
    github.com/compose-spec/compose-go/v2@v2.1.4/loader/loader.go:512 +0x38b
github.com/compose-spec/compose-go/v2/loader.loadModelWithContext({0x5ea29ce88160, 0xc0003eec30}, 0xc00069eb40, 0xc00054fcb0)
    github.com/compose-spec/compose-go/v2@v2.1.4/loader/loader.go:336 +0x105
github.com/compose-spec/compose-go/v2/loader.LoadWithContext({0x5ea29ce88160, 0xc0003eec30}, {{0x0, 0x0}, {0xc000781c80, 0x2b}, {0xc00035c0f0, 0x1, 0x1}, 0xc00057dd70}, ...)
    github.com/compose-spec/compose-go/v2@v2.1.4/loader/loader.go:312 +0xd8
github.com/compose-spec/compose-go/v2/cli.(*ProjectOptions).LoadProject(0xc00054fc20, {0x5ea29ce88160, 0xc0003eec30})
    github.com/compose-spec/compose-go/v2@v2.1.4/cli/options.go:445 +0x128
github.com/docker/compose/v2/cmd/compose.(*ProjectOptions).ToProject(0xc000315720, {0x5ea29ce88160, 0xc0003eec30}, {0x5ea29ce9f160, 0xc000420c80}, {0x5ea29e0fd3a0, 0x0, 0x0}, {0xc00057dce0, 0x6, ...})
    github.com/docker/compose/v2/cmd/compose/compose.go:318 +0x539
github.com/docker/compose/v2/cmd/compose.(*configOptions).ToProject(0xc0005a16d0, {0x5ea29ce88160, 0xc0003eec30}, {0x5ea29ce9f160, 0xc000420c80}, {0x5ea29e0fd3a0, 0x0, 0x0}, {0x0, 0x0, ...})
    github.com/docker/compose/v2/cmd/compose/config.go:62 +0x16a
github.com/docker/compose/v2/cmd/compose.runConfigInterpolate({0x5ea29ce88160, 0xc0003eec30}, {0x5ea29ce9f160, 0xc000420c80}, {0xc000315720, {0x5ea29c4a6c07, 0x4}, {0x0, 0x0}, 0x0, ...}, ...)
    github.com/docker/compose/v2/cmd/compose/config.go:181 +0x72
github.com/docker/compose/v2/cmd/compose.runConfig({0x5ea29ce88160?, 0xc0003eec30?}, {0x5ea29ce9f160, 0xc000420c80}, {0xc000315720, {0x5ea29c4a6c07, 0x4}, {0x0, 0x0}, 0x0, ...}, ...)
    github.com/docker/compose/v2/cmd/compose/config.go:159 +0xc5
github.com/docker/compose/v2/cmd/compose.configCommand.func2({0x5ea29ce88160?, 0xc0003eec30?}, {0x5ea29e0fd3a0?, 0xc0000061c0?, 0x5ea29c47933c?})
    github.com/docker/compose/v2/cmd/compose/config.go:126 +0x105
github.com/docker/compose/v2/cmd/compose.configCommand.Adapt.func4({0x5ea29ce88160?, 0xc0003eec30?}, 0x2?, {0x5ea29e0fd3a0?, 0x5ea29ce6d260?, 0x78b32a?})
    github.com/docker/compose/v2/cmd/compose/compose.go:125 +0x30
github.com/docker/compose/v2/cmd/compose.configCommand.Adapt.AdaptCmd.func7(0xc0001f6f08, {0x5ea29e0fd3a0, 0x0, 0x0})
    github.com/docker/compose/v2/cmd/compose/compose.go:101 +0x154
github.com/docker/cli/cli-plugins/plugin.RunPlugin.func1.1.2(0xc0001f6f08, {0x5ea29e0fd3a0, 0x0, 0x0})
    github.com/docker/cli@v27.0.3+incompatible/cli-plugins/plugin/plugin.go:64 +0x6c
github.com/docker/compose/v2/cmd/cmdtrace.Setup.wrapRunE.func2(0xc0001f6f08?, {0x5ea29e0fd3a0?, 0x0?, 0x0?})
    github.com/docker/compose/v2/cmd/cmdtrace/cmd_span.go:85 +0x63
github.com/spf13/cobra.(*Command).execute(0xc0001f6f08, {0xc00014baf0, 0x0, 0x0})
    github.com/spf13/cobra@v1.8.1/command.go:985 +0xaca
github.com/spf13/cobra.(*Command).ExecuteC(0xc000020308)
    github.com/spf13/cobra@v1.8.1/command.go:1117 +0x3ff
github.com/spf13/cobra.(*Command).Execute(...)
    github.com/spf13/cobra@v1.8.1/command.go:1041
github.com/docker/cli/cli-plugins/plugin.RunPlugin(0xc000420c80, 0xc000440908, {{0x5ea29c4a7737, 0x5}, {0x5ea29c4b1636, 0xb}, {0x5ea29c82072c, 0x6}, {0x0, 0x0}, ...})
    github.com/docker/cli@v27.0.3+incompatible/cli-plugins/plugin/plugin.go:79 +0x145
github.com/docker/cli/cli-plugins/plugin.Run(0x5ea29ce58b18, {{0x5ea29c4a7737, 0x5}, {0x5ea29c4b1636, 0xb}, {0x5ea29c82072c, 0x6}, {0x0, 0x0}, {0x0, ...}})
    github.com/docker/cli@v27.0.3+incompatible/cli-plugins/plugin/plugin.go:94 +0x165
main.pluginMain()
    github.com/docker/compose/v2/cmd/main.go:38 +0xa5
main.main()
    github.com/docker/compose/v2/cmd/main.go:98 +0x19c

Removing :ro (the readonly mount option):

d=/tmp/docker-compose-interpolation-test-extended-without-ro

set -e
mkdir "$d"
cd "$d"

cat > docker-compose.yml <<EOF
services:
  foo:
    extends: { file: foo.yml, service: foo }
EOF

cat > foo.yml <<EOF
services:
  foo:
    image: bash
    volumes:
      - \${FOO:-/dev/null}:/tmp/foo
EOF

docker compose config

Result:

invalid interpolation format for services.foo.volumes.[].source.
You may need to escape any $ with another $.
${FOO

And here's a successful interpolation if I don't extend and use a single yml:

d=/tmp/docker-compose-interpolation-test-not-extended-with-ro

set -e
mkdir "$d"
cd "$d"

cat > docker-compose.yml <<EOF
services:
  foo:
    image: bash
    volumes:
      - \${FOO:-/dev/null}:/tmp/foo:ro
EOF

docker compose config

Result:

name: docker-compose-interpolation-test-not-extended-with-ro
services:
  foo:
    image: bash
    networks:
      default: null
    volumes:
      - type: bind
        source: /dev/null
        target: /tmp/foo
        read_only: true
        bind:
          create_host_path: true
networks:
  default:
    name: docker-compose-interpolation-test-not-extended-with-ro_default

Compose Version

$ docker compose version
Docker Compose version 2.29.0
$ docker-compose version
Docker Compose version 2.29.0

Docker Environment

Client:
 Version:    27.0.3
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  0.15.1
    Path:     /usr/lib/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  2.29.0
    Path:     /usr/lib/docker/cli-plugins/docker-compose

Server:
 Containers: 271
  Running: 1
  Paused: 0
  Stopped: 270
 Images: 284
 Server Version: 27.0.3
 Storage Driver: overlay2
  Backing Filesystem: btrfs
  Supports d_type: true
  Using metacopy: true
  Native Overlay Diff: false
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: ae71819c4f5e67bb4d5ae76a6b735f29cc25774e.m
 runc version: 
 init version: de40ad0
 Security Options:
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.9.6-arch1-1
 Operating System: Arch Linux
 OSType: linux
 Architecture: x86_64
 CPUs: 32
 Total Memory: 125.7GiB
 ID: TVST:4T34:N6W7:FSIT:7HOI:IDG3:NUS4:A4AA:SUGG:WO3F:WZPJ:Y73Y
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Live Restore Enabled: false

Anything else?

No response

Metadata

Metadata

Assignees

Labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions