What is the problem you're trying to solve

With multi-stage builds, it's useful to be able to e.g. access an SSH server only from within the build image. Docker now supports that with BuildKit. It adds new command line flags and Dockerfile syntax. Compose should allow defining build secrets, just like it supports runtime secrets.

Building using BuildKit is enabled through an environment variable (DOCKER_BUILDKIT=1) or a Docker configuration file setting.

Compose already supports runtime secrets, which are files "mounted" inside a container. BuildKit seems very similar, but with some differences:

1. --ssh, which simply mounts a Unix socket and sets the SSH_AUTH_SOCK environment variable for RUN commands that request it.
2. The dst flag, which specifies an alternative directory (default is still /run/secrets/).
3. Though it's undocumented, it seems setting owner and mode is supported.

Describe the solution you'd like

This is a high-level discussion around the requirements of a solution. A more concrete suggestion would probably be more suited for a PR.

Tie in to the existing secrets. A file secret is defined by

• Type, but only file is currently supported
• ID (used to reference the secret in a RUN
• Source (the file path)

An SSH secret is defined by

• ID
• Agent socket paths

Both of them are referenced/dispatched by specifying

• Source (only for file secrets; implied for SSH)
• Target
• Required (boolean)
• UID
• GID
• Mode

This seems similar enough that it should be possible to extend the secrets section, and allow referencing it in an image's build section. The SSH type is specific to BuildKit, so parsing code would have to know which type is allowed in build/runtime. Using a separate section for build secrets is a possibility.

Lastly, adding a "builder" selector that allows choosing "buildkit" as the builder could set the right environment variable to allow parsing these flags. (The Dockerfile still needs to have the syntax line to support the RUN --mount variant.)

Additional context

A spec PR was suggested as the first step, in docker/compose#7296 (comment).

The --secret command line option is parsed into a github.com/moby/buildkit/session/secrets/secretsprovider.FileSource. The --ssh option is parsed into a github.com/moby/buildkit/session/sshforward/sshprovider.AgentConfig.

The secret mount is dispatched at https://github.com/moby/buildkit/blob/195f9e598cc7ba82aabefe60d564df7329cb38dd/frontend/dockerfile/dockerfile2llb/convert_secrets.go, and the ssh mount is dispatched at https://github.com/moby/buildkit/blob/195f9e598cc7ba82aabefe60d564df7329cb38dd/frontend/dockerfile/dockerfile2llb/convert_ssh.go.