Add GHA workflow to build image by vvoland · Pull Request #647 · moby/vpnkit

vvoland · 2025-04-03T10:37:19Z

Based off: https://github.com/moby/moby/blob/master/.github/workflows/bin-image.yml

To avoid it being confused as a git repository. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

3.20 fails Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

vvoland · 2025-04-07T12:27:57Z

Workflow won't run in this PR before merging. Workflow tested in other repo: https://github.com/docker/vvoland-gha-tests/actions/runs/14308322398

thaJeztah · 2025-04-07T13:33:14Z

.github/workflows/bin-image.yml

+jobs:
+  prepare:
+    runs-on: ubuntu-24.04
+    timeout-minutes: 20 # guardrails timeout for the whole job


Is 20 minutes enough for this one? I recall you mentioned building the image here was rather slow

It's only the prepare step. Build has bigger timeout:

vpnkit/.github/workflows/bin-image.yml

Line 71 in 207eb9f

timeout-minutes: 120 # guardrails timeout for the whole job

OTOH, it could even be lower (like 60 minutes).

The workflow is much faster now (it took ~10 minutes in the private repo) thanks to usage of the native github runner:

vpnkit/.github/workflows/bin-image.yml

Line 70 in 207eb9f

runs-on: ${{ contains(matrix.platform, 'arm') && 'ubuntu-24.04-arm' || 'ubuntu-24.04' }}

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

thaJeztah · 2025-04-08T09:31:12Z

I tried if I could modernise the Dockerfile a bit, and to add build-args to allow overriding the versions (also getting rid of that hard-coded /home/opam/.opam/4.14/bin/vpnkit path, but ran into issues with permissions because the image runs as opam, which didn't have permissions to create the output dir 😂

For posterity; this is what I had;

# syntax=docker/dockerfile:1

ARG ALPINE_VERSION=3.19
ARG OCAML_VERSION=4.14
ARG OCAML_TAG=alpine-${ALPINE_VERSION}-ocaml-${OCAML_VERSION}

FROM ocaml/opam:${OCAML_TAG} AS build
RUN opam update

COPY --link . /home/opam/vpnkit
RUN opam pin add vpnkit /home/opam/vpnkit --kind=path -n
RUN opam depext vpnkit -y
RUN opam install vpnkit -y --destdir /home/opam/build

FROM scratch AS binary
COPY --link --from=build /home/opam/build/bin/vpnkit /vpnkit

FROM alpine:latest
COPY --link --from=binary /vpnkit /vpnkit

thaJeztah

LGTM

@crazy-max @djs55 want to give this a second pair of eyes?

crazy-max · 2025-04-08T09:43:51Z

.github/workflows/bin-image.yml

+  prepare:
+    runs-on: ubuntu-24.04
+    timeout-minutes: 20
+    outputs:
+      platforms: ${{ steps.platforms.outputs.matrix }}
+    steps:
+      -
+        name: Checkout
+        uses: actions/checkout@v4
+      -
+        name: Docker meta
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          images: |
+            ${{ env.REPO_SLUG }}
+          tags: |
+            type=semver,pattern={{version}}
+            type=ref,event=branch
+            type=ref,event=pr
+            type=sha
+      -
+        name: Rename meta bake definition file
+        # see https://github.com/docker/metadata-action/issues/381#issuecomment-1918607161
+        run: |
+          bakeFile="${{ steps.meta.outputs.bake-file }}"
+          mv "${bakeFile#cwd://}" "/tmp/bake-meta.json"
+      -
+        name: Upload meta bake definition
+        uses: actions/upload-artifact@v4
+        with:
+          name: bake-meta
+          path: /tmp/bake-meta.json
+          if-no-files-found: error
+          retention-days: 1
+      -
+        name: Create platforms matrix
+        id: platforms
+        run: |
+          echo "matrix=$(docker buildx bake bin-image --print | jq -cr '.target."bin-image".platforms')" >>${GITHUB_OUTPUT}
+
+  build:
+    runs-on: ${{ contains(matrix.platform, 'arm') && 'ubuntu-24.04-arm' || 'ubuntu-24.04' }}
+    timeout-minutes: 45 # guardrails timeout for the whole job
+    needs:
+      - prepare
+    if: always() && !contains(needs.*.result, 'failure') && !contains(needs.*.result, 'cancelled')
+    strategy:
+      fail-fast: false
+      matrix:
+        platform: ${{ fromJson(needs.prepare.outputs.platforms) }}
+    steps:
+      -
+        name: Checkout
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      -
+        name: Prepare
+        run: |
+          platform=${{ matrix.platform }}
+          echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV
+      -
+        name: Download meta bake definition
+        uses: actions/download-artifact@v4
+        with:
+          name: bake-meta
+          path: /tmp
+      -
+        name: Set up QEMU
+        uses: docker/setup-qemu-action@v3
+      -
+        name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+        with:
+          buildkitd-flags: --debug
+      -
+        name: Login to Docker Hub
+        if: github.event_name != 'pull_request' && github.repository == 'moby/vpnkit'
+        uses: docker/login-action@v3
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+      -
+        name: Build
+        id: bake
+        uses: docker/bake-action@v6
+        with:
+          source: .
+          files: |
+            ./docker-bake.hcl
+            /tmp/bake-meta.json
+          targets: bin-image
+          set: |
+            *.platform=${{ matrix.platform }}
+            *.output=type=image,name=${{ env.REPO_SLUG }},push-by-digest=true,name-canonical=true,push=${{ github.event_name != 'pull_request' && github.repository == 'moby/vpnkit' }}
+            *.tags=
+      -
+        name: Export digest
+        if: github.event_name != 'pull_request' && github.repository == 'moby/vpnkit'
+        run: |
+          mkdir -p /tmp/digests
+          digest="${{ fromJSON(steps.bake.outputs.metadata)['bin-image']['containerimage.digest'] }}"
+          touch "/tmp/digests/${digest#sha256:}"
+      -
+        name: Upload digest
+        if: github.event_name != 'pull_request' && github.repository == 'moby/vpnkit'
+        uses: actions/upload-artifact@v4
+        with:
+          name: digests-${{ env.PLATFORM_PAIR }}
+          path: /tmp/digests/*
+          if-no-files-found: error
+          retention-days: 1
+
+  merge:
+    runs-on: ubuntu-24.04
+    timeout-minutes: 20
+    needs:
+      - build
+    if: always() && !contains(needs.*.result, 'failure') && !contains(needs.*.result, 'cancelled') && github.event_name != 'pull_request' && github.repository == 'moby/vpnkit'
+    steps:
+      -
+        name: Download meta bake definition
+        uses: actions/download-artifact@v4
+        with:
+          name: bake-meta
+          path: /tmp
+      -
+        name: Download digests
+        uses: actions/download-artifact@v4
+        with:
+          path: /tmp/digests
+          pattern: digests-*
+          merge-multiple: true
+      -
+        name: Set up Docker Buildx
+        uses: docker/setup-buildx-action@v3
+        with:
+          buildkitd-flags: --debug
+      -
+        name: Login to Docker Hub
+        uses: docker/login-action@v3
+        with:
+          username: ${{ secrets.DOCKERHUB_USERNAME }}
+          password: ${{ secrets.DOCKERHUB_TOKEN }}
+      -
+        name: Create manifest list and push
+        working-directory: /tmp/digests
+        run: |
+          set -x
+          docker buildx imagetools create $(jq -cr '.target."docker-metadata-action".tags | map("-t " + .) | join(" ")' /tmp/bake-meta.json) \
+            $(printf '${{ env.REPO_SLUG }}@sha256:%s ' *)
+      -
+        name: Inspect image
+        run: |
+          set -x
+          docker buildx imagetools inspect ${{ env.REPO_SLUG }}:$(jq -cr '.target."docker-metadata-action".args.DOCKER_META_VERSION' /tmp/bake-meta.json)


I think we could use this reusable workflow https://github.com/crazy-max/.github?tab=readme-ov-file#bake-distribute-mp:

Suggested change

prepare:

runs-on: ubuntu-24.04

timeout-minutes: 20

outputs:

platforms: ${{ steps.platforms.outputs.matrix }}

steps:

-

name: Checkout

uses: actions/checkout@v4

-

name: Docker meta

id: meta

uses: docker/metadata-action@v5

with:

images: |

${{ env.REPO_SLUG }}

tags: |

type=semver,pattern={{version}}

type=ref,event=branch

type=ref,event=pr

type=sha

-

name: Rename meta bake definition file

# see https://github.com/docker/metadata-action/issues/381#issuecomment-1918607161

run: |

bakeFile="${{ steps.meta.outputs.bake-file }}"

mv "${bakeFile#cwd://}" "/tmp/bake-meta.json"

-

name: Upload meta bake definition

uses: actions/upload-artifact@v4

with:

name: bake-meta

path: /tmp/bake-meta.json

if-no-files-found: error

retention-days: 1

-

name: Create platforms matrix

id: platforms

run: |

echo "matrix=$(docker buildx bake bin-image --print | jq -cr '.target."bin-image".platforms')" >>${GITHUB_OUTPUT}

build:

runs-on: ${{ contains(matrix.platform, 'arm') && 'ubuntu-24.04-arm' || 'ubuntu-24.04' }}

timeout-minutes: 45 # guardrails timeout for the whole job

needs:

- prepare

if: always() && !contains(needs.*.result, 'failure') && !contains(needs.*.result, 'cancelled')

strategy:

fail-fast: false

matrix:

platform: ${{ fromJson(needs.prepare.outputs.platforms) }}

steps:

-

name: Checkout

uses: actions/checkout@v4

with:

fetch-depth: 0

-

name: Prepare

run: |

platform=${{ matrix.platform }}

echo "PLATFORM_PAIR=${platform//\//-}" >> $GITHUB_ENV

-

name: Download meta bake definition

uses: actions/download-artifact@v4

with:

name: bake-meta

path: /tmp

-

name: Set up QEMU

uses: docker/setup-qemu-action@v3

-

name: Set up Docker Buildx

uses: docker/setup-buildx-action@v3

with:

buildkitd-flags: --debug

-

name: Login to Docker Hub

if: github.event_name != 'pull_request' && github.repository == 'moby/vpnkit'

uses: docker/login-action@v3

with:

username: ${{ secrets.DOCKERHUB_USERNAME }}

password: ${{ secrets.DOCKERHUB_TOKEN }}

-

name: Build

id: bake

uses: docker/bake-action@v6

with:

source: .

files: |

./docker-bake.hcl

/tmp/bake-meta.json

targets: bin-image

set: |

*.platform=${{ matrix.platform }}

*.output=type=image,name=${{ env.REPO_SLUG }},push-by-digest=true,name-canonical=true,push=${{ github.event_name != 'pull_request' && github.repository == 'moby/vpnkit' }}

*.tags=

-

name: Export digest

if: github.event_name != 'pull_request' && github.repository == 'moby/vpnkit'

run: |

mkdir -p /tmp/digests

digest="${{ fromJSON(steps.bake.outputs.metadata)['bin-image']['containerimage.digest'] }}"

touch "/tmp/digests/${digest#sha256:}"

-

name: Upload digest

if: github.event_name != 'pull_request' && github.repository == 'moby/vpnkit'

uses: actions/upload-artifact@v4

with:

name: digests-${{ env.PLATFORM_PAIR }}

path: /tmp/digests/*

if-no-files-found: error

retention-days: 1

merge:

runs-on: ubuntu-24.04

timeout-minutes: 20

needs:

- build

if: always() && !contains(needs.*.result, 'failure') && !contains(needs.*.result, 'cancelled') && github.event_name != 'pull_request' && github.repository == 'moby/vpnkit'

steps:

-

name: Download meta bake definition

uses: actions/download-artifact@v4

with:

name: bake-meta

path: /tmp

-

name: Download digests

uses: actions/download-artifact@v4

with:

path: /tmp/digests

pattern: digests-*

merge-multiple: true

-

name: Set up Docker Buildx

uses: docker/setup-buildx-action@v3

with:

buildkitd-flags: --debug

-

name: Login to Docker Hub

uses: docker/login-action@v3

with:

username: ${{ secrets.DOCKERHUB_USERNAME }}

password: ${{ secrets.DOCKERHUB_TOKEN }}

-

name: Create manifest list and push

working-directory: /tmp/digests

run: |

set -x

docker buildx imagetools create $(jq -cr '.target."docker-metadata-action".tags | map("-t " + .) | join(" ")' /tmp/bake-meta.json) \

$(printf '${{ env.REPO_SLUG }}@sha256:%s ' *)

-

name: Inspect image

run: |

set -x

docker buildx imagetools inspect ${{ env.REPO_SLUG }}:$(jq -cr '.target."docker-metadata-action".args.DOCKER_META_VERSION' /tmp/bake-meta.json)

build:

uses: crazy-max/.github/.github/workflows/bake-distribute-mp.yml@d9a10e2737504a6e253f96e344cef684b0e00cb5

with:

target: bin-image

push: ${{ github.event_name != 'pull_request' && github.repository == 'moby/vpnkit' }}

cache: true

meta-image: moby/vpnkit

meta-tags: |

type=semver,pattern={{version}}

type=ref,event=branch

type=ref,event=pr

type=sha

secrets:

login-username: ${{ secrets.DOCKERHUB_USERNAME }}

login-password: ${{ secrets.DOCKERHUB_TOKEN }}

Reminds me I need to update moby/moby#49474

Run example here: https://github.com/crazy-max/.github/actions/runs/14197631829/job/39776518851

That won't allow to choose a correct gha runner though?

vpnkit/.github/workflows/bin-image.yml

Line 70 in 3d4f258

runs-on: ${{ contains(matrix.platform, 'arm') && 'ubuntu-24.04-arm' || 'ubuntu-24.04' }}

It's kinda important here because otherwise the build takes ~30 minutes instead of ~5 😅

Ah, actually it does handle it too (runner defaults to auto), nice!

Ah, actually it does handle it too (runner defaults to auto), nice!

Yes indeed auto choose based on target platform

Looks like the warmup stage only builds one platform: https://github.com/docker/vvoland-gha-tests/actions/runs/14330651343/job/40165666013

Effectively this turns a parallel build into sequential as the linux/arm64 doesn't run until the warmup stage finishes (which caches the linux/amd64).

Is there a way to disable the warmup without disabling the gha cache completely?

Updated to use your workflow, I did set cache to false though.

Looks like the warmup stage only builds one platform: https://github.com/docker/vvoland-gha-tests/actions/runs/14330651343/job/40165666013

Effectively this turns a parallel build into sequential as the linux/arm64 doesn't run until the warmup stage finishes (which caches the linux/amd64).

Is there a way to disable the warmup without disabling the gha cache completely?

I can add an option for that, will take a look

djs55

Great, thanks!

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

crazy-max · 2025-04-08T11:09:47Z

.github/workflows/bin-image.yml

+      target: bin-image
+      push: ${{ github.event_name != 'pull_request' && github.repository == 'moby/vpnkit' }}
+      cache: false # See: https://github.com/moby/vpnkit/pull/647/files/3d4f258e7514b9cc878639f724cbb0caffa8fd98#r2032880337
+      meta-image: moby/vpnkit


Ah looking at https://github.com/moby/vpnkit/actions/runs/14331701104/job/40169030245#step:7:764 I think it should be moby/vpnkit-bin

vvoland marked this pull request as draft April 3, 2025 10:42

vvoland force-pushed the gha-binimg branch 3 times, most recently from af05dc3 to 8640481 Compare April 3, 2025 11:53

vvoland marked this pull request as ready for review April 3, 2025 14:53

vvoland added 5 commits April 7, 2025 14:19

Dockerfile: Use explicit --kind=path when pinning package

600556f

To avoid it being confused as a git repository. Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

gha: Build and push image

5ebdd9f

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

Dockerfile: Fix AS casing

b63fd29

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

Bump alpine to 3.19

ca89698

3.20 fails Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

gha: Use arm worker depending on platform

207eb9f

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

vvoland force-pushed the gha-binimg branch from 6ed0260 to 207eb9f Compare April 7, 2025 12:19

vvoland requested review from djs55 and thaJeztah April 7, 2025 12:28

thaJeztah reviewed Apr 7, 2025

View reviewed changes

gha: Lower timeouts

3d4f258

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

vvoland requested a review from thaJeztah April 7, 2025 14:09

thaJeztah approved these changes Apr 8, 2025

View reviewed changes

crazy-max reviewed Apr 8, 2025

View reviewed changes

djs55 approved these changes Apr 8, 2025

View reviewed changes

gha: Use crazy-max/bake-distribute-mp

d78ded9

Signed-off-by: Paweł Gronowski <pawel.gronowski@docker.com>

crazy-max approved these changes Apr 8, 2025

View reviewed changes

djs55 merged commit 77a946e into moby:master Apr 8, 2025
1 of 4 checks passed

crazy-max reviewed Apr 8, 2025

View reviewed changes

Conversation

vvoland commented Apr 3, 2025

Uh oh!

vvoland commented Apr 7, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

vvoland Apr 7, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

thaJeztah commented Apr 8, 2025

Uh oh!

thaJeztah left a comment

Choose a reason for hiding this comment

Uh oh!

crazy-max Apr 8, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

djs55 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

vvoland Apr 7, 2025 •

edited

Loading

crazy-max Apr 8, 2025 •

edited

Loading