kubernetes driver fails to configure multi-node setup due to missing node option #215
Description
Behaviour
Steps to reproduce this issue
- name: Set up Docker Buildx
uses: docker/setup-buildx-action@v2
with:
driver: kubernetes
platforms: linux/amd64
driver-opts: |
"namespace=buildkit"
"nodeselector=""kubernetes.io/arch=amd64"""
append: |
- platforms: linux/arm64
driver-opts:
- '"namespace=buildkit"'
- '"nodeselector=""kubernetes.io/arch=arm64"""'Expected behaviour
Create a working builder with an arm64 and an amd64 node
Actual behaviour
Creating a new builder instance
/usr/bin/docker buildx create --name builder-ec5ab75e-cc9a-4e71-b560-72962afcfd54 --driver kubernetes --driver-opt "namespace=buildkit" --driver-opt "nodeselector=""kubernetes.io/arch=arm64""" --buildkitd-flags --allow-insecure-entitlement security.insecure --allow-insecure-entitlement network.host --platform linux/amd64 --use
builder-ec5ab75e-cc9a-4e71-b560-72962afcfd54
Appending node(s) to builder
/usr/bin/docker buildx create --name builder-ec5ab75e-cc9a-4e71-b560-72962afcfd54 --append --driver-opt "namespace=buildkit" --driver-opt "nodeselector=""kubernetes.io/arch=arm64""" --platform linux/arm64
ERROR: invalid duplicate endpoint kubernetes:///builder-ec5ab75e-cc9a-4e71-b560-72962afcfd54?deployment=&kubeconfig=
Workaround
this seems to be working:
- name: Set up Docker Buildx
id: setup-builder
run: |
UUID=$(uuidgen)
BUILDER=$(docker buildx create --name builder-${UUID} --node builder-amd64-${UUID} --driver kubernetes --driver-opt "namespace=buildkit" --driver-opt "nodeselector=""kubernetes.io/arch=amd64""" --platform linux/amd64 --use)
docker buildx create --name builder-${UUID} --node builder-arm64-${UUID} --append --driver-opt "namespace=buildkit" --driver-opt "nodeselector=""kubernetes.io/arch=arm64""" --platform linux/arm64
docker buildx inspect
echo "name=${BUILDER}" >> $GITHUB_OUTPUT
- name: Docker meta
id: meta
uses: docker/metadata-action@v4
with:
images: ghcr.io/.../...
tags: |
type=ref,event=branch
type=ref,event=pr
type=semver,pattern={{version}}
type=semver,pattern={{major}}.{{minor}}
- name: Login to GitHub Container Registry
uses: docker/login-action@v2
with:
registry: ghcr.io
username: ${{ github.actor }}
password: ${{ secrets.GITHUB_TOKEN }}
- name: Build and push
id: push
uses: docker/build-push-action@v4
with:
builder: ${{ steps.setup-builder.outputs.name }}
context: .
cache-from: type=gha
cache-to: type=gha,mode=max
file: build/package/Dockerfile
push: ${{ github.event_name != 'pull_request' }}
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
platforms: linux/amd64,linux/arm64
- name: Clean up Docker Buildx
if: ${{ always() }}
run: |
docker buildx rm ${{ steps.setup-builder.outputs.name }}