Feat: add metallb on kind / minikube to run e2e locally by lacroixthomas · Pull Request #4318 · agones-dev/agones

lacroixthomas · 2025-10-28T15:45:08Z

What type of PR is this?

Uncomment only one /kind <> line, press enter to put that in a new line, and remove leading whitespace from that line:

/kind documentation
/kind feature

What this PR does / Why we need it:

Added MetalLB in Makefiles for kind and minikube to be able to run e2e tests locally without needing to make a tunnel

Which issue(s) this PR fixes:

Closes #4317

Special notes for your reviewer:

I did not change the current flow from the Makefiles, just added new commands + doc for us to run it locally

markmandel · 2025-10-28T19:10:30Z

🤔 I'm just wondering if metallb should be installed by default?

Any reason not to?

Should we also update https://agones.dev/site/docs/installation/creating-cluster/minikube/ ?

lacroixthomas · 2025-10-28T19:32:00Z

🤔 I'm just wondering if metallb should be installed by default?

Any reason not to?

Should we also update https://agones.dev/site/docs/installation/creating-cluster/minikube/ ?

We could install it by default, but it would only be useful if the make minikube-install or make kind-install use LoadBalancer and not NodePort, could also update it, but could break things if anyone was using it with NodePort (or maybe not ? 🤔)

Oh yep definitively, will add it to this doc

lacroixthomas · 2025-10-30T23:39:35Z

build/includes/minikube.mk

 minikube-install:
 	$(MAKE) install DOCKER_RUN_ARGS="--network=host -v $(minikube_cert_mount)" ALWAYS_PULL_SIDECAR=false \
-		IMAGE_PULL_POLICY=IfNotPresent PING_SERVICE_TYPE=NodePort ALLOCATOR_SERVICE_TYPE=NodePort
+		IMAGE_PULL_POLICY=IfNotPresent PING_SERVICE_TYPE=LoadBalancer ALLOCATOR_SERVICE_TYPE=LoadBalancer


Not sure if we want this to be set to LoadBalancer by default, I've added it on the doc, might gonna rollback to NodePort, WDYT ? @markmandel

I think this should be LB by default - given the other changes, I think this makes sense.

Will try to update this PR today

build/README.md

lacroixthomas · 2025-10-31T19:45:40Z

Still need to update the readme, now that it install MetalLB by default

agones-bot · 2025-11-01T01:00:40Z

Build Succeeded 🥳

Build Id: 85f77368-9300-49bb-8014-1518b419b08b

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.54.0-dev-262d2da
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.54.0-dev-262d2da
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.54.0-dev-262d2da-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.54.0-dev-262d2da
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.54.0-dev-262d2da
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.54.0-dev-262d2da
Linux C++ SDK (build): agonessdk-1.54.0-dev-262d2da-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.54.0-dev-262d2da.zip

A preview of the website (the last 30 builds are retained):

https://262d2da-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4318/head:pr_4318 && git checkout pr_4318
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.54.0-dev-262d2da

agones-bot · 2025-11-01T01:00:41Z

Build Succeeded 🥳

Build Id: 4db6decd-7709-4730-9e5d-2003e4d02e62

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.54.0-dev-7c673e9
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.54.0-dev-7c673e9
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.54.0-dev-7c673e9-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.54.0-dev-7c673e9
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.54.0-dev-7c673e9
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.54.0-dev-7c673e9
Linux C++ SDK (build): agonessdk-1.54.0-dev-7c673e9-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.54.0-dev-7c673e9.zip

A preview of the website (the last 30 builds are retained):

https://7c673e9-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4318/head:pr_4318 && git checkout pr_4318
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.54.0-dev-7c673e9

markmandel

So I just tried this with minikube, and couldn't get a singular e2e test to run (I was running TestAllocatorWithSelectors) without failure.

GOROOT=/home/mark/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.6.linux-amd64 #gosetup
GOPATH=/home/mark/go #gosetup
/home/mark/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.6.linux-amd64/bin/go test -c -race -o /home/mark/.cache/JetBrains/IntelliJIdea2025.2/tmp/GoLand/___TestAllocatorWithSelectors_in_agones_dev_agones_test_e2e.test agones.dev/agones/test/e2e #gosetup
/home/mark/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.6.linux-amd64/bin/go tool test2json -t /home/mark/.cache/JetBrains/IntelliJIdea2025.2/tmp/GoLand/___TestAllocatorWithSelectors_in_agones_dev_agones_test_e2e.test -test.v=test2json -test.paniconexit0 -test.run ^\QTestAllocatorWithSelectors\E$ #gosetup
time="2025-11-01 16:05:15.813" level=info msg="Running test via Intellij. Skipping Test Flag Parsing"
time="2025-11-01 16:05:15.814" level=warning msg="Error creating inClusterConfig, trying to build config from flagsunable to load in-cluster configuration, KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT must be defined" error="unable to load in-cluster configuration, KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT must be defined" source=framework
time="2025-11-01 16:05:15.820" level=info msg="Starting e2e test(s)" cloudProduct=generic featureGates="AutopilotPassthroughPort=true&CountsAndLists=true&DisableResyncOnSDKServer=true&Example=false&FleetAutoscaleRequestMetaData=false&GKEAutopilotExtendedDurationPods=true&PlayerAllocationFilter=false&PlayerTracking=false&PortPolicyNone=true&PortRanges=true&ProcessorAllocator=false&RollingUpdateFix=true&ScheduledAutoscaler=true&SidecarContainers=false&WasmAutoscaler=false" gameServerImage="us-docker.pkg.dev/agones-images/examples/simple-game-server:0.39" namespace= perfOutputDir= pullSecret= stressTestLevel=0 version=
time="2025-11-01 16:05:15.832" level=info msg="Custom namespace is set: 1762038315"
time="2025-11-01 16:05:15.846" level=info msg="Namespace 1762038315 is created"
time="2025-11-01 16:05:15.859" level=info msg="ServiceAccount 1762038315/agones-sdk is created"
time="2025-11-01 16:05:15.863" level=info msg="RoleBinding 1762038315/agones-sdk-access is created"
=== RUN   TestAllocatorWithSelectors
    helper_func.go:113: 
        	Error Trace:	/home/mark/workspace/agones/test/e2e/allochelper/helper_func.go:113
        	            				/home/mark/workspace/agones/test/e2e/allocator_test.go:156
        	Error:      	Not equal: 
        	            	expected: 1
        	            	actual  : 0
        	Test:       	TestAllocatorWithSelectors
--- FAIL: TestAllocatorWithSelectors (0.00s)

Expected :1
Actual   :0

Here's my install console log:

❯ make minikube-test-cluster
mkdir -p ~/.kube/
mkdir -p /home/mark/workspace/agones/build//.gocache
mkdir -p /home/mark/workspace/agones/build//.config/gcloud
mkdir -p ~/.config/helm
mkdir -p ~/.cache/helm
make ensure-image IMAGE_TAG=agones-build:795b8da520 BUILD_TARGET=build-build-image
make[1]: Entering directory '/home/mark/workspace/agones/build'
make[1]: Leaving directory '/home/mark/workspace/agones/build'
minikube start --kubernetes-version v1.32.5 -p agones --driver docker
😄  [agones] minikube v1.37.0 on Debian forky/sid
✨  Using the docker driver based on user configuration
📌  Using Docker driver with root privileges
👍  Starting "agones" primary control-plane node in "agones" cluster
🚜  Pulling base image v0.0.48 ...
🔥  Creating docker container (CPUs=2, Memory=7900MB) ...
🐳  Preparing Kubernetes v1.32.5 on Docker 28.4.0 ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "agones" cluster and "default" namespace by default
make minikube-metallb-helm-install
make[1]: Entering directory '/home/mark/workspace/agones/build'
helm repo add metallb https://metallb.github.io/metallb
"metallb" has been added to your repositories
helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "metallb" chart repository
...Successfully got an update from the "grafana" chart repository
...Successfully got an update from the "prometheus-community" chart repository
...Successfully got an update from the "agones" chart repository
Update Complete. ⎈Happy Helming!⎈
helm install metallb metallb/metallb --namespace metallb-system --create-namespace --version 0.13.12 --wait --timeout 5m
NAME: metallb
LAST DEPLOYED: Sat Nov  1 15:48:22 2025
NAMESPACE: metallb-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
MetalLB is now running in the cluster.

Now you can configure it via its CRs. Please refer to the metallb official docs
on how to use the CRs.
make[1]: Leaving directory '/home/mark/workspace/agones/build'
make minikube-metallb-configure
make[1]: Entering directory '/home/mark/workspace/agones/build'
MINIKUBE_IP=$(minikube ip -p agones); \
NETWORK_PREFIX=$(echo "$MINIKUBE_IP" | cut -d '.' -f 1-3); \
METALLB_IP_RANGE="$NETWORK_PREFIX.50-$NETWORK_PREFIX.250"; \
sed "s|__RANGE__|${METALLB_IP_RANGE}|g" /home/mark/workspace/agones/build//metallb-config.yaml.tpl | kubectl apply -f -
ipaddresspool.metallb.io/default-pool created
l2advertisement.metallb.io/default-advertisement created
make[1]: Leaving directory '/home/mark/workspace/agones/build'

I did go back and realise I needed to install with LB after I had done an install.

❯ make minikube-install PING_SERVICE_TYPE=LoadBalancer ALLOCATOR_SERVICE_TYPE=LoadBalancer
make install DOCKER_RUN_ARGS="--network=host -v ~/.minikube:/home/mark/.minikube" ALWAYS_PULL_SIDECAR=false \
        IMAGE_PULL_POLICY=IfNotPresent PING_SERVICE_TYPE=NodePort ALLOCATOR_SERVICE_TYPE=NodePort
make[1]: Entering directory '/home/mark/workspace/agones/build'
mkdir -p ~/.kube/
mkdir -p /home/mark/workspace/agones/build//.gocache
mkdir -p /home/mark/workspace/agones/build//.config/gcloud
mkdir -p ~/.config/helm
mkdir -p ~/.cache/helm
make ensure-image IMAGE_TAG=agones-build:795b8da520 BUILD_TARGET=build-build-image
make[2]: Entering directory '/home/mark/workspace/agones/build'
make[2]: Leaving directory '/home/mark/workspace/agones/build'
# if IMAGE_PULL_SECRET_FILE is specified, create the agones-system namespace and install the secret
docker run --rm -v /home/mark/workspace/agones/build//.config/gcloud:/root/.config/gcloud -v ~/.kube/:/root/.kube -v ~/.config/helm:/root/.config/helm -v ~/.cache/helm:/root/.cache/helm -v /home/mark/workspace/agones:/go/src/agones.dev/agones -v /home/mark/workspace/agones/build//.gomod:/go/pkg/mod -v /home/mark/workspace/agones/build//.gocache:/root/.cache/go-build -e "KUBECONFIG=/root/.kube/config" -e "GO111MODULE=on" -w /go/src/agones.dev/agones --network=host -v ~/.minikube:/home/mark/.minikube agones-build:795b8da520 bash -c '[[ $(helm status agones -n agones-system --output json | jq -r ".info.status") =~ (failed|pending-.*) ]] && helm uninstall agones --namespace=agones-system || true'
docker run --rm -v /home/mark/workspace/agones/build//.config/gcloud:/root/.config/gcloud -v ~/.kube/:/root/.kube -v ~/.config/helm:/root/.config/helm -v ~/.cache/helm:/root/.cache/helm -v /home/mark/workspace/agones:/go/src/agones.dev/agones -v /home/mark/workspace/agones/build//.gomod:/go/pkg/mod -v /home/mark/workspace/agones/build//.gocache:/root/.cache/go-build -e "KUBECONFIG=/root/.kube/config" -e "GO111MODULE=on" -w /go/src/agones.dev/agones --network=host -v ~/.minikube:/home/mark/.minikube agones-build:795b8da520 \
        helm upgrade --install --atomic --wait --timeout 10m --namespace=agones-system \
        --create-namespace \
        --set agones.image.tag=1.54.0-dev-262d2da,agones.image.registry=us-docker.pkg.dev/agones-mark-dev/images \
        --set agones.image.controller.pullPolicy=IfNotPresent,agones.image.controller.pullSecret= \
        --set agones.image.extensions.pullPolicy=IfNotPresent,agones.image.allocator.pullPolicy=IfNotPresent \
        --set agones.image.ping.pullPolicy=IfNotPresent,agones.image.sdk.alwaysPull=false \
        --set agones.ping.http.serviceType=NodePort,agones.ping.udp.serviceType=NodePort \
        --set agones.allocator.service.serviceType=NodePort \
        --set agones.controller.logLevel="debug" \
        --set agones.crds.cleanupOnDelete=true \
        --set agones.featureGates="PlayerAllocationFilter=true&FleetAutoscaleRequestMetaData=true&PlayerTracking=true&SidecarContainers=true&WasmAutoscaler=true&Example=true" \
        --set agones.allocator.service.loadBalancerIP= \
        --set agones.metrics.serviceMonitor.enabled=false \
         \
        agones /go/src/agones.dev/agones/install/helm/agones/
Release "agones" has been upgraded. Happy Helming!
NAME: agones
LAST DEPLOYED: Sat Nov  1 23:04:18 2025
NAMESPACE: agones-system
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
The Agones components have been installed in the namespace agones-system.

You can get their status by running:
kubectl --namespace agones-system get pods -o wide

Once ready you can create your first GameServer using our examples https://agones.dev/site/docs/getting-started/create-gameserver/ .

Finally don't forget to explore our documentation and usage guides on how to develop and host dedicated game servers on top of Agones:

 - Create a Game Server (https://agones.dev/site/docs/getting-started/create-gameserver/)
 - Integrating the Game Server SDK (https://agones.dev/site/docs/guides/client-sdks/)
 - GameServer Health Checking (https://agones.dev/site/docs/guides/health-checking/)
 - Accessing Agones via the Kubernetes API (https://agones.dev/site/docs/guides/access-api/)
make[1]: Leaving directory '/home/mark/workspace/agones/build'

But as you can see - no external IP for my LB 😞

❯ kubectl get svc --all-namespaces
NAMESPACE        NAME                                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
agones-system    agones-allocator                    NodePort    10.96.153.198    <none>        443:30394/TCP            9m52s
agones-system    agones-allocator-metrics-service    ClusterIP   10.96.169.88     <none>        8080/TCP                 9m52s
agones-system    agones-controller-metrics-service   ClusterIP   10.100.120.175   <none>        8080/TCP                 9m52s
agones-system    agones-controller-service           ClusterIP   10.97.170.186    <none>        443/TCP,8080/TCP         9m52s
agones-system    agones-extensions-metrics-service   ClusterIP   10.110.122.78    <none>        8080/TCP                 9m52s
agones-system    agones-ping-http-service            NodePort    10.108.65.202    <none>        80:31745/TCP             9m52s
agones-system    agones-ping-udp-service             NodePort    10.104.170.182   <none>        50000:30357/UDP          9m52s
default          kubernetes                          ClusterIP   10.96.0.1        <none>        443/TCP                  18m
kube-system      kube-dns                            ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP   18m
metallb-system   metallb-webhook-service             ClusterIP   10.109.162.111   <none>        443/TCP                  18m

🤔 Do I need to uninstall Agones first before re-applying the LB settings on install?

markmandel · 2025-11-01T22:43:10Z

build/README.md

+1. Run the following Makefile targets to install and configure MetalLB:
+   ```sh
+   make minikube-metallb-helm-install
+   make minikube-metallb-configure
+   ```


Do we need this section now it's installed by default?

Not needed anymore 👌🏼

markmandel · 2025-11-01T22:43:36Z

build/README.md

+A minimum of 2-3 nodes is recommended for comprehensive e2e testing:
+
+```bash
+make minikube-test-cluster MINIKUBE_NODES=3 


markmandel · 2025-11-01T22:43:53Z

build/README.md

+
+1. Run the following Makefile targets to install and configure MetalLB:
+   ```sh
+   make kind-metallb-helm-install


Same here - this is on by default now?

Not needed anymore 👌🏼

markmandel · 2025-11-01T22:46:02Z

site/content/en/docs/Installation/Creating Cluster/minikube.md


+For testing scenarios that require multiple nodes, you can add the `--nodes` parameter to create a multi-node cluster.
+```bash
+minikube start --kubernetes-version v{{% minikube-example-cluster-version %}} -p agones --nodes 3


markmandel · 2025-11-01T22:47:27Z

site/content/en/docs/Installation/Creating Cluster/minikube.md


+## Setting up LoadBalancer Support with MetalLB
+
+By default, Minikube doesn't provide LoadBalancer functionality, which means services of type `LoadBalancer` will remain in a "Pending" state. For Agones, LoadBalancer services are often used for the allocator and ping services to provide external access. To enable LoadBalancer support in Minikube, you need to install MetalLB.


Suggested change

By default, Minikube doesn't provide LoadBalancer functionality, which means services of type `LoadBalancer` will remain in a "Pending" state. For Agones, LoadBalancer services are often used for the allocator and ping services to provide external access. To enable LoadBalancer support in Minikube, you need to install MetalLB.

By default, Minikube doesn't provide LoadBalancer's that are exteranally available without some form of tunneling such as `minikube tunnel`, which means services of type `LoadBalancer` will remain in a "Pending" state. For Agones, LoadBalancer services are often used for the allocator and ping services to provide external access. If you want an external IP for those services without a need to tunnel, you need to install MetalLB.

Howzat? I'm trying to explain the use case -- you can have internal load balancers wihout MetalLB -- so we don't want to say that you can't, but if you need external access without a tunnel (for some reason? Now I'm questioning if this is something people will need? WDYT?)

I think it make more sense like that, also, maybe not the you need to install MetalLB. but maybe something like you could install MetalLB (there might be other way to do so) 🤔

markmandel · 2025-11-01T23:04:08Z

build/README.md

+
+2. When installing Agones, set the following variables to use LoadBalancer service type for the Ping and Allocator services:
+   ```sh
+   make minikube-install PING_SERVICE_TYPE=LoadBalancer ALLOCATOR_SERVICE_TYPE=LoadBalancer


I just tried running e2e tests, and ran into this issue -- Given that we're configuring metalLB by default, shall we set these as the default values on minikube-install as well?

Seems to make sense to me, and little downside.

lacroixthomas · 2025-11-05T17:59:34Z

So I just tried this with minikube, and couldn't get a singular e2e test to run (I was running TestAllocatorWithSelectors) without failure.

GOROOT=/home/mark/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.6.linux-amd64 #gosetup
GOPATH=/home/mark/go #gosetup
/home/mark/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.6.linux-amd64/bin/go test -c -race -o /home/mark/.cache/JetBrains/IntelliJIdea2025.2/tmp/GoLand/___TestAllocatorWithSelectors_in_agones_dev_agones_test_e2e.test agones.dev/agones/test/e2e #gosetup
/home/mark/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.24.6.linux-amd64/bin/go tool test2json -t /home/mark/.cache/JetBrains/IntelliJIdea2025.2/tmp/GoLand/___TestAllocatorWithSelectors_in_agones_dev_agones_test_e2e.test -test.v=test2json -test.paniconexit0 -test.run ^\QTestAllocatorWithSelectors\E$ #gosetup
time="2025-11-01 16:05:15.813" level=info msg="Running test via Intellij. Skipping Test Flag Parsing"
time="2025-11-01 16:05:15.814" level=warning msg="Error creating inClusterConfig, trying to build config from flagsunable to load in-cluster configuration, KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT must be defined" error="unable to load in-cluster configuration, KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT must be defined" source=framework
time="2025-11-01 16:05:15.820" level=info msg="Starting e2e test(s)" cloudProduct=generic featureGates="AutopilotPassthroughPort=true&CountsAndLists=true&DisableResyncOnSDKServer=true&Example=false&FleetAutoscaleRequestMetaData=false&GKEAutopilotExtendedDurationPods=true&PlayerAllocationFilter=false&PlayerTracking=false&PortPolicyNone=true&PortRanges=true&ProcessorAllocator=false&RollingUpdateFix=true&ScheduledAutoscaler=true&SidecarContainers=false&WasmAutoscaler=false" gameServerImage="us-docker.pkg.dev/agones-images/examples/simple-game-server:0.39" namespace= perfOutputDir= pullSecret= stressTestLevel=0 version=
time="2025-11-01 16:05:15.832" level=info msg="Custom namespace is set: 1762038315"
time="2025-11-01 16:05:15.846" level=info msg="Namespace 1762038315 is created"
time="2025-11-01 16:05:15.859" level=info msg="ServiceAccount 1762038315/agones-sdk is created"
time="2025-11-01 16:05:15.863" level=info msg="RoleBinding 1762038315/agones-sdk-access is created"
=== RUN   TestAllocatorWithSelectors
    helper_func.go:113: 
        	Error Trace:	/home/mark/workspace/agones/test/e2e/allochelper/helper_func.go:113
        	            				/home/mark/workspace/agones/test/e2e/allocator_test.go:156
        	Error:      	Not equal: 
        	            	expected: 1
        	            	actual  : 0
        	Test:       	TestAllocatorWithSelectors
--- FAIL: TestAllocatorWithSelectors (0.00s)

Expected :1
Actual   :0

Here's my install console log:

❯ make minikube-test-cluster
mkdir -p ~/.kube/
mkdir -p /home/mark/workspace/agones/build//.gocache
mkdir -p /home/mark/workspace/agones/build//.config/gcloud
mkdir -p ~/.config/helm
mkdir -p ~/.cache/helm
make ensure-image IMAGE_TAG=agones-build:795b8da520 BUILD_TARGET=build-build-image
make[1]: Entering directory '/home/mark/workspace/agones/build'
make[1]: Leaving directory '/home/mark/workspace/agones/build'
minikube start --kubernetes-version v1.32.5 -p agones --driver docker
😄  [agones] minikube v1.37.0 on Debian forky/sid
✨  Using the docker driver based on user configuration
📌  Using Docker driver with root privileges
👍  Starting "agones" primary control-plane node in "agones" cluster
🚜  Pulling base image v0.0.48 ...
🔥  Creating docker container (CPUs=2, Memory=7900MB) ...
🐳  Preparing Kubernetes v1.32.5 on Docker 28.4.0 ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: default-storageclass, storage-provisioner
🏄  Done! kubectl is now configured to use "agones" cluster and "default" namespace by default
make minikube-metallb-helm-install
make[1]: Entering directory '/home/mark/workspace/agones/build'
helm repo add metallb https://metallb.github.io/metallb
"metallb" has been added to your repositories
helm repo update
Hang tight while we grab the latest from your chart repositories...
...Successfully got an update from the "metallb" chart repository
...Successfully got an update from the "grafana" chart repository
...Successfully got an update from the "prometheus-community" chart repository
...Successfully got an update from the "agones" chart repository
Update Complete. ⎈Happy Helming!⎈
helm install metallb metallb/metallb --namespace metallb-system --create-namespace --version 0.13.12 --wait --timeout 5m
NAME: metallb
LAST DEPLOYED: Sat Nov  1 15:48:22 2025
NAMESPACE: metallb-system
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
MetalLB is now running in the cluster.

Now you can configure it via its CRs. Please refer to the metallb official docs
on how to use the CRs.
make[1]: Leaving directory '/home/mark/workspace/agones/build'
make minikube-metallb-configure
make[1]: Entering directory '/home/mark/workspace/agones/build'
MINIKUBE_IP=$(minikube ip -p agones); \
NETWORK_PREFIX=$(echo "$MINIKUBE_IP" | cut -d '.' -f 1-3); \
METALLB_IP_RANGE="$NETWORK_PREFIX.50-$NETWORK_PREFIX.250"; \
sed "s|__RANGE__|${METALLB_IP_RANGE}|g" /home/mark/workspace/agones/build//metallb-config.yaml.tpl | kubectl apply -f -
ipaddresspool.metallb.io/default-pool created
l2advertisement.metallb.io/default-advertisement created
make[1]: Leaving directory '/home/mark/workspace/agones/build'

I did go back and realise I needed to install with LB after I had done an install.

❯ make minikube-install PING_SERVICE_TYPE=LoadBalancer ALLOCATOR_SERVICE_TYPE=LoadBalancer
make install DOCKER_RUN_ARGS="--network=host -v ~/.minikube:/home/mark/.minikube" ALWAYS_PULL_SIDECAR=false \
        IMAGE_PULL_POLICY=IfNotPresent PING_SERVICE_TYPE=NodePort ALLOCATOR_SERVICE_TYPE=NodePort
make[1]: Entering directory '/home/mark/workspace/agones/build'
mkdir -p ~/.kube/
mkdir -p /home/mark/workspace/agones/build//.gocache
mkdir -p /home/mark/workspace/agones/build//.config/gcloud
mkdir -p ~/.config/helm
mkdir -p ~/.cache/helm
make ensure-image IMAGE_TAG=agones-build:795b8da520 BUILD_TARGET=build-build-image
make[2]: Entering directory '/home/mark/workspace/agones/build'
make[2]: Leaving directory '/home/mark/workspace/agones/build'
# if IMAGE_PULL_SECRET_FILE is specified, create the agones-system namespace and install the secret
docker run --rm -v /home/mark/workspace/agones/build//.config/gcloud:/root/.config/gcloud -v ~/.kube/:/root/.kube -v ~/.config/helm:/root/.config/helm -v ~/.cache/helm:/root/.cache/helm -v /home/mark/workspace/agones:/go/src/agones.dev/agones -v /home/mark/workspace/agones/build//.gomod:/go/pkg/mod -v /home/mark/workspace/agones/build//.gocache:/root/.cache/go-build -e "KUBECONFIG=/root/.kube/config" -e "GO111MODULE=on" -w /go/src/agones.dev/agones --network=host -v ~/.minikube:/home/mark/.minikube agones-build:795b8da520 bash -c '[[ $(helm status agones -n agones-system --output json | jq -r ".info.status") =~ (failed|pending-.*) ]] && helm uninstall agones --namespace=agones-system || true'
docker run --rm -v /home/mark/workspace/agones/build//.config/gcloud:/root/.config/gcloud -v ~/.kube/:/root/.kube -v ~/.config/helm:/root/.config/helm -v ~/.cache/helm:/root/.cache/helm -v /home/mark/workspace/agones:/go/src/agones.dev/agones -v /home/mark/workspace/agones/build//.gomod:/go/pkg/mod -v /home/mark/workspace/agones/build//.gocache:/root/.cache/go-build -e "KUBECONFIG=/root/.kube/config" -e "GO111MODULE=on" -w /go/src/agones.dev/agones --network=host -v ~/.minikube:/home/mark/.minikube agones-build:795b8da520 \
        helm upgrade --install --atomic --wait --timeout 10m --namespace=agones-system \
        --create-namespace \
        --set agones.image.tag=1.54.0-dev-262d2da,agones.image.registry=us-docker.pkg.dev/agones-mark-dev/images \
        --set agones.image.controller.pullPolicy=IfNotPresent,agones.image.controller.pullSecret= \
        --set agones.image.extensions.pullPolicy=IfNotPresent,agones.image.allocator.pullPolicy=IfNotPresent \
        --set agones.image.ping.pullPolicy=IfNotPresent,agones.image.sdk.alwaysPull=false \
        --set agones.ping.http.serviceType=NodePort,agones.ping.udp.serviceType=NodePort \
        --set agones.allocator.service.serviceType=NodePort \
        --set agones.controller.logLevel="debug" \
        --set agones.crds.cleanupOnDelete=true \
        --set agones.featureGates="PlayerAllocationFilter=true&FleetAutoscaleRequestMetaData=true&PlayerTracking=true&SidecarContainers=true&WasmAutoscaler=true&Example=true" \
        --set agones.allocator.service.loadBalancerIP= \
        --set agones.metrics.serviceMonitor.enabled=false \
         \
        agones /go/src/agones.dev/agones/install/helm/agones/
Release "agones" has been upgraded. Happy Helming!
NAME: agones
LAST DEPLOYED: Sat Nov  1 23:04:18 2025
NAMESPACE: agones-system
STATUS: deployed
REVISION: 2
TEST SUITE: None
NOTES:
The Agones components have been installed in the namespace agones-system.

You can get their status by running:
kubectl --namespace agones-system get pods -o wide

Once ready you can create your first GameServer using our examples https://agones.dev/site/docs/getting-started/create-gameserver/ .

Finally don't forget to explore our documentation and usage guides on how to develop and host dedicated game servers on top of Agones:

 - Create a Game Server (https://agones.dev/site/docs/getting-started/create-gameserver/)
 - Integrating the Game Server SDK (https://agones.dev/site/docs/guides/client-sdks/)
 - GameServer Health Checking (https://agones.dev/site/docs/guides/health-checking/)
 - Accessing Agones via the Kubernetes API (https://agones.dev/site/docs/guides/access-api/)
make[1]: Leaving directory '/home/mark/workspace/agones/build'

But as you can see - no external IP for my LB 😞

❯ kubectl get svc --all-namespaces
NAMESPACE        NAME                                TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                  AGE
agones-system    agones-allocator                    NodePort    10.96.153.198    <none>        443:30394/TCP            9m52s
agones-system    agones-allocator-metrics-service    ClusterIP   10.96.169.88     <none>        8080/TCP                 9m52s
agones-system    agones-controller-metrics-service   ClusterIP   10.100.120.175   <none>        8080/TCP                 9m52s
agones-system    agones-controller-service           ClusterIP   10.97.170.186    <none>        443/TCP,8080/TCP         9m52s
agones-system    agones-extensions-metrics-service   ClusterIP   10.110.122.78    <none>        8080/TCP                 9m52s
agones-system    agones-ping-http-service            NodePort    10.108.65.202    <none>        80:31745/TCP             9m52s
agones-system    agones-ping-udp-service             NodePort    10.104.170.182   <none>        50000:30357/UDP          9m52s
default          kubernetes                          ClusterIP   10.96.0.1        <none>        443/TCP                  18m
kube-system      kube-dns                            ClusterIP   10.96.0.10       <none>        53/UDP,53/TCP,9153/TCP   18m
metallb-system   metallb-webhook-service             ClusterIP   10.109.162.111   <none>        443/TCP                  18m

🤔 Do I need to uninstall Agones first before re-applying the LB settings on install?

Hmm good question, you might need to re-install it if it was setup with NodePort, will need to double check that

agones-bot · 2025-11-05T18:26:05Z

Build Failed 😭

Build Id: b656f6dc-eb75-43d4-8314-eb66cda00c31

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

lacroixthomas · 2025-11-05T18:31:30Z

/gcbrun

agones-bot · 2025-11-05T20:12:14Z

Build Succeeded 🥳

Build Id: f7b6fbe4-777b-453d-acf4-2de52470d703

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.54.0-dev-68760e4
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.54.0-dev-68760e4
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.54.0-dev-68760e4-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.54.0-dev-68760e4
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.54.0-dev-68760e4
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.54.0-dev-68760e4
Linux C++ SDK (build): agonessdk-1.54.0-dev-68760e4-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.54.0-dev-68760e4.zip

A preview of the website (the last 30 builds are retained):

https://68760e4-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4318/head:pr_4318 && git checkout pr_4318
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.54.0-dev-68760e4

agones-bot · 2025-11-06T01:50:07Z

Build Succeeded 🥳

Build Id: f980d8db-cd5d-44cf-92b1-f271b1ecb1c7

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.54.0-dev-bb575d0
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.54.0-dev-bb575d0
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.54.0-dev-bb575d0-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.54.0-dev-bb575d0
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.54.0-dev-bb575d0
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.54.0-dev-bb575d0
Linux C++ SDK (build): agonessdk-1.54.0-dev-bb575d0-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.54.0-dev-bb575d0.zip

A preview of the website (the last 30 builds are retained):

https://bb575d0-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4318/head:pr_4318 && git checkout pr_4318
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.54.0-dev-bb575d0

markmandel

Boom! Can confirm this works. Amazing.

Only thing needed is that Apache header and we are good to merge.

markmandel · 2025-11-07T17:20:12Z

build/metallb-config.yaml.tpl

@@ -0,0 +1,17 @@
+apiVersion: metallb.io/v1beta1


Apache header plz

markmandel · 2025-11-07T17:20:40Z

site/content/en/docs/Installation/Creating Cluster/minikube.md


+## Setting up LoadBalancer Support with MetalLB
+
+By default, Minikube doesn't provide LoadBalancer's that are exteranally available without some form of tunneling such as `minikube tunnel`, which means services of type `LoadBalancer` will remain in a "Pending" state. For Agones, LoadBalancer services are often used for the allocator and ping services to provide external access. If you want an external IP for those services without a need to tunnel, you need to install MetalLB.


agones-bot · 2025-11-07T20:12:19Z

Build Failed 😭

Build Id: 7bf12bb8-d5ee-4c29-98f5-e72d139b090d

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

markmandel · 2025-11-07T20:24:20Z

/gcbrun

agones-bot · 2025-11-07T21:59:47Z

Build Succeeded 🥳

Build Id: 149b559c-1a26-4ec8-aa08-f5ba27f9bb02

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.54.0-dev-2bca1d9
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.54.0-dev-2bca1d9
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.54.0-dev-2bca1d9-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.54.0-dev-2bca1d9
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.54.0-dev-2bca1d9
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.54.0-dev-2bca1d9
Linux C++ SDK (build): agonessdk-1.54.0-dev-2bca1d9-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.54.0-dev-2bca1d9.zip

A preview of the website (the last 30 builds are retained):

https://2bca1d9-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4318/head:pr_4318 && git checkout pr_4318
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.54.0-dev-2bca1d9

) * feat: add metallb on kind / minikube to run e2e locally * feat: enable multi node for minikube and default with metallb * feat: add doc * feat: update doc around minikube and e2e * feat: update doc around minikube and e2e * feat: make env variable consistent from the doc * feat: rollback to nodeport * feat: make loadbalancer type by default for minikube / kind * feat: update doc * feat: fix the minikube setup with env vars * feat: fix the kind setup with env vars * feat: remove not needed env vars * feat: ensure all override scenarios works * feat: add missing apache header

lacroixthomas requested review from igooch and markmandel October 28, 2025 15:45

github-actions bot added kind/documentation Documentation for Agones size/M labels Oct 28, 2025

lacroixthomas commented Oct 30, 2025

View reviewed changes

build/README.md Outdated Show resolved Hide resolved

markmandel reviewed Nov 1, 2025

View reviewed changes

lacroixthomas and others added 9 commits November 5, 2025 13:00

feat: add metallb on kind / minikube to run e2e locally

a0ca85d

feat: enable multi node for minikube and default with metallb

608d00b

feat: add doc

71815d6

feat: update doc around minikube and e2e

d427a04

feat: update doc around minikube and e2e

f8c9256

feat: make env variable consistent from the doc

b6a3f9e

feat: rollback to nodeport

cfe1802

feat: make loadbalancer type by default for minikube / kind

049bfe5

feat: update doc

68760e4

lacroixthomas force-pushed the features/e2e-test-locally branch from 05aaa45 to 68760e4 Compare November 5, 2025 18:01

lacroixthomas added 4 commits November 5, 2025 19:12

feat: fix the minikube setup with env vars

e5e809a

feat: fix the kind setup with env vars

4533c39

feat: remove not needed env vars

2459913

feat: ensure all override scenarios works

9b1209a

Merge branch 'main' into features/e2e-test-locally

bb575d0

markmandel reviewed Nov 7, 2025

View reviewed changes

feat: add missing apache header

2bca1d9

markmandel approved these changes Nov 7, 2025

View reviewed changes

markmandel enabled auto-merge (squash) November 7, 2025 20:24

markmandel merged commit c202b19 into agones-dev:main Nov 7, 2025
4 checks passed


		## Setting up LoadBalancer Support with MetalLB

		By default, Minikube doesn't provide LoadBalancer functionality, which means services of type `LoadBalancer` will remain in a "Pending" state. For Agones, LoadBalancer services are often used for the allocator and ping services to provide external access. To enable LoadBalancer support in Minikube, you need to install MetalLB.

Conversation

lacroixthomas commented Oct 28, 2025

Uh oh!

markmandel commented Oct 28, 2025

Uh oh!

lacroixthomas commented Oct 28, 2025

Uh oh!

lacroixthomas Oct 30, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

lacroixthomas commented Oct 31, 2025

Uh oh!

agones-bot commented Nov 1, 2025

Uh oh!

agones-bot commented Nov 1, 2025

Uh oh!

markmandel left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lacroixthomas commented Nov 5, 2025

Uh oh!

agones-bot commented Nov 5, 2025

Uh oh!

lacroixthomas commented Nov 5, 2025

Uh oh!

agones-bot commented Nov 5, 2025

Uh oh!

agones-bot commented Nov 6, 2025

Uh oh!

markmandel left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

agones-bot commented Nov 7, 2025

Uh oh!

markmandel commented Nov 7, 2025

Uh oh!

agones-bot commented Nov 7, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

lacroixthomas Oct 30, 2025 •

edited

Loading