flakey: add resource requirements to SuperTuxKart e2e test by markmandel · Pull Request #4280 · agones-dev/agones

markmandel · 2025-09-18T04:32:46Z

What type of PR is this?

Uncomment only one /kind <> line, press enter to put that in a new line, and remove leading whitespace from that line:

/kind breaking

/kind bug

/kind cleanup
/kind documentation
/kind feature
/kind hotfix
/kind release

What this PR does / Why we need it:

This test has been flaky on GKE Autopilot. Adding explicit memory (700Mi) and CPU (200m) resource requests to the container to ensure adequate resource allocation and improve test stability on GKE Autopilot clusters. 🤞🏻

Which issue(s) this PR fixes:

N/A

Special notes for your reviewer:

This is the same as what we do for Xonotic and the Xonotic one doesn't flake.

agones-bot · 2025-09-18T05:53:47Z

Build Failed 😭

Build Id: 1c170c4e-df97-406d-aa3c-77fa493a13e0

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

markmandel · 2025-09-18T16:26:50Z

Same flake:

VERBOSE: === RUN   TestSuperTuxKartGameServerReady
VERBOSE: === PAUSE TestSuperTuxKartGameServerReady
VERBOSE: === CONT  TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:52:58.430" level=info msg="GameServer created, waiting for Ready" gs=supertuxkart-qwhww test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:52:58.530" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Creating gs=supertuxkart-qwhww test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:52:59.509" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=supertuxkart-qwhww test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:53:00.499" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=supertuxkart-qwhww test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:53:01.500" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-qwhww test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:53:02.504" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-qwhww test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:53:03.500" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-qwhww test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:53:04.498" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-qwhww test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:53:05.500" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-qwhww test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:53:06.504" level=error msg="GameServer reached terminal state" awaitingState=Ready currentState=Unhealthy gs=supertuxkart-qwhww test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:53:06.505" level=info msg="Game Server Events:" test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:53:06.505" level=info msg="Dumping Events:" kind= test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:53:06.603" level=info msg="Event!" lastTimestamp="2025-09-18 05:52:58 +0000 UTC" message="Port allocated" reason=PortAllocation test=TestSuperTuxKartGameServerReady type=Normal
VERBOSE: time="2025-09-18 05:53:06.604" level=info msg="Event!" lastTimestamp="2025-09-18 05:52:58 +0000 UTC" message="Pod supertuxkart-qwhww created" reason=Creating test=TestSuperTuxKartGameServerReady type=Normal
VERBOSE: time="2025-09-18 05:53:06.604" level=info msg="Event!" lastTimestamp="2025-09-18 05:53:00 +0000 UTC" message="Address and port populated" reason=Scheduled test=TestSuperTuxKartGameServerReady type=Normal
VERBOSE: time="2025-09-18 05:53:06.604" level=info msg="Event!" lastTimestamp="2025-09-18 05:53:06 +0000 UTC" message="Issue with Gameserver pod" reason=Unhealthy test=TestSuperTuxKartGameServerReady type=Warning
VERBOSE: time="2025-09-18 05:53:06.604" level=info msg="Game Server Pod Events:" test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:53:06.707" level=info msg="Dumping Events:" kind= test=TestSuperTuxKartGameServerReady
VERBOSE: time="2025-09-18 05:53:06.815" level=info msg="Event!" lastTimestamp="2025-09-18 05:52:58 +0000 UTC" message="Successfully assigned 1758174777/supertuxkart-qwhww to gk3-gke-autopilot-e2e-te-nap-1tui7d3k-5c115de6-llk8" reason=Scheduled test=TestSuperTuxKartGameServerReady type=Normal
VERBOSE: time="2025-09-18 05:53:06.815" level=info msg="Event!" lastTimestamp="2025-09-18 05:52:59 +0000 UTC" message="Pulling image \"us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-371c28d\"" reason=Pulling test=TestSuperTuxKartGameServerReady type=Normal
VERBOSE: time="2025-09-18 05:53:06.815" level=info msg="Event!" lastTimestamp="2025-09-18 05:52:59 +0000 UTC" message="Successfully pulled image \"us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-371c28d\" in 369ms (369ms including waiting). Image size: 33759011 bytes." reason=Pulled test=TestSuperTuxKartGameServerReady type=Normal
VERBOSE: time="2025-09-18 05:53:06.815" level=info msg="Event!" lastTimestamp="2025-09-18 05:52:59 +0000 UTC" message="Created container: agones-gameserver-sidecar" reason=Created test=TestSuperTuxKartGameServerReady type=Normal
VERBOSE: time="2025-09-18 05:53:06.815" level=info msg="Event!" lastTimestamp="2025-09-18 05:52:59 +0000 UTC" message="Started container agones-gameserver-sidecar" reason=Started test=TestSuperTuxKartGameServerReady type=Normal
VERBOSE: time="2025-09-18 05:53:06.816" level=info msg="Event!" lastTimestamp="2025-09-18 05:53:00 +0000 UTC" message="Pulling image \"us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19\"" reason=Pulling test=TestSuperTuxKartGameServerReady type=Normal
VERBOSE: time="2025-09-18 05:53:06.816" level=info msg="Event!" lastTimestamp="2025-09-18 05:53:01 +0000 UTC" message="Successfully pulled image \"us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19\" in 1.117s (1.117s including waiting). Image size: 827257650 bytes." reason=Pulled test=TestSuperTuxKartGameServerReady type=Normal
VERBOSE: time="2025-09-18 05:53:06.816" level=info msg="Event!" lastTimestamp="2025-09-18 05:53:01 +0000 UTC" message="Created container: supertuxkart" reason=Created test=TestSuperTuxKartGameServerReady type=Normal
VERBOSE: time="2025-09-18 05:53:06.816" level=info msg="Event!" lastTimestamp="2025-09-18 05:53:01 +0000 UTC" message="Started container supertuxkart" reason=Started test=TestSuperTuxKartGameServerReady type=Normal
VERBOSE: time="2025-09-18 05:53:06.816" level=info msg="Event!" lastTimestamp="2025-09-18 05:53:03 +0000 UTC" message="Stopping container agones-gameserver-sidecar" reason=Killing test=TestSuperTuxKartGameServerReady type=Normal
VERBOSE:     examples_test.go:89: 
VERBOSE:         	Error Trace:	/go/src/agones.dev/agones/test/e2e/examples_test.go:89
VERBOSE:         	Error:      	Received unexpected error:
VERBOSE:         	            	waiting for {supertuxkart [{default  Dynamic <nil> 8080 0 UDP}] {false 60 0 30}  { 0 0} {{      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []} {[] [] [{supertuxkart us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19 [] []  [] [] [{ENABLE_PLAYER_TRACKING false nil}] {map[] map[cpu:{{200 -3} {<nil>} 200m DecimalSI} memory:{{734003200 0} {<nil>} 700Mi BinarySI}] []} [] <nil> [] [] nil nil nil nil    nil false false false}] []  <nil> <nil>  map[]   <nil>  false false false <nil> nil []   nil  [] []  <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil <nil> [] [] nil}} <nil> map[] map[] <nil>} GameServer instance readiness timed out (): waiting for GameServer 1758174777/supertuxkart-qwhww to be Ready: GameServer reached terminal state Unhealthy
VERBOSE:         	Test:       	TestSuperTuxKartGameServerReady
VERBOSE: --- FAIL: TestSuperTuxKartGameServerReady (8.55s)
VERBOSE: FAIL

agones-bot · 2025-09-18T22:15:35Z

Build Failed 😭

Build Id: 2b380234-e02d-4379-ab2f-2229ca61010a

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

agones-bot · 2025-09-19T18:06:51Z

Build Succeeded 🥳

Build Id: 18a2ebaa-47f8-4882-ac83-0a8dc07d6ea1

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.53.0-dev-8c16697
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.53.0-dev-8c16697
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-8c16697-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.53.0-dev-8c16697
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.53.0-dev-8c16697
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.53.0-dev-8c16697
Linux C++ SDK (build): agonessdk-1.53.0-dev-8c16697-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.53.0-dev-8c16697.zip

A preview of the website (the last 30 builds are retained):

https://8c16697-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4280/head:pr_4280 && git checkout pr_4280
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.53.0-dev-8c16697

agones-bot · 2025-09-19T19:10:08Z

Build Succeeded 🥳

Build Id: 7152b0a6-ef9a-42e5-8da1-0924ffee172c

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.53.0-dev-96be4bf
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.53.0-dev-96be4bf
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-96be4bf-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.53.0-dev-96be4bf
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.53.0-dev-96be4bf
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.53.0-dev-96be4bf
Linux C++ SDK (build): agonessdk-1.53.0-dev-96be4bf-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.53.0-dev-96be4bf.zip

A preview of the website (the last 30 builds are retained):

https://96be4bf-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4280/head:pr_4280 && git checkout pr_4280
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.53.0-dev-96be4bf

markmandel · 2025-09-19T19:48:27Z

Of course everything passes now 😁

@igooch - any chance there was something changed in Autopilot? This has been less flaky today.

markmandel · 2025-09-19T19:48:32Z

/gcbrun

agones-bot · 2025-09-19T20:46:39Z

Build Succeeded 🥳

Build Id: a5bfefe6-22c7-4a82-a3b1-caaa12c2a138

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.53.0-dev-96be4bf
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.53.0-dev-96be4bf
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-96be4bf-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.53.0-dev-96be4bf
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.53.0-dev-96be4bf
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.53.0-dev-96be4bf
Linux C++ SDK (build): agonessdk-1.53.0-dev-96be4bf-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.53.0-dev-96be4bf.zip

A preview of the website (the last 30 builds are retained):

https://96be4bf-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4280/head:pr_4280 && git checkout pr_4280
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.53.0-dev-96be4bf

markmandel · 2025-09-19T21:31:18Z

Ready for review? I guess? At least if it fails again, we should have more info.

test/e2e/framework/framework.go

This test has been flaky on GKE Autopilot. Adding explicit memory (700Mi) and CPU (200m) resource requests to the container to ensure adequate resource allocation and improve test stability on GKE Autopilot clusters.

* Tweaking some resource levels, to see if it makes a difference.

agones-bot · 2025-09-20T06:36:31Z

Build Failed 😭

Build Id: 37e81e0e-2bb7-44ed-8cd0-f6e04d8d5479

Status: FAILURE

To get permission to view the Cloud Build view, join the agones-discuss Google Group.

markmandel · 2025-09-20T16:04:09Z

Source: https://console.cloud.google.com/cloud-build/builds/2b315b92-a54b-4bcd-ae98-48d7c50d1f4b;step=1?project=agones-images

So this is interesting, this line fails first:

=== CONT  TestSuperTuxKartGameServerReady
    examples_test.go:93: 
        	Error Trace:	/go/src/agones.dev/agones/test/e2e/examples_test.go:93
        	Error:      	Received unexpected error:
        	            	waiting for {supertuxkart [{default  Dynamic <nil> 8080 0 UDP}] {false 60 0 30}  { 0 0} {{      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []} {[] [] [{supertuxkart us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19 [] []  [] [] [{ENABLE_PLAYER_TRACKING false nil}] {map[] map[cpu:{{500 -3} {<nil>} 500m DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}] []} [] <nil> [] [] nil nil nil nil    nil false false false}] []  <nil> <nil>  map[]   <nil>  false false false <nil> nil []   nil  [] []  <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil <nil> [] [] nil}} <nil> map[] map[] <nil>} GameServer instance readiness timed out (): waiting for GameServer 1758349574/supertuxkart-glxfb to be Ready: GameServer reached terminal state Unhealthy
        	Test:       	TestSuperTuxKartGameServerReady
--- FAIL: TestSuperTuxKartGameServerReady (34.26s)
time="2025-09-20 06:30:49.280" level=info msg="Waiting for states to match" awaitingState=Ready currentState=RequestReady gs=game-serverx97dl test=TestPlayerConnectWithCapacityZero
time="2025-09-20 06:30:49.492" level=info msg="GameServer created, waiting for Ready" gs=game-servercxptr test=TestGameServerStaticUdpProtocol
time="2025-09-20 06:30:49.661" level=info msg="GameServer states match" awaitingState=Ready currentState=Ready gs=game-servergrxjp test=TestSideCarCommunicatesWhileTerminating
time="2025-09-20 06:30:49.661" level=info msg="GameServer Ready" gs=game-servergrxjp
time="2025-09-20 06:30:49.900" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=game-server59s2h test=TestGameServerSetPlayerCapacity/no_initial_capacity_set
time="2025-09-20 06:30:50.262" level=info msg="GameServer states match" awaitingState=Ready currentState=Ready gs=game-serverq8f2c test=TestGameServerReserve
time="2025-09-20 06:30:50.483" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=game-serverz554z test=TestPlayerConnectAndDisconnect
FAIL test/e2e.TestSuperTuxKartGameServerReady (34.26s)

Then consistent failure here:

=== RUN   TestSuperTuxKartGameServerReady
=== PAUSE TestSuperTuxKartGameServerReady
=== CONT  TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:50.893" level=info msg="GameServer created, waiting for Ready" gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:50.931" level=info msg="Waiting for states to match" awaitingState=Ready currentState=PortAllocation gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:51.909" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:52.905" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:53.909" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:54.908" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:55.907" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:56.906" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:57.905" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:58.917" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:59.907" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:00.911" level=error msg="GameServer reached terminal state" awaitingState=Ready currentState=Unhealthy gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:00.912" level=info msg="Game Server Events:" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:00.912" level=info msg="Dumping Events:" kind= test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:00.960" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:50 +0000 UTC" message="Port allocated" reason=PortAllocation test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:00.960" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:50 +0000 UTC" message="Pod supertuxkart-86qvs created" reason=Creating test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:00.960" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:53 +0000 UTC" message="Address and port populated" reason=Scheduled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:00.960" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:00 +0000 UTC" message="Issue with Gameserver pod" reason=Unhealthy test=TestSuperTuxKartGameServerReady type=Warning
time="2025-09-20 06:36:00.961" level=info msg="Game Server Pod Events:" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.001" level=info msg="Dumping Events:" kind= test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.051" level=info msg="Event!" lastTimestamp="0001-01-01 00:00:00 +0000 UTC" message="Successfully assigned 1758350150/supertuxkart-86qvs to gke-standard-e2e-test-cluster-default-fa26fae1-gxcn" reason=Scheduled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.052" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:51 +0000 UTC" message="Pulling image \"us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-658225f\"" reason=Pulling test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.052" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:53 +0000 UTC" message="Successfully pulled image \"us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-658225f\" in 1.395s (1.395s including waiting). Image size: 33751483 bytes." reason=Pulled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.052" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:53 +0000 UTC" message="Created container: agones-gameserver-sidecar" reason=Created test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.053" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:53 +0000 UTC" message="Started container agones-gameserver-sidecar" reason=Started test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.053" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:53 +0000 UTC" message="Pulling image \"us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19\"" reason=Pulling test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.053" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:54 +0000 UTC" message="Successfully pulled image \"us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19\" in 1.118s (1.118s including waiting). Image size: 827257650 bytes." reason=Pulled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.053" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:54 +0000 UTC" message="Created container: supertuxkart" reason=Created test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.054" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:54 +0000 UTC" message="Started container supertuxkart" reason=Started test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.054" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:57 +0000 UTC" message="Stopping container agones-gameserver-sidecar" reason=Killing test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.054" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:57 +0000 UTC" message="Liveness probe failed: Get \"http://10.32.24.4:8080/healthz\": dial tcp 10.32.24.4:8080: connect: connection refused" reason=Unhealthy test=TestSuperTuxKartGameServerReady type=Warning
time="2025-09-20 06:36:01.054" level=info msg="Game Server Container Logs:" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.054" level=info msg="Logs for Pod:" namespace=1758350150 pod=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.133" level=info msg="Logs for container" container=supertuxkart previous=false test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.133" level=info msg="[wrapper] 2025/09/20 06:35:54 Connecting to Agones with the SDK\n[wrapper] 2025/09/20 06:35:54 Starting health checking\n[wrapper] 2025/09/20 06:35:54 Starting wrapper for SuperTuxKart\n[wrapper] 2025/09/20 06:35:54 Command being run for SuperTuxKart server: /usr/games/supertuxkart --server-config=/home/supertuxkart/server_config.xml \n2025/09/20 06:35:55 Waiting for /home/supertuxkart/.config/supertuxkart/config-0.10/server_config.log to appear...\n[wrapper] 2025/09/20 06:35:55 tail ended\n" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.133" level=info msg="---End of container logs---" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.171" level=warning msg="Error opening log stream for container" container=supertuxkart error="previous terminated container \"supertuxkart\" in pod \"supertuxkart-86qvs\" not found" previous=true test=TestSuperTuxKartGameServerReady
    examples_test.go:93: 
        	Error Trace:	/go/src/agones.dev/agones/test/e2e/examples_test.go:93
        	Error:      	Received unexpected error:
        	            	waiting for {supertuxkart [{default  Dynamic <nil> 8080 0 UDP}] {false 60 0 30}  { 0 0} {{      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []} {[] [] [{supertuxkart us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19 [] []  [] [] [{ENABLE_PLAYER_TRACKING false nil}] {map[] map[cpu:{{500 -3} {<nil>} 500m DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}] []} [] <nil> [] [] nil nil nil nil    nil false false false}] []  <nil> <nil>  map[]   <nil>  false false false <nil> nil []   nil  [] []  <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil <nil> [] [] nil}} <nil> map[] map[] <nil>} GameServer instance readiness timed out (): waiting for GameServer 1758350150/supertuxkart-86qvs to be Ready: GameServer reached terminal state Unhealthy
        	Test:       	TestSuperTuxKartGameServerReady
--- FAIL: TestSuperTuxKartGameServerReady (10.36s)
FAIL test/e2e.TestSuperTuxKartGameServerReady (re-run 1) (10.36s)
time="2025-09-20 06:36:01.266" level=info msg="Namespace 1758350150 is deleted"
FAIL test/e2e

DONE 2 runs, 139 tests, 24 skipped, 2 failures in 586.882s

time="2025-09-20 06:36:06.539" level=warning msg="Error creating inClusterConfig, trying to build config from flagsunable to load in-cluster configuration, KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT must be defined" error="unable to load in-cluster configuration, KUBERNETES_SERVICE_HOST and KUBERNETES_SERVICE_PORT must be defined" source=framework
time="2025-09-20 06:36:06.553" level=info msg="Starting e2e test(s)" cloudProduct=generic featureGates="AutopilotPassthroughPort=false&CountsAndLists=false&DisableResyncOnSDKServer=true&Example=true&FleetAutoscaleRequestMetaData=true&GKEAutopilotExtendedDurationPods=false&PlayerAllocationFilter=true&PlayerTracking=true&PortPolicyNone=false&PortRanges=false&ProcessorAllocator=false&RollingUpdateFix=false&ScheduledAutoscaler=false&SidecarContainers=true&WasmAutoscaler=false" gameServerImage="us-docker.pkg.dev/agones-images/examples/simple-game-server:0.39" namespace= perfOutputDir= pullSecret= stressTestLevel=0 version=
time="2025-09-20 06:36:06.673" level=info msg="Namespace 1758349574 is deleted"
time="2025-09-20 06:36:06.780" level=info msg="Namespace 1758350150 is deleted"
time="2025-09-20 06:36:06.780" level=info msg="Custom namespace is set: 1758350166"
time="2025-09-20 06:36:06.833" level=info msg="Namespace 1758350166 is created"
time="2025-09-20 06:36:06.882" level=info msg="ServiceAccount 1758350166/agones-sdk is created"
time="2025-09-20 06:36:06.971" level=info msg="RoleBinding 1758350166/agones-sdk-access is created"
=== RUN   TestSuperTuxKartGameServerReady
=== PAUSE TestSuperTuxKartGameServerReady
=== CONT  TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:07.066" level=info msg="GameServer created, waiting for Ready" gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:07.101" level=info msg="Waiting for states to match" awaitingState=Ready currentState=PortAllocation gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:08.079" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:09.079" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:10.084" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:11.080" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:12.083" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:13.082" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:14.080" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:15.092" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:16.080" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.080" level=error msg="GameServer reached terminal state" awaitingState=Ready currentState=Unhealthy gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.081" level=info msg="Game Server Events:" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.081" level=info msg="Dumping Events:" kind= test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.122" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:07 +0000 UTC" message="Port allocated" reason=PortAllocation test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.122" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:07 +0000 UTC" message="Pod supertuxkart-xrnf6 created" reason=Creating test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.122" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:09 +0000 UTC" message="Address and port populated" reason=Scheduled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.123" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:16 +0000 UTC" message="Issue with Gameserver pod" reason=Unhealthy test=TestSuperTuxKartGameServerReady type=Warning
time="2025-09-20 06:36:17.123" level=info msg="Game Server Pod Events:" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.163" level=info msg="Dumping Events:" kind= test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.214" level=info msg="Event!" lastTimestamp="0001-01-01 00:00:00 +0000 UTC" message="Successfully assigned 1758350166/supertuxkart-xrnf6 to gke-standard-e2e-test-cluster-default-fa26fae1-64rn" reason=Scheduled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.214" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:07 +0000 UTC" message="Pulling image \"us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-658225f\"" reason=Pulling test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.214" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:09 +0000 UTC" message="Successfully pulled image \"us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-658225f\" in 1.492s (1.492s including waiting). Image size: 33751483 bytes." reason=Pulled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.214" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:09 +0000 UTC" message="Created container: agones-gameserver-sidecar" reason=Created test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.215" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:09 +0000 UTC" message="Started container agones-gameserver-sidecar" reason=Started test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.215" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:09 +0000 UTC" message="Pulling image \"us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19\"" reason=Pulling test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.215" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:10 +0000 UTC" message="Successfully pulled image \"us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19\" in 1.145s (1.145s including waiting). Image size: 827257650 bytes." reason=Pulled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.215" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:10 +0000 UTC" message="Created container: supertuxkart" reason=Created test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.216" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:10 +0000 UTC" message="Started container supertuxkart" reason=Started test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.216" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:13 +0000 UTC" message="Stopping container agones-gameserver-sidecar" reason=Killing test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.216" level=info msg="Game Server Container Logs:" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.216" level=info msg="Logs for Pod:" namespace=1758350166 pod=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.287" level=info msg="Logs for container" container=supertuxkart previous=false test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.287" level=info msg="[wrapper] 2025/09/20 06:36:11 Connecting to Agones with the SDK\n[wrapper] 2025/09/20 06:36:11 Starting health checking\n[wrapper] 2025/09/20 06:36:11 Starting wrapper for SuperTuxKart\n[wrapper] 2025/09/20 06:36:11 Command being run for SuperTuxKart server: /usr/games/supertuxkart --server-config=/home/supertuxkart/server_config.xml \n2025/09/20 06:36:12 Waiting for /home/supertuxkart/.config/supertuxkart/config-0.10/server_config.log to appear...\n[wrapper] 2025/09/20 06:36:12 tail ended\n" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.287" level=info msg="---End of container logs---" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.311" level=warning msg="Error opening log stream for container" container=supertuxkart error="previous terminated container \"supertuxkart\" in pod \"supertuxkart-xrnf6\" not found" previous=true test=TestSuperTuxKartGameServerReady
    examples_test.go:93: 
        	Error Trace:	/go/src/agones.dev/agones/test/e2e/examples_test.go:93
        	Error:      	Received unexpected error:
        	            	waiting for {supertuxkart [{default  Dynamic <nil> 8080 0 UDP}] {false 60 0 30}  { 0 0} {{      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []} {[] [] [{supertuxkart us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19 [] []  [] [] [{ENABLE_PLAYER_TRACKING false nil}] {map[] map[cpu:{{500 -3} {<nil>} 500m DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}] []} [] <nil> [] [] nil nil nil nil    nil false false false}] []  <nil> <nil>  map[]   <nil>  false false false <nil> nil []   nil  [] []  <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil <nil> [] [] nil}} <nil> map[] map[] <nil>} GameServer instance readiness timed out (): waiting for GameServer 1758350166/supertuxkart-xrnf6 to be Ready: GameServer reached terminal state Unhealthy
        	Test:       	TestSuperTuxKartGameServerReady
--- FAIL: TestSuperTuxKartGameServerReady (10.34s)
FAIL test/e2e.TestSuperTuxKartGameServerReady (re-run 2) (10.34s)
time="2025-09-20 06:36:17.401" level=info msg="Namespace 1758350166 is deleted"
FAIL test/e2e

=== Failed
=== FAIL: test/e2e TestSuperTuxKartGameServerReady (34.26s)
    examples_test.go:93: 
        	Error Trace:	/go/src/agones.dev/agones/test/e2e/examples_test.go:93
        	Error:      	Received unexpected error:
        	            	waiting for {supertuxkart [{default  Dynamic <nil> 8080 0 UDP}] {false 60 0 30}  { 0 0} {{      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []} {[] [] [{supertuxkart us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19 [] []  [] [] [{ENABLE_PLAYER_TRACKING false nil}] {map[] map[cpu:{{500 -3} {<nil>} 500m DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}] []} [] <nil> [] [] nil nil nil nil    nil false false false}] []  <nil> <nil>  map[]   <nil>  false false false <nil> nil []   nil  [] []  <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil <nil> [] [] nil}} <nil> map[] map[] <nil>} GameServer instance readiness timed out (): waiting for GameServer 1758349574/supertuxkart-glxfb to be Ready: GameServer reached terminal state Unhealthy
        	Test:       	TestSuperTuxKartGameServerReady
time="2025-09-20 06:30:49.280" level=info msg="Waiting for states to match" awaitingState=Ready currentState=RequestReady gs=game-serverx97dl test=TestPlayerConnectWithCapacityZero
time="2025-09-20 06:30:49.492" level=info msg="GameServer created, waiting for Ready" gs=game-servercxptr test=TestGameServerStaticUdpProtocol
time="2025-09-20 06:30:49.661" level=info msg="GameServer states match" awaitingState=Ready currentState=Ready gs=game-servergrxjp test=TestSideCarCommunicatesWhileTerminating
time="2025-09-20 06:30:49.661" level=info msg="GameServer Ready" gs=game-servergrxjp
time="2025-09-20 06:30:49.900" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=game-server59s2h test=TestGameServerSetPlayerCapacity/no_initial_capacity_set
time="2025-09-20 06:30:50.262" level=info msg="GameServer states match" awaitingState=Ready currentState=Ready gs=game-serverq8f2c test=TestGameServerReserve
time="2025-09-20 06:30:50.483" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=game-serverz554z test=TestPlayerConnectAndDisconnect

=== FAIL: test/e2e TestSuperTuxKartGameServerReady (re-run 1) (10.36s)
time="2025-09-20 06:35:50.893" level=info msg="GameServer created, waiting for Ready" gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:50.931" level=info msg="Waiting for states to match" awaitingState=Ready currentState=PortAllocation gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:51.909" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:52.905" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:53.909" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:54.908" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:55.907" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:56.906" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:57.905" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:58.917" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:35:59.907" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:00.911" level=error msg="GameServer reached terminal state" awaitingState=Ready currentState=Unhealthy gs=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:00.912" level=info msg="Game Server Events:" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:00.912" level=info msg="Dumping Events:" kind= test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:00.960" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:50 +0000 UTC" message="Port allocated" reason=PortAllocation test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:00.960" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:50 +0000 UTC" message="Pod supertuxkart-86qvs created" reason=Creating test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:00.960" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:53 +0000 UTC" message="Address and port populated" reason=Scheduled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:00.960" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:00 +0000 UTC" message="Issue with Gameserver pod" reason=Unhealthy test=TestSuperTuxKartGameServerReady type=Warning
time="2025-09-20 06:36:00.961" level=info msg="Game Server Pod Events:" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.001" level=info msg="Dumping Events:" kind= test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.051" level=info msg="Event!" lastTimestamp="0001-01-01 00:00:00 +0000 UTC" message="Successfully assigned 1758350150/supertuxkart-86qvs to gke-standard-e2e-test-cluster-default-fa26fae1-gxcn" reason=Scheduled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.052" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:51 +0000 UTC" message="Pulling image \"us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-658225f\"" reason=Pulling test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.052" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:53 +0000 UTC" message="Successfully pulled image \"us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-658225f\" in 1.395s (1.395s including waiting). Image size: 33751483 bytes." reason=Pulled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.052" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:53 +0000 UTC" message="Created container: agones-gameserver-sidecar" reason=Created test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.053" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:53 +0000 UTC" message="Started container agones-gameserver-sidecar" reason=Started test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.053" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:53 +0000 UTC" message="Pulling image \"us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19\"" reason=Pulling test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.053" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:54 +0000 UTC" message="Successfully pulled image \"us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19\" in 1.118s (1.118s including waiting). Image size: 827257650 bytes." reason=Pulled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.053" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:54 +0000 UTC" message="Created container: supertuxkart" reason=Created test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.054" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:54 +0000 UTC" message="Started container supertuxkart" reason=Started test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.054" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:57 +0000 UTC" message="Stopping container agones-gameserver-sidecar" reason=Killing test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:01.054" level=info msg="Event!" lastTimestamp="2025-09-20 06:35:57 +0000 UTC" message="Liveness probe failed: Get \"http://10.32.24.4:8080/healthz\": dial tcp 10.32.24.4:8080: connect: connection refused" reason=Unhealthy test=TestSuperTuxKartGameServerReady type=Warning
time="2025-09-20 06:36:01.054" level=info msg="Game Server Container Logs:" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.054" level=info msg="Logs for Pod:" namespace=1758350150 pod=supertuxkart-86qvs test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.133" level=info msg="Logs for container" container=supertuxkart previous=false test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.133" level=info msg="[wrapper] 2025/09/20 06:35:54 Connecting to Agones with the SDK\n[wrapper] 2025/09/20 06:35:54 Starting health checking\n[wrapper] 2025/09/20 06:35:54 Starting wrapper for SuperTuxKart\n[wrapper] 2025/09/20 06:35:54 Command being run for SuperTuxKart server: /usr/games/supertuxkart --server-config=/home/supertuxkart/server_config.xml \n2025/09/20 06:35:55 Waiting for /home/supertuxkart/.config/supertuxkart/config-0.10/server_config.log to appear...\n[wrapper] 2025/09/20 06:35:55 tail ended\n" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.133" level=info msg="---End of container logs---" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:01.171" level=warning msg="Error opening log stream for container" container=supertuxkart error="previous terminated container \"supertuxkart\" in pod \"supertuxkart-86qvs\" not found" previous=true test=TestSuperTuxKartGameServerReady
    examples_test.go:93: 
        	Error Trace:	/go/src/agones.dev/agones/test/e2e/examples_test.go:93
        	Error:      	Received unexpected error:
        	            	waiting for {supertuxkart [{default  Dynamic <nil> 8080 0 UDP}] {false 60 0 30}  { 0 0} {{      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []} {[] [] [{supertuxkart us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19 [] []  [] [] [{ENABLE_PLAYER_TRACKING false nil}] {map[] map[cpu:{{500 -3} {<nil>} 500m DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}] []} [] <nil> [] [] nil nil nil nil    nil false false false}] []  <nil> <nil>  map[]   <nil>  false false false <nil> nil []   nil  [] []  <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil <nil> [] [] nil}} <nil> map[] map[] <nil>} GameServer instance readiness timed out (): waiting for GameServer 1758350150/supertuxkart-86qvs to be Ready: GameServer reached terminal state Unhealthy
        	Test:       	TestSuperTuxKartGameServerReady

=== FAIL: test/e2e TestSuperTuxKartGameServerReady (re-run 2) (10.34s)
time="2025-09-20 06:36:07.066" level=info msg="GameServer created, waiting for Ready" gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:07.101" level=info msg="Waiting for states to match" awaitingState=Ready currentState=PortAllocation gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:08.079" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:09.079" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Starting gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:10.084" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:11.080" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:12.083" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:13.082" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:14.080" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:15.092" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:16.080" level=info msg="Waiting for states to match" awaitingState=Ready currentState=Scheduled gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.080" level=error msg="GameServer reached terminal state" awaitingState=Ready currentState=Unhealthy gs=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.081" level=info msg="Game Server Events:" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.081" level=info msg="Dumping Events:" kind= test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.122" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:07 +0000 UTC" message="Port allocated" reason=PortAllocation test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.122" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:07 +0000 UTC" message="Pod supertuxkart-xrnf6 created" reason=Creating test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.122" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:09 +0000 UTC" message="Address and port populated" reason=Scheduled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.123" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:16 +0000 UTC" message="Issue with Gameserver pod" reason=Unhealthy test=TestSuperTuxKartGameServerReady type=Warning
time="2025-09-20 06:36:17.123" level=info msg="Game Server Pod Events:" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.163" level=info msg="Dumping Events:" kind= test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.214" level=info msg="Event!" lastTimestamp="0001-01-01 00:00:00 +0000 UTC" message="Successfully assigned 1758350166/supertuxkart-xrnf6 to gke-standard-e2e-test-cluster-default-fa26fae1-64rn" reason=Scheduled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.214" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:07 +0000 UTC" message="Pulling image \"us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-658225f\"" reason=Pulling test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.214" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:09 +0000 UTC" message="Successfully pulled image \"us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-658225f\" in 1.492s (1.492s including waiting). Image size: 33751483 bytes." reason=Pulled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.214" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:09 +0000 UTC" message="Created container: agones-gameserver-sidecar" reason=Created test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.215" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:09 +0000 UTC" message="Started container agones-gameserver-sidecar" reason=Started test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.215" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:09 +0000 UTC" message="Pulling image \"us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19\"" reason=Pulling test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.215" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:10 +0000 UTC" message="Successfully pulled image \"us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19\" in 1.145s (1.145s including waiting). Image size: 827257650 bytes." reason=Pulled test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.215" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:10 +0000 UTC" message="Created container: supertuxkart" reason=Created test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.216" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:10 +0000 UTC" message="Started container supertuxkart" reason=Started test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.216" level=info msg="Event!" lastTimestamp="2025-09-20 06:36:13 +0000 UTC" message="Stopping container agones-gameserver-sidecar" reason=Killing test=TestSuperTuxKartGameServerReady type=Normal
time="2025-09-20 06:36:17.216" level=info msg="Game Server Container Logs:" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.216" level=info msg="Logs for Pod:" namespace=1758350166 pod=supertuxkart-xrnf6 test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.287" level=info msg="Logs for container" container=supertuxkart previous=false test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.287" level=info msg="[wrapper] 2025/09/20 06:36:11 Connecting to Agones with the SDK\n[wrapper] 2025/09/20 06:36:11 Starting health checking\n[wrapper] 2025/09/20 06:36:11 Starting wrapper for SuperTuxKart\n[wrapper] 2025/09/20 06:36:11 Command being run for SuperTuxKart server: /usr/games/supertuxkart --server-config=/home/supertuxkart/server_config.xml \n2025/09/20 06:36:12 Waiting for /home/supertuxkart/.config/supertuxkart/config-0.10/server_config.log to appear...\n[wrapper] 2025/09/20 06:36:12 tail ended\n" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.287" level=info msg="---End of container logs---" test=TestSuperTuxKartGameServerReady
time="2025-09-20 06:36:17.311" level=warning msg="Error opening log stream for container" container=supertuxkart error="previous terminated container \"supertuxkart\" in pod \"supertuxkart-xrnf6\" not found" previous=true test=TestSuperTuxKartGameServerReady
    examples_test.go:93: 
        	Error Trace:	/go/src/agones.dev/agones/test/e2e/examples_test.go:93
        	Error:      	Received unexpected error:
        	            	waiting for {supertuxkart [{default  Dynamic <nil> 8080 0 UDP}] {false 60 0 30}  { 0 0} {{      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []} {[] [] [{supertuxkart us-docker.pkg.dev/agones-images/examples/supertuxkart-example:0.19 [] []  [] [] [{ENABLE_PLAYER_TRACKING false nil}] {map[] map[cpu:{{500 -3} {<nil>} 500m DecimalSI} memory:{{1073741824 0} {<nil>} 1Gi BinarySI}] []} [] <nil> [] [] nil nil nil nil    nil false false false}] []  <nil> <nil>  map[]   <nil>  false false false <nil> nil []   nil  [] []  <nil> nil [] <nil> <nil> <nil> map[] [] <nil> nil <nil> [] [] nil}} <nil> map[] map[] <nil>} GameServer instance readiness timed out (): waiting for GameServer 1758350166/supertuxkart-xrnf6 to be Ready: GameServer reached terminal state Unhealthy
        	Test:       	TestSuperTuxKartGameServerReady

Interestingly - this is a standard GKE cluster.

* Updated log output to be easier to read.

agones-bot · 2025-09-20T17:58:27Z

Build Succeeded 🥳

Build Id: d6375bd6-0969-454f-9f4c-84f1cf0c2b43

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.53.0-dev-f35fc1a
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.53.0-dev-f35fc1a
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-f35fc1a-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.53.0-dev-f35fc1a
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.53.0-dev-f35fc1a
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.53.0-dev-f35fc1a
Linux C++ SDK (build): agonessdk-1.53.0-dev-f35fc1a-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.53.0-dev-f35fc1a.zip

A preview of the website (the last 30 builds are retained):

https://f35fc1a-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4280/head:pr_4280 && git checkout pr_4280
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.53.0-dev-f35fc1a

markmandel · 2025-09-20T18:06:14Z

/gcbrun

agones-bot · 2025-09-20T19:08:41Z

Build Succeeded 🥳

Build Id: 2eb5509a-8098-4686-afdb-1842299d96c6

The following development artifacts have been built, and will exist for the next 30 days:

image: us-docker.pkg.dev/agones-images/ci/agones-controller:1.53.0-dev-f35fc1a
image: us-docker.pkg.dev/agones-images/ci/agones-extensions:1.53.0-dev-f35fc1a
image: us-docker.pkg.dev/agones-images/ci/agones-sdk:1.53.0-dev-f35fc1a-linux
image: us-docker.pkg.dev/agones-images/ci/agones-ping:1.53.0-dev-f35fc1a
image: us-docker.pkg.dev/agones-images/ci/agones-allocator:1.53.0-dev-f35fc1a
image: us-docker.pkg.dev/agones-images/ci/agones-processor:1.53.0-dev-f35fc1a
Linux C++ SDK (build): agonessdk-1.53.0-dev-f35fc1a-linux-arch_64.tar.gz
SDK Server: agonessdk-server-1.53.0-dev-f35fc1a.zip

A preview of the website (the last 30 builds are retained):

https://f35fc1a-dot-preview-dot-agones-images.appspot.com/

To install this version:

git fetch https://github.com/googleforgames/agones.git pull/4280/head:pr_4280 && git checkout pr_4280
helm install agones ./install/helm/agones --namespace agones-system --set agones.image.registry=us-docker.pkg.dev/agones-images/ci --set agones.image.tag=1.53.0-dev-f35fc1a

lacroixthomas

LGTM, as you were saying, if it fails again, we should have more info now ! 😄

…v#4280) * flakey: add resource requirements to SuperTuxKart e2e test This test has been flaky on GKE Autopilot. Adding explicit memory (700Mi) and CPU (200m) resource requests to the container to ensure adequate resource allocation and improve test stability on GKE Autopilot clusters. * Add the ability to get Pod logs to the e2e framework. * Also bump the memory a little? * Tweaks to logging functions. * * Review updates * Tweaking some resource levels, to see if it makes a difference. * * Tweaks to test parameters * Updated log output to be easier to read.

github-actions bot added kind/bug These are bugs. size/XS labels Sep 18, 2025

markmandel requested review from igooch and lacroixthomas September 18, 2025 04:33

markmandel added the area/tests Unit tests, e2e tests, anything to make sure things don't break label Sep 18, 2025

github-actions bot added the size/S label Sep 18, 2025

lacroixthomas reviewed Sep 20, 2025

View reviewed changes

test/e2e/framework/framework.go Outdated Show resolved Hide resolved

markmandel added 5 commits September 19, 2025 23:06

flakey: add resource requirements to SuperTuxKart e2e test

b82794e

This test has been flaky on GKE Autopilot. Adding explicit memory (700Mi) and CPU (200m) resource requests to the container to ensure adequate resource allocation and improve test stability on GKE Autopilot clusters.

Add the ability to get Pod logs to the e2e framework.

8da01e1

Also bump the memory a little?

b223d93

Tweaks to logging functions.

e7a3960

* Review updates

658225f

* Tweaking some resource levels, to see if it makes a difference.

markmandel force-pushed the flaky/TestSuperTuxKartGameServerReady branch from 96be4bf to 658225f Compare September 20, 2025 06:06

* Tweaks to test parameters

f35fc1a

* Updated log output to be easier to read.

lacroixthomas approved these changes Sep 23, 2025

View reviewed changes

markmandel merged commit a1f321c into agones-dev:main Sep 23, 2025
4 checks passed

markmandel deleted the flaky/TestSuperTuxKartGameServerReady branch September 23, 2025 22:00

Conversation

markmandel commented Sep 18, 2025

Uh oh!

agones-bot commented Sep 18, 2025

Uh oh!

markmandel commented Sep 18, 2025

Uh oh!

agones-bot commented Sep 18, 2025

Uh oh!

agones-bot commented Sep 19, 2025

Uh oh!

agones-bot commented Sep 19, 2025

Uh oh!

markmandel commented Sep 19, 2025

Uh oh!

markmandel commented Sep 19, 2025

Uh oh!

agones-bot commented Sep 19, 2025

Uh oh!

markmandel commented Sep 19, 2025

Uh oh!

Uh oh!

agones-bot commented Sep 20, 2025

Uh oh!

markmandel commented Sep 20, 2025

Uh oh!

agones-bot commented Sep 20, 2025

Uh oh!

markmandel commented Sep 20, 2025

Uh oh!

agones-bot commented Sep 20, 2025

Uh oh!

lacroixthomas left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants