What happened:

Allocated game servers are not efficiently packed, even though they have plenty of Counter space:

As you can see, when making allocation requests, Agones starts to fill up the first server, but then, before that server is anywhere close to its 300 Counter capacity, it starts sending players to the Ready buffer server. Now there are multiple game servers active, but even though those servers also have plenty of space, Agones will still occasionally allocate more Ready buffer servers.

This also occurred when scaling agones-allocator replicas down to 1 (from 3), and actually, the packing was worse in this scenario, even with an allocation rate as low as 5/s.

What you expected to happen:
Agones should always pick an Allocated game server with available Counter capacity before choosing a Ready buffer server.

How to reproduce it (as minimally and precisely as possible):
Game server settings like the following:

Kind: FleetAutoscaler
Spec:
  Policy:
    Type: Buffer
    Buffer:
      Buffer Size: 1

kind: Fleet
spec:
  scheduling: Packed
  template:
    spec:
      counters:
        players:
          capacity: 300

where a game server runs for a period of time and players continually cycle in and out (using SDK's DecrementCounterAsync(string key, long amount) upon game end);

and an allocation request like the following

GameServerSelectors = {
    new GameServerSelector {
        GameServerState = GameServerSelector.Types.GameServerState.Allocated,
        MatchLabels = {
            { "version", "1.2.3" },
        },
        Counters = {
            {
                "players", new CounterSelector {
                    MinAvailable = 1,
                }
            },
        },
    },
    new GameServerSelector {
        GameServerState = GameServerSelector.Types.GameServerState.Ready,
        MatchLabels = {
            { "version", "1.2.3" },
        },
        Counters = {
            {
                "players", new CounterSelector {
                    MinAvailable = 1,
                }
            },
        },
    },
},
Counters = {
    {
        "players", new CounterAction {
            Action = "Increment",
            Amount = 1,
        }
    },
},

using https://github.com/googleforgames/agones/blob/release-1.41.0/proto/allocation/allocation.proto and gRPC

agones:
  agones:
    allocator:
      service:
        http:
          enabled: false
        grpc:
          enabled: true
        serviceType: ClusterIP

Anything else we need to know?:
agones-allocator game server state does not seem to be shared between replicas (which seems like it should also be fixed, maybe Redis or something?), but as mentioned, even with only one replica, the packing was as bad or worse

Environment:

• Agones version: 1.41.0
• Kubernetes version (use kubectl version): 1.28.7-gke.1026001
• Cloud provider or hardware configuration: GCP
• Install method (yaml/helm): helm
• Troubleshooting guide log(s):
• Others: