TestE2E/ConnectionLimit  Close_connections_over_limit e2e test seems to constantly fail

Seems that this test constantly fails now:
https://github.com/envoyproxy/gateway/actions/runs/8434551780/job/23098264792?pr=3022
https://github.com/envoyproxy/gateway/actions/runs/8432274812/job/23091217953


