Summary

Currently the deploying order does matter, the agents need to be available first, and only then the inference gateway could be deployed so the connection is established.

In a cloud native environment the approach is normally eventually consistency. That said, it means there must be a mechanism to auto retry to establish a connection for a configured amount of time, default to 3 times perhaps, to the agents.

AC's

• Connection to each a2a agent is retried once failed at least 3 times
• When connection is lost, it should retry to connect again at least x times (configured)
• Deployments order of the components doesn't matter - inference gateway could be deployed first, then the agents and vice versa