Configure HTTP client connection pool limits to prevent connection exhaustion and improve reliability. Many places create new httpx.AsyncClient instances per request, preventing connection reuse.

Current State Analysis

Already Optimized (Persistent Clients)

These services correctly create a persistent HTTP client:

Problem Areas (New Client Per Request)

These files create new clients per request, preventing connection reuse:

Example Problem Pattern

# Found in translate.py:1174, sso_service.py:460, etc.
async with httpx.AsyncClient(headers=headers, timeout=...) as client:
    response = await client.get(url)
    # Client is closed immediately after - no connection reuse!

Proposed Solution

Create a shared HTTP client factory:

# mcpgateway/utils/http_client.py
from functools import lru_cache
import httpx

@lru_cache
def get_http_client() -> httpx.AsyncClient:
    return httpx.AsyncClient(
        limits=httpx.Limits(
            max_keepalive_connections=100,
            max_connections=200,
            keepalive_expiry=30.0,
        ),
        timeout=httpx.Timeout(30.0, connect=10.0),
    )

Configuration

HTTP_CLIENT_MAX_KEEPALIVE=100
HTTP_CLIENT_MAX_CONNECTIONS=200
HTTP_CLIENT_KEEPALIVE_EXPIRY=30.0
HTTP_CLIENT_TIMEOUT=30.0
HTTP_CLIENT_CONNECT_TIMEOUT=10.0

Acceptance Criteria

• Shared HTTP client factory created
• Problem files updated to use shared client
• Connection pool limits configurable via environment
• Lifespan cleanup in main.py
• Connection pool metrics exposed
• All HTTP-related tests pass
• Passes make verify