Summary of Changes

Hello @slin1237, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly refactors the OpenAI router by integrating it with the new worker management system. The primary goal is to centralize the discovery, health checking, and selection of OpenAI-compatible API endpoints through a WorkerRegistry. This change streamlines the router's internal logic, making it more robust and scalable by abstracting away direct endpoint management and delegating these responsibilities to the Worker abstraction. It ensures that the router dynamically adapts to available and healthy workers, improving overall system reliability and maintainability.

Highlights

• OpenAI Router Refactoring: The OpenAI router has been refactored to leverage the new worker crate, centralizing worker management and model discovery within the WorkerRegistry.
• WorkerRegistry Integration: The OpenAIRouter now relies on the AppContext's WorkerRegistry to find and select appropriate workers for models, replacing its previous direct management of worker URLs and endpoint probing logic.
• Simplified Router Creation: The create_openai_router function no longer requires a list of worker URLs, as workers are expected to be registered externally via the WorkerRegistry.
• Decentralized Circuit Breaker: Circuit breaker logic has been moved from the OpenAIRouter to individual Worker instances, allowing for more granular control and fault tolerance per worker.
• Enhanced Model Discovery: The get_models endpoint now aggregates model information directly from the ModelCards of registered workers in the WorkerRegistry, including primary IDs and aliases, rather than probing external endpoints.
• Updated Test Infrastructure: Test cases have been updated to reflect the new worker-centric design, using helper functions to register external workers in the AppContext before router initialization.