Summary

The sgl-router codebase has evolved to support multiple routing modes (Regular, PrefillDecode, OpenAI) and deployment configurations (single-router vs multi-router IGW mode). However, this evolution has resulted in architectural inconsistencies that create maintenance challenges and limit functionality:

1. Inconsistent Worker Management: HTTP routers use centralized WorkerRegistry while gRPC routers manage workers internally, creating two different worker management systems in the same codebase.
2. Constructor Coupling: All routers create and register workers in their constructors, violating separation of concerns and making routers hard to test without real worker infrastructure.
3. IGW Mode Incompatibility: gRPC routers cannot work in IGW (multi-router) mode because they bypass the centralized registry system that RouterManager depends on.
4. Code Duplication: Multiple implementations of health checking, policy management, and worker storage exist across different router types.
5. Initialization Complexity: The split between IGW and non-IGW modes in server.rs creates complex conditional logic and different initialization paths.

Goals

gRPC Router Refactoring

• Update grpc::Router to use WorkerRegistry
• Update grpc::PDRouter to use WorkerRegistry

Extract Worker Creation

• Create WorkerInitializer module
• Update all router constructors
• Update server.rs initialization flow

Unify Router Management

• Improve RouterManager for single-router mode
• Remove conditional logic in server.rs