Summary

• Enable IGW mode to spin up gRPC routers (regular and PD) and select them preferentially when matching workers are present.
• Auto-enable IGW when service discovery is requested, aligning router initialization with discovery-driven worker registration.
• Improve readiness reporting by returning 200 when any registered worker is healthy.

Motivation

• IGW mode internally uses all types of routers; previously parser factories and router creation logic were tied to explicit single gRPC instance, leaving IGW without the necessary gRPC router coverage.
• Service discovery implies IGW behavior; requiring a separate flag led to misconfiguration risk and silent misalignment between discovery and router mode.
• Operators need the router manager to pick the best transport (gRPC vs HTTP, PD vs regular) based on available workers and to surface health when capacity exists.

Modifications

• sgl-model-gateway/src/app_context.rs: Initialize reasoning/tool parser factories when either gRPC or IGW is enabled, reflecting IGW’s gRPC router usage.
• sgl-model-gateway/src/main.rs: Decouple routing-mode selection from IGW, streamline PD routing config, and automatically flip enable_igw on when service discovery is requested (with an info log).
• sgl-model-gateway/src/routers/router_manager.rs: Always create a gRPC regular router in IGW mode; create HTTP and gRPC PD routers only when PD disaggregation is enabled; choose routers based on worker connection mode/role priority (grpc-pd > http-pd > grpc-regular > http-regular); update health endpoint to report ready if any worker is healthy.

Accuracy Tests

Benchmarking and Profiling

Screenshot

Checklist

• Format your code according to the Format code with pre-commit.
• Add unit tests according to the Run and add unit tests.
• Update documentation according to Write documentations.
• Provide accuracy and speed benchmark results according to Test the accuracy and Benchmark the speed.
• Follow the SGLang code style guidance.
• Work with maintainers to merge your PR. See the PR Merge Process