Points: 2-3 days
Description: Implement the core utility endpoints needed by all OpenAI-compatible clients.
Deliverables:
- Implement
/v1/models endpoint to list available models
- Add engine lifecycle management in the lifespan handler, following
http_server.py in python/sglang/srt/http_server.py
- Add metrics middleware
- Add all openai endpoints, skip implementation for now
Testing:
- Verify proper model listing
- Ensure metrics are correctly exposed
Points: 2-3 days
Description: Implement the core utility endpoints needed by all OpenAI-compatible clients.
Deliverables:
/v1/modelsendpoint to list available modelshttp_server.pyinpython/sglang/srt/http_server.pyTesting: