Sglang looks great to me, but at my work, we use the Azure-OpenAI API. I don't see how to access this with sglang.
It would need two inputs in addition to the API-key, because at minimum I need to create the client like this:
client = openai.AzureOpenAI(
api_key="<your-api-key>",
base_url="https://<your-project-name>.openai.azure.com/openai",
api_version="<your-api-version>", # for example "2023-05-15"
)Also, for some reason the models are called "gpt-35-turbo" instead of "gpt-3.5-turbo" (missing dot); and I believe that you can call your models whatever you want. This should be supported, too.
If this already works somehow, I would appreciate an explicit mention in the README.md.
Sglang looks great to me, but at my work, we use the Azure-OpenAI API. I don't see how to access this with sglang.
It would need two inputs in addition to the API-key, because at minimum I need to create the client like this:
Also, for some reason the models are called "gpt-35-turbo" instead of "gpt-3.5-turbo" (missing dot); and I believe that you can call your models whatever you want. This should be supported, too.
If this already works somehow, I would appreciate an explicit mention in the
README.md.