-
Notifications
You must be signed in to change notification settings - Fork 614
Description
Is your feature request related to a problem? Please describe.
The application currently lacks support for the Groq API provider. Groq is renowned for its extremely high-speed inference capabilities with large language models. Not being able to use Groq limits the choice of providers and prevents users from leveraging potentially significant performance gains for their AI-powered features.
Describe the solution you'd like
I would like to request the addition of Groq as a new provider. Since Groq offers an OpenAI-compatible API, the integration should be relatively straightforward. The key information for the provider would be:
- Provider Name: Groq
- API Base URL:
https://api.groq.com/openai/v1 - Authentication: Bearer Token using
GROQ_API_KEY.
Describe alternatives you've considered
Additional context
The official website for the service is: https://console.groq.com/docs/overview
Below is a sample cURL command demonstrating how to call their OpenAI-compatible chat completions endpoint:
curl -X POST https://api.groq.com/openai/v1/chat/completions \
-H "Authorization: Bearer $GROQ_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "llama-3.1-70b-versatile",
"messages": [{
"role": "user",
"content": "Explain the importance of fast language models"
}]
}'你的功能请求是否与某个问题有关?请描述一下。
当前应用不支持 Groq 作为 LLM provider。Groq 以其极高的语言模型推理速度而闻名,缺少对它的支持,使用户无法利用其提供的高性能服务,限制了 Provider 的选择范围和应用的性能优化空间。
请描述你希望的解决方案
希望能支持添加 Groq 作为一个新的 Provider 选项。Groq 提供了与 OpenAI 兼容的 API,因此集成应可参照现有的 OpenAI provider 实现。关键配置信息如下:
- Provider 名称: Groq
- API Base URL:
https://api.groq.com/openai/v1 - 认证方式: 使用
GROQ_API_KEY进行 Bearer Token 认证。
请描述你考虑过的其他替代方案
附加背景
该服务的官方网站是:https://console.groq.com/docs/overview
下方是一个调用其与 OpenAI 兼容的 chat/completions 接口的 cURL 请求示例:
curl -X POST https://api.groq.com/openai/v1/chat/completions \
-H "Authorization: Bearer $GROQ_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "llama-3.1-70b-versatile",
"messages": [{
"role": "user",
"content": "解释一下为什么快速的语言模型很重要"
}]
}'