Skip to content

[Feature] Add Groq as Provider #602

@zerob13

Description

@zerob13

Is your feature request related to a problem? Please describe.
The application currently lacks support for the Groq API provider. Groq is renowned for its extremely high-speed inference capabilities with large language models. Not being able to use Groq limits the choice of providers and prevents users from leveraging potentially significant performance gains for their AI-powered features.

Describe the solution you'd like
I would like to request the addition of Groq as a new provider. Since Groq offers an OpenAI-compatible API, the integration should be relatively straightforward. The key information for the provider would be:

  • Provider Name: Groq
  • API Base URL: https://api.groq.com/openai/v1
  • Authentication: Bearer Token using GROQ_API_KEY.

Describe alternatives you've considered

Additional context
The official website for the service is: https://console.groq.com/docs/overview

Below is a sample cURL command demonstrating how to call their OpenAI-compatible chat completions endpoint:

curl -X POST https://api.groq.com/openai/v1/chat/completions \
-H "Authorization: Bearer $GROQ_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "llama-3.1-70b-versatile",
"messages": [{
    "role": "user",
    "content": "Explain the importance of fast language models"
}]
}'

你的功能请求是否与某个问题有关?请描述一下。
当前应用不支持 Groq 作为 LLM provider。Groq 以其极高的语言模型推理速度而闻名,缺少对它的支持,使用户无法利用其提供的高性能服务,限制了 Provider 的选择范围和应用的性能优化空间。

请描述你希望的解决方案
希望能支持添加 Groq 作为一个新的 Provider 选项。Groq 提供了与 OpenAI 兼容的 API,因此集成应可参照现有的 OpenAI provider 实现。关键配置信息如下:

  • Provider 名称: Groq
  • API Base URL: https://api.groq.com/openai/v1
  • 认证方式: 使用 GROQ_API_KEY 进行 Bearer Token 认证。

请描述你考虑过的其他替代方案

附加背景
该服务的官方网站是:https://console.groq.com/docs/overview

下方是一个调用其与 OpenAI 兼容的 chat/completions 接口的 cURL 请求示例:

curl -X POST https://api.groq.com/openai/v1/chat/completions \
-H "Authorization: Bearer $GROQ_API_KEY" \
-H "Content-Type: application/json" \
-d '{
"model": "llama-3.1-70b-versatile",
"messages": [{
    "role": "user",
    "content": "解释一下为什么快速的语言模型很重要"
}]
}'

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or requesthelp wantedExtra attention is needed

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions