Conversation
|
Note Other AI code review bot(s) detectedCodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review. 📝 WalkthroughWalkthroughAdds a new Groq provider implementation and registration, exposes GROQ_API_KEY via env/config, replaces time-based IDs with UUIDs in Anthropic and Gemini providers, adds github.com/google/uuid dependency, and includes tests for the Groq provider. Changes
Sequence DiagramsequenceDiagram
participant Client
participant GroqProvider as Groq Provider
participant GroqAPI as Groq API
participant Converter as Stream Converter
Client->>GroqProvider: StreamResponses(ctx, ResponsesRequest)
GroqProvider->>GroqAPI: StreamChatCompletion(ctx, ChatRequest)
GroqAPI-->>GroqProvider: SSE / streaming payloads
rect rgb(240,248,255)
Note over GroqProvider,Converter: streaming conversion loop
GroqProvider->>Converter: newGroqResponsesStreamConverter(stream, model)
Converter->>Converter: parse SSE, extract deltas
Converter->>Client: emit events (response.created, response.output_text.delta, response.done)
end
Client->>Converter: Read() consume events
Client->>GroqProvider: Close() -> Converter.Close()
GroqProvider->>GroqAPI: close underlying connection
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: Organization UI Review profile: ASSERTIVE Plan: Pro 📒 Files selected for processing (2)
🧰 Additional context used📓 Path-based instructions (3)internal/providers/**/*.go📄 CodeRabbit inference engine (CLAUDE.md)
Files:
internal/providers/*/*.go📄 CodeRabbit inference engine (CLAUDE.md)
Files:
internal/**/*_test.go📄 CodeRabbit inference engine (CLAUDE.md)
Files:
🧠 Learnings (4)📓 Common learnings📚 Learning: 2025-12-26T16:40:36.115ZApplied to files:
📚 Learning: 2025-12-26T16:40:36.115ZApplied to files:
📚 Learning: 2025-12-26T16:40:36.115ZApplied to files:
🧬 Code graph analysis (1)internal/providers/groq/groq_test.go (4)
🔇 Additional comments (10)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 3
📜 Review details
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
📒 Files selected for processing (4)
.env.templatecmd/gomodel/main.goconfig/config.gointernal/providers/groq/groq.go
🧰 Additional context used
📓 Path-based instructions (4)
cmd/gomodel/main.go
📄 CodeRabbit inference engine (CLAUDE.md)
Provider packages must be imported in
cmd/gomodel/main.gousing blank imports (_ "gomodel/internal/providers/{name}")
Files:
cmd/gomodel/main.go
config/*.go
📄 CodeRabbit inference engine (CLAUDE.md)
Configuration must be loaded via Viper from environment variables and .env file, with at least one provider API key required
Files:
config/config.go
internal/providers/**/*.go
📄 CodeRabbit inference engine (CLAUDE.md)
internal/providers/**/*.go: Provider implementations must implement thecore.Providerinterface with methods: ChatCompletion, StreamChatCompletion, ListModels, Responses, and StreamResponses
Streaming responses must returnio.ReadCloserand the caller is responsible for closing the stream
Files:
internal/providers/groq/groq.go
internal/providers/*/*.go
📄 CodeRabbit inference engine (CLAUDE.md)
Providers must self-register with the factory using an
init()function that callsproviders.RegisterFactory()with the provider name
Files:
internal/providers/groq/groq.go
🧠 Learnings (10)
📓 Common learnings
Learnt from: CR
Repo: ENTERPILOT/GOModel PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-26T16:40:36.115Z
Learning: Applies to internal/providers/**/*.go : Provider implementations must implement the `core.Provider` interface with methods: ChatCompletion, StreamChatCompletion, ListModels, Responses, and StreamResponses
📚 Learning: 2025-12-26T16:40:36.115Z
Learnt from: CR
Repo: ENTERPILOT/GOModel PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-26T16:40:36.115Z
Learning: Applies to cmd/gomodel/main.go : Provider packages must be imported in `cmd/gomodel/main.go` using blank imports (`_ "gomodel/internal/providers/{name}"`)
Applied to files:
cmd/gomodel/main.go
📚 Learning: 2025-12-26T16:40:36.115Z
Learnt from: CR
Repo: ENTERPILOT/GOModel PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-26T16:40:36.115Z
Learning: Applies to internal/providers/*/*.go : Providers must self-register with the factory using an `init()` function that calls `providers.RegisterFactory()` with the provider name
Applied to files:
cmd/gomodel/main.gointernal/providers/groq/groq.go
📚 Learning: 2025-12-26T16:40:36.115Z
Learnt from: CR
Repo: ENTERPILOT/GOModel PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-26T16:40:36.115Z
Learning: Applies to internal/providers/**/*.go : Provider implementations must implement the `core.Provider` interface with methods: ChatCompletion, StreamChatCompletion, ListModels, Responses, and StreamResponses
Applied to files:
cmd/gomodel/main.gointernal/providers/groq/groq.go
📚 Learning: 2025-12-26T16:40:36.115Z
Learnt from: CR
Repo: ENTERPILOT/GOModel PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-26T16:40:36.115Z
Learning: Applies to internal/providers/router.go : Use the ModelRegistry to determine which provider handles each model; return `ErrRegistryNotInitialized` if registry is used before models are loaded
Applied to files:
cmd/gomodel/main.go
📚 Learning: 2025-12-26T16:40:36.115Z
Learnt from: CR
Repo: ENTERPILOT/GOModel PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-26T16:40:36.115Z
Learning: Applies to internal/providers/registry.go : Registry must load models from cache synchronously at startup, then refresh asynchronously in the background every 5 minutes to enable non-blocking I/O and instant startup
Applied to files:
cmd/gomodel/main.go
📚 Learning: 2025-12-26T16:40:36.115Z
Learnt from: CR
Repo: ENTERPILOT/GOModel PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-26T16:40:36.115Z
Learning: Applies to internal/cache/**/*.go : Model registry must support both local file cache and Redis cache backends for instant startup and multi-instance deployments
Applied to files:
cmd/gomodel/main.go
📚 Learning: 2025-12-26T16:40:36.115Z
Learnt from: CR
Repo: ENTERPILOT/GOModel PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-26T16:40:36.115Z
Learning: Applies to internal/providers/registry.go : Use thread-safe access with RWMutex in the ModelRegistry for concurrent access to model mappings
Applied to files:
cmd/gomodel/main.go
📚 Learning: 2025-12-26T16:40:36.115Z
Learnt from: CR
Repo: ENTERPILOT/GOModel PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-26T16:40:36.115Z
Learning: Applies to internal/providers/**/*.go : Streaming responses must return `io.ReadCloser` and the caller is responsible for closing the stream
Applied to files:
cmd/gomodel/main.gointernal/providers/groq/groq.go
📚 Learning: 2025-12-26T16:40:36.115Z
Learnt from: CR
Repo: ENTERPILOT/GOModel PR: 0
File: CLAUDE.md:0-0
Timestamp: 2025-12-26T16:40:36.115Z
Learning: Applies to config/*.go : Configuration must be loaded via Viper from environment variables and .env file, with at least one provider API key required
Applied to files:
config/config.go
🧬 Code graph analysis (1)
internal/providers/groq/groq.go (4)
internal/providers/factory.go (2)
RegisterProvider(28-38)GetGlobalHooks(66-68)internal/pkg/llmclient/client.go (3)
Client(110-116)Hooks(46-54)Request(178-183)internal/core/types.go (6)
ChatRequest(4-10)ChatResponse(19-26)ModelsResponse(51-54)Model(43-48)Message(13-16)Usage(36-40)internal/core/responses.go (5)
ResponsesRequest(5-14)ResponsesResponse(30-39)ResponsesOutputItem(42-48)ResponsesContentItem(51-55)ResponsesUsage(58-62)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
- GitHub Check: Analyze (go)
🔇 Additional comments (9)
cmd/gomodel/main.go (1)
20-20: LGTM! Groq provider correctly registered.The blank import follows the established pattern and will trigger the provider's
init()registration at startup..env.template (1)
37-39: LGTM! Configuration template is consistent.The Groq API key configuration follows the same pattern as existing providers.
config/config.go (1)
141-146: LGTM! Environment-based configuration is consistent.The Groq provider configuration follows the same pattern as existing providers (OpenAI, Anthropic, Gemini).
internal/providers/groq/groq.go (6)
24-33: LGTM! Provider registration and structure are correct.The
init()function properly self-registers the provider with the factory, and theProviderstruct follows established patterns. As per coding guidelines, provider self-registration is correctly implemented.
35-58: LGTM! Constructors and configuration methods are well-implemented.Both constructors properly apply global hooks for metrics collection, and
SetBaseURLenables custom base URL configuration as required by the factory pattern.
60-100: LGTM! Core interface methods correctly implemented.All methods properly implement the
core.Providerinterface:
ChatCompletion,StreamChatCompletion, andListModelsare correctly implemented- Streaming responses return
io.ReadCloseras required by coding guidelines- Request headers and endpoints follow Groq API conventions
As per coding guidelines and learnings.
204-232: LGTM! Responses API methods correctly implemented.Both
ResponsesandStreamResponsesmethods properly implement thecore.Providerinterface by converting between Responses and Chat formats. The streaming method correctly returnsio.ReadCloseras required by coding guidelines.
234-254: LGTM! Stream converter structure is well-designed.The
groqResponsesStreamConverterproperly encapsulates streaming state with appropriate buffer sizes and initialization logic.
417-420: LGTM! Close method is properly implemented.The
Closemethod correctly sets the closed flag and delegates to the underlying reader'sClosemethod, ensuring proper cleanup.
| // Parse the chat completion chunk | ||
| var chunk map[string]interface{} | ||
| if err := json.Unmarshal(data, &chunk); err != nil { | ||
| continue | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider logging failed chunk parsing for debugging.
When JSON unmarshaling fails for a chat completion chunk, the code silently continues without any indication. While this may be acceptable for malformed data, adding a debug-level log would help troubleshoot streaming issues.
🔎 Suggested improvement
// Parse the chat completion chunk
var chunk map[string]interface{}
if err := json.Unmarshal(data, &chunk); err != nil {
+ slog.Debug("failed to unmarshal chat chunk", "error", err, "data", string(data), "response_id", sc.responseID)
continue
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // Parse the chat completion chunk | |
| var chunk map[string]interface{} | |
| if err := json.Unmarshal(data, &chunk); err != nil { | |
| continue | |
| } | |
| // Parse the chat completion chunk | |
| var chunk map[string]interface{} | |
| if err := json.Unmarshal(data, &chunk); err != nil { | |
| slog.Debug("failed to unmarshal chat chunk", "error", err, "data", string(data), "response_id", sc.responseID) | |
| continue | |
| } |
🤖 Prompt for AI Agents
In internal/providers/groq/groq.go around lines 341 to 345, JSON unmarshaling
failures for chat completion chunks are currently ignored; add a debug-level log
that records the unmarshal error and a safe representation of the raw chunk
(e.g., string(data) trimmed or truncated) before continuing so streaming issues
can be diagnosed; keep the continue behavior but include the error and snippet
in the log call (use the existing logger in the file and ensure logs are guarded
at debug level to avoid noise).
0bb03bb to
ba37217
Compare
There was a problem hiding this comment.
Pull request overview
This PR adds a native Groq provider to the LLM gateway, enabling direct integration with Groq's API alongside existing providers like OpenAI, Anthropic, and Gemini.
Key changes:
- Implements a full-featured Groq provider with chat completions, streaming, model listing, and Responses API support
- Standardizes ID generation across providers by replacing time-based IDs with UUIDs in Gemini and Anthropic
- Adds configuration support for Groq via environment variables and YAML config
Reviewed changes
Copilot reviewed 8 out of 9 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| internal/providers/groq/groq.go | New Groq provider implementation with OpenAI-compatible API endpoints and stream format conversion |
| internal/providers/gemini/gemini.go | Replaced time-based ID generation with UUID for response and message IDs |
| internal/providers/anthropic/anthropic.go | Replaced time-based ID generation with UUID for response and message IDs |
| go.mod | Added github.com/google/uuid v1.6.0 dependency |
| go.sum | Added checksums for UUID package |
| config/config.yaml | Added groq-primary provider configuration |
| config/config.go | Added automatic Groq provider initialization from GROQ_API_KEY environment variable |
| cmd/gomodel/main.go | Added import for Groq provider package to trigger registration |
| .env.template | Added GROQ_API_KEY environment variable template |
Comments suppressed due to low confidence (1)
config/config.yaml:59
- The commented-out example for configuring Groq as an OpenAI-compatible provider is now redundant and potentially confusing since a native Groq provider has been added. Consider either removing this example or updating the comment to clarify that while a native Groq provider now exists, Groq can also be used via the OpenAI-compatible approach if needed.
# Example: Groq (OpenAI-compatible)
# groq:
# type: "openai"
# base_url: "https://api.groq.com/openai/v1"
# api_key: "${GROQ_API_KEY}"
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
internal/providers/groq/groq.go
Outdated
| } | ||
| jsonData, err := json.Marshal(deltaEvent) | ||
| if err != nil { | ||
| return 0, fmt.Errorf("failed to marshal content delta event: %w", err) |
There was a problem hiding this comment.
Error from marshaling the content delta event is not being logged or handled. Consider logging the error similar to how the Gemini provider handles marshaling errors in its stream converter. This would help with debugging if JSON marshaling fails.
internal/providers/groq/groq.go
Outdated
| return 0, fmt.Errorf("failed to marshal final response.done event: %w", err) | ||
| } | ||
| doneMsg := fmt.Sprintf("event: response.done\ndata: %s\n\ndata: [DONE]\n\n", jsonData) | ||
| sc.buffer = append(sc.buffer, []byte(doneMsg)...) |
There was a problem hiding this comment.
Error from marshaling the final response.done event is not being logged or handled. The Gemini provider logs this error and only appends the buffer if marshaling succeeds. This inconsistency could lead to silent failures that are difficult to debug.
| return 0, fmt.Errorf("failed to marshal final response.done event: %w", err) | |
| } | |
| doneMsg := fmt.Sprintf("event: response.done\ndata: %s\n\ndata: [DONE]\n\n", jsonData) | |
| sc.buffer = append(sc.buffer, []byte(doneMsg)...) | |
| // Log the error but do not fail the read; skip appending the final done event. | |
| fmt.Printf("failed to marshal final response.done event: %v\n", err) | |
| } else { | |
| doneMsg := fmt.Sprintf("event: response.done\ndata: %s\n\ndata: [DONE]\n\n", jsonData) | |
| sc.buffer = append(sc.buffer, []byte(doneMsg)...) | |
| } |
| // Package groq provides Groq API integration for the LLM gateway. | ||
| package groq | ||
|
|
||
| import ( | ||
| "bytes" | ||
| "context" | ||
| "encoding/json" | ||
| "fmt" | ||
| "io" | ||
| "net/http" | ||
| "strings" | ||
| "time" | ||
|
|
||
| "github.com/google/uuid" | ||
|
|
||
| "gomodel/internal/core" | ||
| "gomodel/internal/pkg/llmclient" | ||
| "gomodel/internal/providers" | ||
| ) | ||
|
|
||
| const ( | ||
| defaultBaseURL = "https://api.groq.com/openai/v1" | ||
| ) | ||
|
|
||
| func init() { | ||
| // Self-register with the factory | ||
| providers.RegisterProvider("groq", New) | ||
| } | ||
|
|
||
| // Provider implements the core.Provider interface for Groq | ||
| type Provider struct { | ||
| client *llmclient.Client | ||
| apiKey string | ||
| } | ||
|
|
||
| // New creates a new Groq provider | ||
| func New(apiKey string) *Provider { | ||
| p := &Provider{apiKey: apiKey} | ||
| cfg := llmclient.DefaultConfig("groq", defaultBaseURL) | ||
| // Apply global hooks if available | ||
| cfg.Hooks = providers.GetGlobalHooks() | ||
| p.client = llmclient.New(cfg, p.setHeaders) | ||
| return p | ||
| } | ||
|
|
||
| // NewWithHTTPClient creates a new Groq provider with a custom HTTP client | ||
| func NewWithHTTPClient(apiKey string, httpClient *http.Client) *Provider { | ||
| p := &Provider{apiKey: apiKey} | ||
| cfg := llmclient.DefaultConfig("groq", defaultBaseURL) | ||
| // Apply global hooks if available | ||
| cfg.Hooks = providers.GetGlobalHooks() | ||
| p.client = llmclient.NewWithHTTPClient(httpClient, cfg, p.setHeaders) | ||
| return p | ||
| } | ||
|
|
||
| // SetBaseURL allows configuring a custom base URL for the provider | ||
| func (p *Provider) SetBaseURL(url string) { | ||
| p.client.SetBaseURL(url) | ||
| } | ||
|
|
||
| // setHeaders sets the required headers for Groq API requests | ||
| func (p *Provider) setHeaders(req *http.Request) { | ||
| req.Header.Set("Authorization", "Bearer "+p.apiKey) | ||
| } | ||
|
|
||
| // ChatCompletion sends a chat completion request to Groq | ||
| func (p *Provider) ChatCompletion(ctx context.Context, req *core.ChatRequest) (*core.ChatResponse, error) { | ||
| var resp core.ChatResponse | ||
| err := p.client.Do(ctx, llmclient.Request{ | ||
| Method: http.MethodPost, | ||
| Endpoint: "/chat/completions", | ||
| Body: req, | ||
| }, &resp) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| return &resp, nil | ||
| } | ||
|
|
||
| // StreamChatCompletion returns a raw response body for streaming (caller must close) | ||
| func (p *Provider) StreamChatCompletion(ctx context.Context, req *core.ChatRequest) (io.ReadCloser, error) { | ||
| return p.client.DoStream(ctx, llmclient.Request{ | ||
| Method: http.MethodPost, | ||
| Endpoint: "/chat/completions", | ||
| Body: req.WithStreaming(), | ||
| }) | ||
| } | ||
|
|
||
| // ListModels retrieves the list of available models from Groq | ||
| func (p *Provider) ListModels(ctx context.Context) (*core.ModelsResponse, error) { | ||
| var resp core.ModelsResponse | ||
| err := p.client.Do(ctx, llmclient.Request{ | ||
| Method: http.MethodGet, | ||
| Endpoint: "/models", | ||
| }, &resp) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
| return &resp, nil | ||
| } | ||
|
|
||
| // convertResponsesRequestToChat converts a ResponsesRequest to ChatRequest for Groq | ||
| func convertResponsesRequestToChat(req *core.ResponsesRequest) *core.ChatRequest { | ||
| chatReq := &core.ChatRequest{ | ||
| Model: req.Model, | ||
| Messages: make([]core.Message, 0), | ||
| Temperature: req.Temperature, | ||
| Stream: req.Stream, | ||
| } | ||
|
|
||
| if req.MaxOutputTokens != nil { | ||
| chatReq.MaxTokens = req.MaxOutputTokens | ||
| } | ||
|
|
||
| // Add system instruction if provided | ||
| if req.Instructions != "" { | ||
| chatReq.Messages = append(chatReq.Messages, core.Message{ | ||
| Role: "system", | ||
| Content: req.Instructions, | ||
| }) | ||
| } | ||
|
|
||
| // Convert input to messages | ||
| switch input := req.Input.(type) { | ||
| case string: | ||
| chatReq.Messages = append(chatReq.Messages, core.Message{ | ||
| Role: "user", | ||
| Content: input, | ||
| }) | ||
| case []interface{}: | ||
| for _, item := range input { | ||
| if msgMap, ok := item.(map[string]interface{}); ok { | ||
| role, _ := msgMap["role"].(string) | ||
| content := extractContentFromInput(msgMap["content"]) | ||
| if role != "" && content != "" { | ||
| chatReq.Messages = append(chatReq.Messages, core.Message{ | ||
| Role: role, | ||
| Content: content, | ||
| }) | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| return chatReq | ||
| } | ||
|
|
||
| // extractContentFromInput extracts text content from responses input | ||
| func extractContentFromInput(content interface{}) string { | ||
| switch c := content.(type) { | ||
| case string: | ||
| return c | ||
| case []interface{}: | ||
| // Array of content parts - extract text | ||
| var texts []string | ||
| for _, part := range c { | ||
| if partMap, ok := part.(map[string]interface{}); ok { | ||
| if text, ok := partMap["text"].(string); ok { | ||
| texts = append(texts, text) | ||
| } | ||
| } | ||
| } | ||
| return strings.Join(texts, " ") | ||
| } | ||
| return "" | ||
| } | ||
|
|
||
| // convertChatResponseToResponses converts a ChatResponse to ResponsesResponse | ||
| func convertChatResponseToResponses(resp *core.ChatResponse) *core.ResponsesResponse { | ||
| content := "" | ||
| if len(resp.Choices) > 0 { | ||
| content = resp.Choices[0].Message.Content | ||
| } | ||
|
|
||
| return &core.ResponsesResponse{ | ||
| ID: resp.ID, | ||
| Object: "response", | ||
| CreatedAt: resp.Created, | ||
| Model: resp.Model, | ||
| Status: "completed", | ||
| Output: []core.ResponsesOutputItem{ | ||
| { | ||
| ID: "msg_" + uuid.New().String(), | ||
| Type: "message", | ||
| Role: "assistant", | ||
| Status: "completed", | ||
| Content: []core.ResponsesContentItem{ | ||
| { | ||
| Type: "output_text", | ||
| Text: content, | ||
| Annotations: []string{}, | ||
| }, | ||
| }, | ||
| }, | ||
| }, | ||
| Usage: &core.ResponsesUsage{ | ||
| InputTokens: resp.Usage.PromptTokens, | ||
| OutputTokens: resp.Usage.CompletionTokens, | ||
| TotalTokens: resp.Usage.TotalTokens, | ||
| }, | ||
| } | ||
| } | ||
|
|
||
| // Responses sends a Responses API request to Groq (converted to chat format) | ||
| func (p *Provider) Responses(ctx context.Context, req *core.ResponsesRequest) (*core.ResponsesResponse, error) { | ||
| // Convert ResponsesRequest to ChatRequest | ||
| chatReq := convertResponsesRequestToChat(req) | ||
|
|
||
| // Use the existing ChatCompletion method | ||
| chatResp, err := p.ChatCompletion(ctx, chatReq) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| return convertChatResponseToResponses(chatResp), nil | ||
| } | ||
|
|
||
| // StreamResponses returns a raw response body for streaming Responses API (caller must close) | ||
| func (p *Provider) StreamResponses(ctx context.Context, req *core.ResponsesRequest) (io.ReadCloser, error) { | ||
| // Convert ResponsesRequest to ChatRequest | ||
| chatReq := convertResponsesRequestToChat(req) | ||
| chatReq.Stream = true | ||
|
|
||
| // Get the streaming response from chat completions | ||
| stream, err := p.StreamChatCompletion(ctx, chatReq) | ||
| if err != nil { | ||
| return nil, err | ||
| } | ||
|
|
||
| // Wrap the stream to convert chat completion format to Responses API format | ||
| return newGroqResponsesStreamConverter(stream, req.Model), nil | ||
| } | ||
|
|
||
| // groqResponsesStreamConverter wraps a chat completion stream and converts it to Responses API format | ||
| type groqResponsesStreamConverter struct { | ||
| reader io.ReadCloser | ||
| model string | ||
| responseID string | ||
| buffer []byte | ||
| lineBuffer []byte | ||
| closed bool | ||
| sentCreate bool | ||
| sentDone bool | ||
| } | ||
|
|
||
| func newGroqResponsesStreamConverter(reader io.ReadCloser, model string) *groqResponsesStreamConverter { | ||
| return &groqResponsesStreamConverter{ | ||
| reader: reader, | ||
| model: model, | ||
| responseID: "resp_" + uuid.New().String(), | ||
| buffer: make([]byte, 0, 4096), | ||
| lineBuffer: make([]byte, 0, 1024), | ||
| } | ||
| } | ||
|
|
||
| func (sc *groqResponsesStreamConverter) Read(p []byte) (n int, err error) { | ||
| if sc.closed { | ||
| return 0, io.EOF | ||
| } | ||
|
|
||
| // If we have buffered data, return it first | ||
| if len(sc.buffer) > 0 { | ||
| n = copy(p, sc.buffer) | ||
| sc.buffer = sc.buffer[n:] | ||
| return n, nil | ||
| } | ||
|
|
||
| // Send response.created event first | ||
| if !sc.sentCreate { | ||
| sc.sentCreate = true | ||
| createdEvent := map[string]interface{}{ | ||
| "type": "response.created", | ||
| "response": map[string]interface{}{ | ||
| "id": sc.responseID, | ||
| "object": "response", | ||
| "status": "in_progress", | ||
| "model": sc.model, | ||
| "created_at": time.Now().Unix(), | ||
| }, | ||
| } | ||
| jsonData, err := json.Marshal(createdEvent) | ||
| if err != nil { | ||
| return 0, fmt.Errorf("failed to marshal response.created event: %w", err) | ||
| } | ||
| created := fmt.Sprintf("event: response.created\ndata: %s\n\n", jsonData) | ||
| sc.buffer = append(sc.buffer, []byte(created)...) | ||
| n = copy(p, sc.buffer) | ||
| sc.buffer = sc.buffer[n:] | ||
| return n, nil | ||
| } | ||
|
|
||
| // Read from the underlying stream | ||
| tempBuf := make([]byte, 1024) | ||
| nr, readErr := sc.reader.Read(tempBuf) | ||
| if nr > 0 { | ||
| sc.lineBuffer = append(sc.lineBuffer, tempBuf[:nr]...) | ||
|
|
||
| // Process complete lines | ||
| for { | ||
| idx := bytes.Index(sc.lineBuffer, []byte("\n")) | ||
| if idx == -1 { | ||
| break | ||
| } | ||
|
|
||
| line := sc.lineBuffer[:idx] | ||
| sc.lineBuffer = sc.lineBuffer[idx+1:] | ||
|
|
||
| line = bytes.TrimSpace(line) | ||
| if len(line) == 0 { | ||
| continue | ||
| } | ||
|
|
||
| if bytes.HasPrefix(line, []byte("data: ")) { | ||
| data := bytes.TrimPrefix(line, []byte("data: ")) | ||
| if bytes.Equal(data, []byte("[DONE]")) { | ||
| // Send done event | ||
| if !sc.sentDone { | ||
| sc.sentDone = true | ||
| doneEvent := map[string]interface{}{ | ||
| "type": "response.done", | ||
| "response": map[string]interface{}{ | ||
| "id": sc.responseID, | ||
| "object": "response", | ||
| "status": "completed", | ||
| "model": sc.model, | ||
| "created_at": time.Now().Unix(), | ||
| }, | ||
| } | ||
| jsonData, err := json.Marshal(doneEvent) | ||
| if err != nil { | ||
| return 0, fmt.Errorf("failed to marshal response.done event: %w", err) | ||
| } | ||
| doneMsg := fmt.Sprintf("event: response.done\ndata: %s\n\ndata: [DONE]\n\n", jsonData) | ||
| sc.buffer = append(sc.buffer, []byte(doneMsg)...) | ||
| } | ||
| continue | ||
| } | ||
|
|
||
| // Parse the chat completion chunk | ||
| var chunk map[string]interface{} | ||
| if err := json.Unmarshal(data, &chunk); err != nil { | ||
| continue | ||
| } | ||
|
|
||
| // Extract content delta | ||
| if choices, ok := chunk["choices"].([]interface{}); ok && len(choices) > 0 { | ||
| if choice, ok := choices[0].(map[string]interface{}); ok { | ||
| if delta, ok := choice["delta"].(map[string]interface{}); ok { | ||
| if content, ok := delta["content"].(string); ok && content != "" { | ||
| deltaEvent := map[string]interface{}{ | ||
| "type": "response.output_text.delta", | ||
| "delta": content, | ||
| } | ||
| jsonData, err := json.Marshal(deltaEvent) | ||
| if err != nil { | ||
| return 0, fmt.Errorf("failed to marshal content delta event: %w", err) | ||
| } | ||
| sc.buffer = append(sc.buffer, []byte(fmt.Sprintf("event: response.output_text.delta\ndata: %s\n\n", jsonData))...) | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
| } | ||
|
|
||
| if readErr != nil { | ||
| if readErr == io.EOF { | ||
| // Send final done event if we haven't already | ||
| if !sc.sentDone { | ||
| sc.sentDone = true | ||
| doneEvent := map[string]interface{}{ | ||
| "type": "response.done", | ||
| "response": map[string]interface{}{ | ||
| "id": sc.responseID, | ||
| "object": "response", | ||
| "status": "completed", | ||
| "model": sc.model, | ||
| "created_at": time.Now().Unix(), | ||
| }, | ||
| } | ||
| jsonData, err := json.Marshal(doneEvent) | ||
| if err != nil { | ||
| return 0, fmt.Errorf("failed to marshal final response.done event: %w", err) | ||
| } | ||
| doneMsg := fmt.Sprintf("event: response.done\ndata: %s\n\ndata: [DONE]\n\n", jsonData) | ||
| sc.buffer = append(sc.buffer, []byte(doneMsg)...) | ||
| } | ||
|
|
||
| if len(sc.buffer) > 0 { | ||
| n = copy(p, sc.buffer) | ||
| sc.buffer = sc.buffer[n:] | ||
| return n, nil | ||
| } | ||
|
|
||
| sc.closed = true | ||
| _ = sc.reader.Close() | ||
| return 0, io.EOF | ||
| } | ||
| return 0, readErr | ||
| } | ||
|
|
||
| if len(sc.buffer) > 0 { | ||
| n = copy(p, sc.buffer) | ||
| sc.buffer = sc.buffer[n:] | ||
| return n, nil | ||
| } | ||
|
|
||
| // No data yet, try again | ||
| return 0, nil | ||
| } | ||
|
|
||
| func (sc *groqResponsesStreamConverter) Close() error { | ||
| sc.closed = true | ||
| return sc.reader.Close() | ||
| } |
There was a problem hiding this comment.
The Groq provider lacks test coverage. All other providers (OpenAI, Gemini, Anthropic) have comprehensive test files with hundreds of lines of tests. Consider adding a groq_test.go file with tests for ChatCompletion, StreamChatCompletion, ListModels, Responses, StreamResponses, and the stream converter functionality to maintain consistency with other providers.
internal/providers/groq/groq.go
Outdated
| } | ||
| jsonData, err := json.Marshal(createdEvent) | ||
| if err != nil { | ||
| return 0, fmt.Errorf("failed to marshal response.created event: %w", err) |
There was a problem hiding this comment.
Error from marshaling the response.created event is not being logged or handled. Consider logging the error similar to how the Gemini provider handles marshaling errors in its stream converter. This would help with debugging if JSON marshaling fails.
internal/providers/groq/groq.go
Outdated
| } | ||
| jsonData, err := json.Marshal(doneEvent) | ||
| if err != nil { | ||
| return 0, fmt.Errorf("failed to marshal response.done event: %w", err) |
There was a problem hiding this comment.
Error from marshaling the response.done event is not being logged or handled. Consider logging the error similar to how the Gemini provider handles marshaling errors in its stream converter. This would help with debugging if JSON marshaling fails.
Summary by CodeRabbit
New Features
Refactor
Tests
Chores
✏️ Tip: You can customize this high-level summary in your review settings.