Skip to content

ChatCompletionResponse lost .choices and ChatCompletionRequest lost n field #620

@3mei

Description

@3mei

Priority Level

High (Major functionality broken)

Describe the bug

Between earlier data-designer versions and 0.5.9, the ModelFacade.completion() return shape changed from a litellm-style response (with .choices: list) to a single-message dataclass:

@dataclass
class ChatCompletionResponse:
    message: AssistantMessage
    usage: Usage | None = None
    raw: Any | None = None

At the same time, ChatCompletionRequest no longer exposes an n field, so one can't request multiple completions in a single HTTP call.

This breaks two patterns that worked previously and have no clean replacement in 0.5.9:

  1. Iterating over response.choices[0].message — the canonical pattern now raises AttributeError: 'ChatCompletionResponse' object has no attribute 'choices'.
  2. Requesting n > 1 completions (e.g., majority voting / best-of-N) — has to be emulated as n sequential HTTP calls, which (a) is n× slower, (b) defeats provider-side batching that some endpoints offer, and (c) collapses to identical outputs at low temperature unless one also forces a non-zero temperature distribution.

Steps/Code to reproduce bug

from data_designer.engine.models.facade import ModelFacade  # via ModelRegistry in a real plugin
# 1. Old-style access (was supported pre-0.5.9):
response = facade.completion(messages=[ChatMessage.as_user("hi")])
top = response.choices[0].message   # AttributeError in 0.5.9
# 2. Multiple completions in one call (was supported pre-0.5.9):
response = facade.completion(messages=[ChatMessage.as_user("hi")], n=4)
# n is silently dropped (logged as "Unknown kwargs ... dropped")
# response.message is a single AssistantMessage; no way to get 4 completions per call.

Expected behavior

Restore n on ChatCompletionRequest and have ChatCompletionResponse return a list of messages (or add a .choices property that exposes them) when n > 1.

Agent Diagnostic / Prior Investigation

No response

Additional context

No response

Checklist

  • I reproduced this issue or provided a minimal example
  • I searched the docs/issues myself, or had my agent do so
  • If I used an agent, I included its diagnostics above

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions