Skip to content

ChatCompletionResponse lost .choices and ChatCompletionRequest lost n field #620

@3mei

Description

@3mei

Priority Level

High (Major functionality broken)

Describe the bug

Between earlier data-designer versions and 0.5.9, the ModelFacade.completion() return shape changed from a litellm-style response (with .choices: list) to a single-message dataclass:

@dataclass
class ChatCompletionResponse:
    message: AssistantMessage
    usage: Usage | None = None
    raw: Any | None = None

At the same time, ChatCompletionRequest no longer exposes an n field, so one can't request multiple completions in a single HTTP call.

This breaks two patterns that worked previously and have no clean replacement in 0.5.9:

  1. Iterating over response.choices[0].message — the canonical pattern now raises AttributeError: 'ChatCompletionResponse' object has no attribute 'choices'.
  2. Requesting n > 1 completions (e.g., majority voting / best-of-N) — has to be emulated as n sequential HTTP calls, which (a) is n× slower, (b) defeats provider-side batching that some endpoints offer, and (c) collapses to identical outputs at low temperature unless one also forces a non-zero temperature distribution.

Steps/Code to reproduce bug

from data_designer.engine.models.facade import ModelFacade  # via ModelRegistry in a real plugin
# 1. Old-style access (was supported pre-0.5.9):
response = facade.completion(messages=[ChatMessage.as_user("hi")])
top = response.choices[0].message   # AttributeError in 0.5.9
# 2. Multiple completions in one call (was supported pre-0.5.9):
response = facade.completion(messages=[ChatMessage.as_user("hi")], n=4)
# n is silently dropped (logged as "Unknown kwargs ... dropped")
# response.message is a single AssistantMessage; no way to get 4 completions per call.

Expected behavior

Restore n on ChatCompletionRequest and have ChatCompletionResponse return a list of messages (or add a .choices property that exposes them) when n > 1.

Agent Diagnostic / Prior Investigation

No response

Additional context

No response

Checklist

  • I reproduced this issue or provided a minimal example
  • I searched the docs/issues myself, or had my agent do so
  • If I used an agent, I included its diagnostics above

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions