Skip to content

[Bug] json_mode=True fails on Alibaba/Qwen providers — "messages must contain the word 'json'" #663

@kevin-ho

Description

@kevin-ho

The OpenAI API spec requires that when response_format of type json_object is used, the word "json" must appear somewhere in the messages. Most OpenAI-compatible providers (OpenAI itself, NVIDIA, etc.) are forgiving about enforcing this — they'll return valid JSON regardless. However, Alibaba-backed Qwen models strictly enforce it and reject the call:

[400]: Error from provider (Alibaba): <400> InternalError.Algo.InvalidParameter: 'messages' must contain the word 'json' in some form, to use 'response_format' of type 'json_object'.

This affects the deriver, which calls honcho_llm_call(json_mode=True, response_model=PromptRepresentation). The OpenAI SDK's parse() method sends response_format as json_schema, but Alibaba internally converts this to json_object and enforces the keyword check — the word "json" must appear somewhere in the messages.

Root cause:

Alibaba/Qwen strictly enforces the OpenAI spec requirement that "json" (case-insensitive) must appear in the messages whenever response_format is set, regardless of whether it's json_object or json_schema. Other providers are lenient about this check. The OpenAI backend never ensures the keyword is present, so any json_mode=True call routed to a Qwen model fails immediately.

Proposed fix:

Add an _ensure_json_keyword() helper to the OpenAI backend that checks whether "json" appears in the messages when json_mode=True is set. If not, appends a brief instruction ("Respond in valid JSON format.") to the system message. This is gated on json_mode specifically — not on every response_format call — so it only activates when the caller explicitly requests JSON output, avoiding noise for structured output calls to providers that don't need it.

Reproduction:

# Via any OpenAI-compatible proxy pointing to Alibaba Qwen
response = await client.chat.completions.create(
    model="qwen3.6-plus",
    messages=[{"role": "user", "content": "Hello"}],
    response_format={"type": "json_object"},
)
# → 400: 'messages' must contain the word 'json'

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions