fix(providers): route Responses-only models on custom OpenAI providers#2985
Conversation
Custom OpenAI-compatible providers (providers: section) without an explicit api_type were hard-defaulted to openai_chatcompletions, disabling the per-request endpoint auto-detection that the built-in openai/github-copilot providers rely on. Models served only via the Responses API (gpt-4.1, o-series, gpt-5, Codex) were therefore sent to /chat/completions and rejected with 400 unsupported_api_for_model — e.g. a github-copilot endpoint defined as a custom provider running gpt-5.3-codex. Default the api_type from the same modelinfo.SupportsResponsesAPI predicate so both paths stay consistent. An explicit provider- or model-level api_type still wins. Fixes #2303
docker-agent
left a comment
There was a problem hiding this comment.
Assessment: 🟡 NEEDS ATTENTION
One medium-severity behavioral regression was confirmed in the changed code.
Summary of findings:
- 🟡 MEDIUM
pkg/model/provider/defaults.go:47— Model-name-basedapi_typedefaulting silently breaks non-OpenAI custom providers whose model names match OpenAI patterns
| // modelinfo.SupportsResponsesAPI predicate keeps both paths consistent so a | ||
| // provider pointed at the OpenAI/Copilot API works without a manual | ||
| // api_type override. See https://github.com/docker/docker-agent/issues/2303. | ||
| func defaultOpenAIAPIType(model string) string { |
There was a problem hiding this comment.
[MEDIUM] Model-name-based api_type defaulting silently breaks non-OpenAI custom providers
defaultOpenAIAPIType now routes any model whose name matches SupportsResponsesAPI prefixes (gpt-4.1*, gpt-5*, codex*, o-series) to openai_responses. This is correct for providers actually pointing at the OpenAI or Copilot API — but the gate that triggers this logic (isOpenAICompatibleProvider(resolveEffectiveProvider(src))) is too broad:
- A custom provider declared with only a
base_urland no explicitProviderfield resolves its effective type to"openai"(viaresolveEffectiveProvider), which makesisOpenAICompatibleProviderreturntrue. - This means any URL-based third-party gateway (a local vLLM instance, LiteLLM proxy, Azure relay, etc.) that names its model
gpt-5-local,o3-preview, orcodex-customwill silently have requests routed to/responsesinstead of/chat/completions, causing 404 or 400 errors.
Before this PR: all custom providers defaulted to openai_chatcompletions — a safe universal default.
After this PR: the default is model-name–driven, and any pattern match sends traffic to /responses.
The workaround exists (api_type: openai_chatcompletions in the provider config), but users have no way to know they need it unless they read the source code.
Suggested mitigation: consider narrowing the condition — e.g., only apply defaultOpenAIAPIType when src.BaseURL contains openai.com or githubcopilot.com, or add a first-class boolean flag (responses_api_routing: true) that operators must opt into for custom providers.
Updates docs/providers/custom/index.md to reflect that `api_type` is now auto-detected based on the model name for custom OpenAI-compatible providers. Newer models (gpt-4.1, o-series, gpt-5, Codex) default to `openai_responses`; others default to `openai_chatcompletions`. Previously the docs showed a hardcoded default of `openai_chatcompletions` which was incorrect post-merge. Source PR: #2985
Problem
Custom OpenAI-compatible providers declared in the
providers:section withoutan explicit
api_typewere hard-defaulted toopenai_chatcompletions. Modelsthat are only served via the Responses API (gpt-4.1, the o-series, gpt-5, Codex)
were therefore sent to
/chat/completionsand rejected with400 unsupported_api_for_model— e.g. a github-copilot endpoint declared as acustom provider running
gpt-5.3-codex.Fixes #2303
Fix
pkg/model/provider/defaults.godefaultOpenAIAPIType(model): returnsopenai_responseswhenmodelinfo.SupportsResponsesAPI(model)is true, otherwiseopenai_chatcompletions.mergeFromProviderConfignow usesdefaultOpenAIAPIType(dst.Model)for theOpenAI-compatible fallback instead of always setting
openai_chatcompletions,reusing the same predicate as the built-in routing.
Tests
pkg/model/provider/custom_provider_test.goTestApplyProviderDefaults_DefaultsResponsesAPIForNewerModels: a customprovider with no
api_typeroutesgpt-5.3-codex,gpt-5,o3-minitoopenai_responses, whilegpt-4ostays onopenai_chatcompletions.TestApplyProviderDefaults_ExplicitAPITypeWinsOverModel: an explicitprovider-level
api_typeis honored even for a Responses-default model.