fix(models): consolidated validation + picker — anthropic_messages, native Anthropic, Gemini prefix, OpenAI catalog#15136
Merged
Merged
Conversation
…are-protected endpoints - probe_api_models: add api_mode param; use x-api-key + anthropic-version headers for anthropic_messages mode (Anthropic's native Models API auth) - probe_api_models: add User-Agent header to avoid Cloudflare 403 blocks on third-party OpenAI-compatible endpoints - validate_requested_model: pass api_mode through from switch_model - validate_requested_model: for anthropic_messages mode, attempt probe with correct auth; if probe fails (many proxies don't implement /v1/models), accept the model with an informational warning instead of rejecting - fetch_api_models: propagate api_mode to probe_api_models
The generic /v1/models probe in validate_requested_model() sent a plain 'Authorization: Bearer <key>' header, which works for OpenAI-compatible endpoints but results in a 401 Unauthorized from Anthropic's API. Anthropic requires x-api-key + anthropic-version headers (or Bearer for OAuth tokens from Claude Code). Add a provider-specific branch for normalized == 'anthropic' that calls the existing _fetch_anthropic_models() helper, which already handles both regular API keys and Claude Code OAuth tokens correctly. This mirrors the pattern already used for openai-codex, copilot, and bedrock. The branch also includes: - fuzzy auto-correct (cutoff 0.9) for near-exact model ID typos - fuzzy suggestions (cutoff 0.5) when the model is not listed - graceful fall-through when the token cannot be resolved or the network is unreachable (accepts with a warning rather than hard-fail) - a note that newer/preview/snapshot model IDs can be gate-listed and may still work even if not returned by /v1/models Fixes Anthropic provider users seeing 'service unreachable' errors when running /model <claude-model> because every probe 401'd.
Salvage of the Gemini-specific piece from PR #12585 by @briandevans. Gemini's OpenAI-compat /v1beta/openai/models endpoint returns IDs prefixed with 'models/' (native Gemini-API convention), so set-membership against curated bare IDs drops every model. Strip the prefix before comparison. The Anthropic static-catalog piece of #12585 was subsumed by #12618's _fetch_anthropic_models() branch landing earlier in the same salvage PR. Full branch cherry-pick was skipped because it also carried unrelated catalog-version regressions.
This was referenced Apr 24, 2026
19 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
6 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Consolidated salvage of 4 model validation /
/modelpicker PRs —anthropic_messagesmode routing, native Anthropic validator, Geminimodels/prefix, and OpenAI live/modelcounts. Attribution preserved via rebase-merge.Changes
hermes_cli/models.py+hermes_cli/main.py+hermes_cli/model_switch.py(anthropic_messages + Cloudflare) —probe_api_models()now takesapi_modeand sendsx-api-key+anthropic-versionheaders whenapi_mode == "anthropic_messages"instead ofAuthorization: Bearer. Always setsUser-Agent: hermes-cli/<version>(Cloudflare 1010 bypass). Custom endpoints where the/modelsprobe is unreachable no longer hard-reject — the model is persisted with a warning so proxy endpoints that don't implement/v1/modelsstill work.api_modeis threaded through the pipeline (validate_requested_model→probe_api_models→fetch_api_models). From @Wangshengyang2004 (fix(cli): model validation fails for anthropic_messages and Cloudflare-protected endpoints #12950). Subsumes @cedric-common's fix(models): use x-api-key for Anthropic API probing #13189 (URL-based) with mode-based detection.hermes_cli/models.py(native Anthropic validator) —validate_requested_modelgained a dedicatednormalized == "anthropic"branch that calls_fetch_anthropic_models()(properx-api-key+anthropic-versionheaders; also handles Claude Code OAuth tokens). Fuzzy auto-correction on close matches, suggestions on unknown IDs, accept with warning on snapshot / early-access IDs that Anthropic gates outside/v1/models. From @H-Ali13381 (fix(models): use Anthropic-native headers for model validation #12618).hermes_cli/models.py(Geminimodels/prefix) — Gemini's OpenAI-compat/v1beta/openai/modelsendpoint returns IDs prefixed withmodels/(native Gemini-API convention). The set-membership check dropped every curated Gemini ID. Strip the prefix before comparison whennormalized == "gemini". Fixes Gateway /model picker fails for Gemini and Anthropic providers (validate_requested_model rejects curated models) #12532. From @briandevans (fix(models): accept Gemini + Anthropic in gateway /model picker (#12532) #12585 — Gemini piece only; the Anthropic piece of that PR was subsumed by fix(models): use Anthropic-native headers for model validation #12618 above, and the full branch carried unrelated catalog-version regressions).hermes_cli/model_switch.py+hermes_cli/models.py(OpenAI picker counts) —_PROVIDER_MODELS["openai"]gained a curated static catalog.provider_model_ids("openai")probes/v1/modelslive whenOPENAI_API_KEYis set, falling back to the catalog.list_authenticated_providers()uses the catalog when a provider row'smodelsdict is empty and the base_url isapi.openai.com, soOpenAI/OpenAI Directrows no longer show0 models. Fixes bug(model-picker): OpenAI and OpenAI Direct show 0 models on latest main #14651. From @XieNBi (fix(cli): non-zero /model counts for native OpenAI and direct API rows #14753).Credit
Validation
181/181 passing.
hermes_cli/models.py,hermes_cli/model_switch.py,hermes_cli/main.pyall compile.One stale test updated —
test_custom_endpoint_warns_with_probed_url_and_v1_hintexpectedpersist=Falseon probe failure; after #12950 it'spersist=True(the intentional behavior change, with a comment pointing at #12950).Conflict resolutions
_HERMES_USER_AGENTconstant.anthropicbranch AFTER the MiniMax branch but BEFORE theapi_mode=="anthropic_messages"branch; both coexist (different provider vs. different transport).--author=briandevans.Not included
x-api-keytrigger subsumed by fix(cli): model validation fails for anthropic_messages and Cloudflare-protected endpoints #12950's mode-based detection. Will close with credit.