fix(cli): model validation fails for anthropic_messages and Cloudflare-protected endpoints#12950
Closed
Wangshengyang2004 wants to merge 1 commit into
Closed
Conversation
…are-protected endpoints - probe_api_models: add api_mode param; use x-api-key + anthropic-version headers for anthropic_messages mode (Anthropic's native Models API auth) - probe_api_models: add User-Agent header to avoid Cloudflare 403 blocks on third-party OpenAI-compatible endpoints - validate_requested_model: pass api_mode through from switch_model - validate_requested_model: for anthropic_messages mode, attempt probe with correct auth; if probe fails (many proxies don't implement /v1/models), accept the model with an informational warning instead of rejecting - fetch_api_models: propagate api_mode to probe_api_models
7acbf7a to
a53489f
Compare
3 tasks
Collaborator
Collaborator
This was referenced Apr 23, 2026
Contributor
|
Thanks for this fix, @Wangshengyang2004! The changes from this PR have already landed on This is an automated hermes-sweeper review. Evidence:
Closing as implemented. 🙏 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
The
/modelcommand fails to switch to providers usingapi_mode: anthropic_messagesand to some OpenAI-compatible endpoints behind Cloudflare, because
validate_requested_model()always probes with
Authorization: Bearer(OpenAI-style auth) viaurllib.requestwithout a
User-Agentheader.Root causes
Anthropic Models API uses different auth: Anthropic's
GET /v1/modelsrequires
x-api-key+anthropic-versionheaders, notAuthorization: Bearer.The response shape (
data[].id) is identical to OpenAI, but the probe alwaysfailed with auth errors.
Cloudflare blocks requests without User-Agent:
urllib.requestdoes notset a
User-Agentheader by default. Some third-party OpenAI-compatibleendpoints behind Cloudflare return 403, causing the probe to fail even though
the endpoint works fine with normal HTTP clients.
Hard rejection on probe failure: When the
/modelsprobe failed foranthropic_messagesproviders,validate_requested_model()returnedaccepted: False, blocking the model switch entirely. Many Anthropic-compatibleproxies do not implement the Models API at all, so this was a permanent block.
Changes
hermes_cli/models.pyprobe_api_models(): Addedapi_modeparameter. Whenapi_mode == "anthropic_messages", sendsx-api-keyandanthropic-versionheadersinstead of
Authorization: Bearer. Always setsUser-Agent: Hermes/1.0.fetch_api_models(): Propagatesapi_modetoprobe_api_models().validate_requested_model(): Addedapi_modeparameter.anthropic_messagesmode: attempts probe with correct Anthropic auth.If the proxy implements the Models API (like the official Anthropic endpoint),
the model is verified normally. If the probe fails (common for third-party
proxies), the model is still accepted with an informational warning
instead of being rejected.
"custom"branch: same logic — tries Anthropic auth first,falls back to accept on failure.
hermes_cli/model_switch.pyswitch_model(): Passesapi_modethrough tovalidate_requested_model().How to test
api_mode: anthropic_messagesin~/.hermes/config.yamlundercustom_providers./model <model-name>with that provider — it should accept and switch.endpoint — the
/modelprobe should now succeed.pytest tests/hermes_cli/ -vpasses (614 passed; 2pre-existing failures unrelated to this change).
Platforms tested