Problem
When using a custom OpenAI-compatible provider whose /v1/models endpoint returns errors (e.g., HTTP 500 or malformed response), Hermes fails to validate and switch to models defined in custom_providers config, even though:
- The model IS listed in the config's
models field
- The chat completions API works perfectly fine
- Only the
/v1/models endpoint is broken
Root Cause
Two issues in hermes_cli/models.py:
Issue 1: Provider format mismatch
The condition if normalized == "custom" doesn't match custom:provider-name format:
# Current code (line ~2013)
if normalized == "custom": # Only matches "custom", not "custom:some_provider"
Issue 2: No fallback to config models list
When /v1/models is unavailable, Hermes returns failure immediately instead of checking the user-defined models list in config:
# Current behavior: returns failure when api_models is None
# Expected: fallback to custom_providers[].models for validation
Proposed Fix
- Match all
custom:* formats:
if normalized == "custom" or normalized.startswith("custom:"):
- Fallback to config models list when API is unavailable:
if api_models is None:
# Try to validate from config's models list
from hermes_cli.config import get_compatible_custom_providers
custom_providers = get_compatible_custom_providers()
provider_name = provider.replace("custom:", "")
for cp in custom_providers:
if cp.get("name") == provider_name:
config_models = cp.get("models", {})
if requested in config_models:
return {"accepted": True, ...}
Why This Matters
Some OpenAI-compatible providers (especially regional/domestic cloud providers) have broken /v1/models endpoints but fully functional chat APIs. Users who have already defined their models in config should not be blocked by a broken discovery endpoint.
Workaround
Currently requires patching hermes_cli/models.py locally.
Environment
- Hermes version: latest
- Python: 3.11+
Problem
When using a custom OpenAI-compatible provider whose
/v1/modelsendpoint returns errors (e.g., HTTP 500 or malformed response), Hermes fails to validate and switch to models defined incustom_providersconfig, even though:modelsfield/v1/modelsendpoint is brokenRoot Cause
Two issues in
hermes_cli/models.py:Issue 1: Provider format mismatch
The condition
if normalized == "custom"doesn't matchcustom:provider-nameformat:Issue 2: No fallback to config models list
When
/v1/modelsis unavailable, Hermes returns failure immediately instead of checking the user-defined models list in config:Proposed Fix
custom:*formats:Why This Matters
Some OpenAI-compatible providers (especially regional/domestic cloud providers) have broken
/v1/modelsendpoints but fully functional chat APIs. Users who have already defined their models in config should not be blocked by a broken discovery endpoint.Workaround
Currently requires patching
hermes_cli/models.pylocally.Environment