Bug Description
The /model command displays providers and model lists that are inconsistent with the user's config.yaml configuration. Specifically:
- Providers without
allowed_models in config.yaml still appear, using hardcoded model lists from _PROVIDER_MODELS or OPENROUTER_MODELS in models.py.
- Model lists for configured providers may differ between what
/model shows and what is actually defined in config.yaml, because multiple code paths (_PROVIDER_MODELS, provider_model_ids(), OPENROUTER_MODELS) each maintain their own model lists independently of config.yaml.
Steps to Reproduce
- Set
OPENROUTER_API_KEY in ~/.hermes/.env (or any provider's API key).
- Do NOT add an
openrouter entry to config.yaml's providers: section.
- Run
/model in the CLI or gateway.
Expected: OpenRouter should not appear in the provider list (since there is no config.yaml entry).
Actual: OpenRouter appears with 3 models from the hardcoded OPENROUTER_MODELS list in models.py:
OPENROUTER_MODELS = [
("anthropic/claude-opus-4.6", "recommended"),
("anthropic/claude-sonnet-4.6", ""),
("openai/gpt-5-mini", ""),
]
Root Cause
In model_switch.py, list_authenticated_providers() Step 1 (line 764):
for hermes_id, mdev_id in PROVIDER_TO_MODELS_DEV.items():
# ...
pconfig = PROVIDER_REGISTRY.get(hermes_id)
if pconfig and pconfig.api_key_env_vars:
env_vars = list(pconfig.api_key_env_vars)
else:
env_vars = pdata.get("env", []) # falls back to models.dev
has_creds = any(os.environ.get(ev) for ev in env_vars)
if not has_creds:
continue
model_ids = curated.get(hermes_id, []) # falls back to OPENROUTER_MODELS
Two issues here:
Issue A — Unconfigured providers appear when they should not
PROVIDER_REGISTRY does not contain "openrouter", so pconfig = None.
- Falls back to
pdata.get("env", []) → ["OPENROUTER_API_KEY"] from models.dev.
- If
OPENROUTER_API_KEY is set, has_creds = True, and the provider appears.
- The
curated dict is built from _PROVIDER_MODELS / OPENROUTER_MODELS first, then overridden by config.yaml allowed_models. But since there is no openrouter entry in config, the hardcoded list is used.
- No mechanism to hide a provider — if credentials exist, it shows up regardless of config.
Issue B — Multiple data sources for model lists
The model list shown in /model comes from curated, which is correct when config.yaml has allowed_models. But other functions like provider_model_ids() in models.py read from _PROVIDER_MODELS (hardcoded) or live API directly, not from config.yaml. This means:
/model display → reads config.yaml (correct)
- Model validation/switching → reads
_PROVIDER_MODELS (may be stale/inconsistent)
Additionally, _PROVIDER_MODELS has model IDs like "gpt-5.4-mini" while config.yaml uses "gpt-5-mini" — these may not match.
Impact
- Users see providers they did not configure
- Model lists shown may not reflect what is actually available or configured
- Confusing UX — config.yaml is not the single source of truth
Suggested Fix
- In
list_authenticated_providers() Step 1: If a provider exists in PROVIDER_TO_MODELS_DEV but has no entry in config.yaml providers:, skip it (or require explicit opt-in).
- Unify model list sources: Either read from config.yaml everywhere, or ensure
_PROVIDER_MODELS is always in sync with config.yaml. Consider making config.yaml the single source of truth for allowed models.
- Normalize model IDs: Ensure consistency between
_PROVIDER_MODELS model IDs and what users put in config.yaml (e.g., gpt-5.4-mini vs gpt-5-mini).
Environment
- hermes-agent: latest (NousResearch/hermes-agent)
- OS: macOS
- Config:
~/.hermes/config.yaml has providers for nous, copilot, deepseek, minimax-cn — but NOT openrouter
- Env:
OPENROUTER_API_KEY is set in ~/.hermes/.env
Bug Description
The
/modelcommand displays providers and model lists that are inconsistent with the user'sconfig.yamlconfiguration. Specifically:allowed_modelsin config.yaml still appear, using hardcoded model lists from_PROVIDER_MODELSorOPENROUTER_MODELSinmodels.py./modelshows and what is actually defined inconfig.yaml, because multiple code paths (_PROVIDER_MODELS,provider_model_ids(),OPENROUTER_MODELS) each maintain their own model lists independently ofconfig.yaml.Steps to Reproduce
OPENROUTER_API_KEYin~/.hermes/.env(or any provider's API key).openrouterentry toconfig.yaml'sproviders:section./modelin the CLI or gateway.Expected: OpenRouter should not appear in the provider list (since there is no config.yaml entry).
Actual: OpenRouter appears with 3 models from the hardcoded
OPENROUTER_MODELSlist inmodels.py:Root Cause
In
model_switch.py,list_authenticated_providers()Step 1 (line 764):Two issues here:
Issue A — Unconfigured providers appear when they should not
PROVIDER_REGISTRYdoes not contain"openrouter", sopconfig = None.pdata.get("env", [])→["OPENROUTER_API_KEY"]from models.dev.OPENROUTER_API_KEYis set,has_creds = True, and the provider appears.curateddict is built from_PROVIDER_MODELS/OPENROUTER_MODELSfirst, then overridden byconfig.yaml allowed_models. But since there is noopenrouterentry in config, the hardcoded list is used.Issue B — Multiple data sources for model lists
The model list shown in
/modelcomes fromcurated, which is correct when config.yaml hasallowed_models. But other functions likeprovider_model_ids()inmodels.pyread from_PROVIDER_MODELS(hardcoded) or live API directly, not from config.yaml. This means:/modeldisplay → reads config.yaml (correct)_PROVIDER_MODELS(may be stale/inconsistent)Additionally,
_PROVIDER_MODELShas model IDs like"gpt-5.4-mini"while config.yaml uses"gpt-5-mini"— these may not match.Impact
Suggested Fix
list_authenticated_providers()Step 1: If a provider exists inPROVIDER_TO_MODELS_DEVbut has no entry in config.yamlproviders:, skip it (or require explicit opt-in)._PROVIDER_MODELSis always in sync with config.yaml. Consider making config.yaml the single source of truth for allowed models._PROVIDER_MODELSmodel IDs and what users put in config.yaml (e.g.,gpt-5.4-minivsgpt-5-mini).Environment
~/.hermes/config.yamlhas providers for nous, copilot, deepseek, minimax-cn — but NOT openrouterOPENROUTER_API_KEYis set in~/.hermes/.env