Skip to content

fix(model-switch): resolve bare model names via custom_providers catalog#13764

Closed
rzbdz wants to merge 1 commit into
NousResearch:mainfrom
rzbdz:fix/custom-provider-model-switch-v2
Closed

fix(model-switch): resolve bare model names via custom_providers catalog#13764
rzbdz wants to merge 1 commit into
NousResearch:mainfrom
rzbdz:fix/custom-provider-model-switch-v2

Conversation

@rzbdz

@rzbdz rzbdz commented Apr 22, 2026

Copy link
Copy Markdown

Problem

When a user types /model claude-sonnet-4.6 while on a custom provider (e.g. smt-claude), the switch fails with:

✗ Note: `claude-sonnet-4.6` was not found in this custom endpoint's model listing (https://openrouter.ai/api/v1/models).
  Similar models: `anthropic/claude-sonnet-4.6`, ...

Three root causes:

  1. cli.py: _handle_model_switch only loaded custom_providers in the no-args picker path. When a model name was given, switch_model was called with custom_providers=None.

  2. model_switch.py: No step searched the user's custom_providers models catalog before falling through to a live API probe, which could hit the wrong URL.

  3. model_switch.py / models.py: custom:smt-claude slugs weren't recognized as custom in the is_custom guard or in validate_requested_model.

Fix

  • New _find_model_in_custom_providers() helper scans every custom_providers entry's models dict/list. Matching is case-insensitive and normalises dots↔hyphens (claude-sonnet-4.6 matches claude-sonnet-4-6).
  • Inserted as step c2 in PATH B of switch_model() — before any live API probe.
  • cli.py loads custom_provs unconditionally before calling switch_model().
  • is_custom guard extended to cover custom:* slugs.
  • validate_requested_model normalises custom:*"custom".

Tests

5 new tests in tests/hermes_cli/test_model_switch_custom_providers.py. All 2440 existing tests pass.

When a user types /model <name> while on a custom provider, the pipeline
previously fell through to a live API probe against the endpoint's /models
listing. If the endpoint returned model IDs in a different format (or the
probe hit OpenRouter's URL due to stale state), validation failed and the
switch was rejected — leaving the user stuck on the default model.

Three fixes:

1. model_switch.py: Add _find_model_in_custom_providers() helper that
   scans every custom_providers entry's models dict/list for a matching
   model name (case-insensitive, dots-vs-hyphens normalised). Insert it
   as step c2 in PATH B so bare names like 'claude-sonnet-4.6' resolve
   to the correct custom:smt-claude provider before hitting live probing.

2. model_switch.py: Extend the is_custom guard (step e) to cover
   custom:* slugs, preventing detect_provider_for_model() from
   incorrectly routing custom provider models to OpenRouter/Anthropic.

3. cli.py: Load custom_provs before calling switch_model() in
   _handle_model_switch(), not just in the picker path. Previously the
   switch call always received custom_providers=None, so step c2 never
   had data to search.

4. models.py: Treat custom:* slugs identically to 'custom' in
   validate_requested_model() so named custom providers probe their own
   endpoint rather than falling through to the generic error path.
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/cli CLI entry point, hermes_cli/, setup wizard labels Apr 22, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Note: #13760 was a prior closed attempt at the same fix; this is the replacement PR.

1 similar comment
@alt-glitch

Copy link
Copy Markdown
Collaborator

Note: #13760 was a prior closed attempt at the same fix; this is the replacement PR.

@trevorgordon981

Copy link
Copy Markdown

I have tested this solution locally. The model-switch logic now correctly resolves bare model names via the custom_providers catalog. This ensures that users can switch to custom providers using simple model names without needing fully qualified strings. The fix is stable and ready for merge.\n\nTested and confirmed. ✅

1 similar comment
@trevorgordon981

Copy link
Copy Markdown

I have tested this solution locally. The model-switch logic now correctly resolves bare model names via the custom_providers catalog. This ensures that users can switch to custom providers using simple model names without needing fully qualified strings. The fix is stable and ready for merge.\n\nTested and confirmed. ✅

@zons-zhaozhy

Copy link
Copy Markdown

This fix covers custom_providers path, but providers: dict has same problem.

Reproduce:

# config.yaml
model:
  default: sensenova-6.7-flash-lite
  provider: sensenova
providers:
  sensenova:
    base_url: https://token.sensenova.cn/v1
    api_key: sk-xxx

Run /model sensenova-6.7-flash-lite under sensenova provider, model name got sensenova/ prefix added, provider silently switched to OpenRouter.

Root cause is same — detect_provider_for_model() matched model in OpenRouter catalog, but is_custom guard not cover non-built-in provider defined in providers: dict.

I did a general check using get_provider(current_provider) is None, both paths covered: zons-zhaozhy@917afabc6

Tests: zons-zhaozhy@c64056e30

@teknium1

Copy link
Copy Markdown
Contributor

Closing as superseded by #28908.

Triage notes (medium confidence):
Merged PR #28908 'fix(model): match bare custom provider by active base URL' (plus #17891, #18185) added custom_providers resolution before live API probe in hermes_cli/model_switch.py (see custom_providers and isinstance(custom_providers, list) block ~line 948).

Thanks for the contribution — the underlying problem this PR addresses has been resolved by the linked PR on current main. If you believe this was closed in error, please comment and we'll reopen.

(Bulk-closed during a CLI PR triage sweep.)

@teknium1 teknium1 closed this May 24, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants