fix(model-switch): resolve bare model names via custom_providers catalog#13760
Closed
rzbdz wants to merge 2 commits into
Closed
fix(model-switch): resolve bare model names via custom_providers catalog#13760rzbdz wants to merge 2 commits into
rzbdz wants to merge 2 commits into
Conversation
Three related bugs caused custom_providers per-model context_length overrides to be ignored, particularly when switching models at runtime: 1. __init__: self._config_context_length was assigned *before* the custom_providers lookup, so per-model overrides were never stored on the agent instance. Moved the assignment after the lookup. 2. switch_model: re-resolve config_context_length from config.yaml for the new (model, base_url) pair instead of reusing the stale value from __init__. Checks both top-level model.context_length and custom_providers per-model overrides. 3. get_model_context_length: when a custom endpoint's /models response omits context_length, fall through to models.dev / OpenRouter / hardcoded defaults instead of returning 128K immediately. Proxy endpoints (LiteLLM, NewAPI, etc.) often omit context_length but serve well-known models whose metadata is available downstream. Signed-off-by: Troy Mitchell <i@troy-y.org>
When a user types /model <name> while on a custom provider, the pipeline previously fell through to a live API probe against the endpoint's /models listing. If the endpoint returned model IDs in a different format (or the probe hit OpenRouter's URL due to stale state), validation failed and the switch was rejected — leaving the user stuck on the default model. Three fixes: 1. model_switch.py: Add _find_model_in_custom_providers() helper that scans every custom_providers entry's models dict/list for a matching model name (case-insensitive, dots-vs-hyphens normalised). Insert it as step c2 in PATH B so bare names like 'claude-sonnet-4.6' resolve to the correct custom:smt-claude provider before hitting live probing. 2. model_switch.py: Extend the is_custom guard (step e) to cover custom:* slugs, preventing detect_provider_for_model() from incorrectly routing custom provider models to OpenRouter/Anthropic. 3. cli.py: Load custom_provs before calling switch_model() in _handle_model_switch(), not just in the picker path. Previously the switch call always received custom_providers=None, so step c2 never had data to search. 4. models.py: Treat custom:* slugs identically to 'custom' in validate_requested_model() so named custom providers probe their own endpoint rather than falling through to the generic error path.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When a user types
/model claude-sonnet-4.6while on a custom provider (e.g.smt-claude), the switch fails with:The session then silently falls back to the custom provider group's default model instead of switching. Three root causes:
cli.py:_handle_model_switchonly loadedcustom_providersin the picker path (no-args). When a model name was given,switch_modelwas called withcustom_providers=None.model_switch.py: No step searched the user'scustom_providersmodels catalog before falling through to a live API probe. The probe could hit the wrong URL (e.g. OpenRouter) if base_url state was stale.model_switch.py: Theis_customguard only matched"custom"and"local"exactly — notcustom:smt-claudeslugs — sodetect_provider_for_model()could incorrectly re-route the model to OpenRouter.models.py:validate_requested_modeldidn't treatcustom:*slugs as"custom", so named custom providers fell through to the generic error path.Fix
_find_model_in_custom_providers()— new helper that scans everycustom_providersentry'smodelsdict/list for a matching model name. Matching is case-insensitive and normalises dots↔hyphens (claude-sonnet-4.6matchesclaude-sonnet-4-6).switch_model()— runs before the live API probe, so configured models always win.is_customguard extended to covercustom:*slugs.cli.pyloadscustom_provsunconditionally before callingswitch_model().validate_requested_modelnormalisescustom:*→"custom".Testing
5 new tests in
tests/hermes_cli/test_model_switch_custom_providers.py:test_find_model_in_custom_providers_dict_formattest_find_model_in_custom_providers_singular_modeltest_find_model_in_custom_providers_no_matchtest_switch_model_bare_name_resolves_via_custom_providerstest_switch_model_bare_name_dot_hyphen_normalisationAll 2440 existing tests pass (2 pre-existing failures in
test_gateway_service.pyare unrelated).