fix(model-switcher): custom endpoint persistence, pre-fill, and silent validation failures#3264
Open
Mibayy wants to merge 1 commit into
Open
Conversation
…t failures Closes NousResearch#3263 Three distinct fixes for custom endpoint reliability: 1. Surface connection errors in model_switch.py (silent validation failure) The except block in Step 5 was returning accepted:True for all exceptions, including connection refused and timeouts on local endpoints. This reported a successful switch to an unreachable Ollama/LM Studio server. Fix: detect connection-class errors (refused, timeout, 401, 403, 404) on custom/localhost targets and return success:False with the actual error message. Cloud providers keep the previous lenient behaviour so temporary API outages don't block switching. 2. Pre-fill base_url and model from config.yaml in _model_flow_custom The interactive custom endpoint setup flow read OPENAI_BASE_URL from env but ignored model.base_url and model.default in config.yaml, presenting empty prompts even after a successful previous setup. Fix: load config.yaml first, fall back to env vars. Show existing values as bracket hints, and apply them as defaults when the user presses Enter without typing a new value (empty input = keep current). Also displays the current model name in the summary header. 3. Config.yaml custom endpoint survives OPENROUTER_API_KEY in .env When requested_norm was 'auto' and cfg_provider was 'custom', the use_config_base_url flag was not set, so the base_url resolution chain fell through to OPENROUTER_BASE_URL and flipped the active provider back to OpenRouter on CLI restart. Fix: add elif cfg_provider == 'custom' branch inside the 'auto' block so that an explicitly saved custom endpoint is always honoured regardless of which env vars are present. 8 new tests, 44 total pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #3263
Three independent fixes for the custom endpoint experience with Ollama, LM Studio, and other local/custom OpenAI-compatible servers.
Fix 1 — Surface connection errors (silent validation failure)
File:
hermes_cli/model_switch.pyThe
except Exceptionblock in Step 5 was swallowing all validation errors and returningaccepted: True, so switching to an unreachablehttp://localhost:11434/v1would report "Switched successfully" even when Ollama wasn't running.Before:
After:
The fix detects connection-class errors (refused, timeout, 401, 403, 404) specifically on custom/localhost targets. Cloud providers keep the previous lenient behaviour — a temporary OpenRouter outage should not block switching to a model that's likely fine.
Fix 2 — Pre-fill base_url and model from config.yaml
File:
hermes_cli/main.py→_model_flow_custom()The interactive setup flow read
OPENAI_BASE_URLfrom env but ignoredmodel.base_urlandmodel.defaultinconfig.yaml, presenting blank prompts every time even after a successful previous setup.Before:
After:
Empty input → keeps the current value. The config is read from
config.yamlfirst, with env vars as fallback.Fix 3 — config.yaml custom endpoint survives
OPENROUTER_API_KEYin.envFile:
hermes_cli/runtime_provider.pyWhen
requested_normwas"auto"(normal CLI startup) andcfg_providerwas"custom",use_config_base_urlwas not set. Thebase_urlresolution chain then fell through toOPENROUTER_BASE_URL, flipping the active provider back to OpenRouter on every restart.The existing
#1760fix only handledrequested_norm == "custom"explicitly, missing the"auto"path used at startup.Fix: added
elif cfg_provider == "custom"inside the"auto"block so that an explicitly saved custom endpoint always wins, regardless of what env vars are present.Tests
8 new tests in
tests/test_model_switcher_custom_3263.py:test_connection_refused_surfaced_for_localhosttest_timeout_surfaced_for_custom_endpointtest_404_surfaced_for_custom_endpointtest_cloud_provider_validation_error_still_acceptstest_model_flow_custom_prefills_from_configtest_model_flow_custom_uses_config_base_url_when_empty_inputtest_custom_config_wins_over_openrouter_keytest_openrouter_key_still_works_without_custom_config44 total pass (36 existing + 8 new).