fix(model): preserve custom endpoint credentials during /model switch#15826
Merged
Conversation
…els not in /v1/models When switching models on a custom endpoint (ollama-launch): - Same-provider switches no longer re-resolve credentials (fixes base_url being lost for 'custom' provider on subsequent switches) - Named providers (ollama-launch) are resolved via user_providers so switch_model can find their base_url from config - Models not in the /v1/models probe but present in the user's saved provider config are accepted with a warning instead of rejected - CLI /model and TUI /model both pass user_providers/custom_providers to switch_model so the config model list is available for validation Closes #15088
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Salvage of #15088 by @kshitijk4poor onto current main.
Summary
/model <name>on custom/local endpoints (ollama-launch, lmstudio, vllm, LAN servers, etc.) no longer silently probes OpenRouter and fails. Named custom providers keep their known-good credentials during switch, and cloud/aliased models absent from/v1/modelsare accepted instead of rejected.Changes
hermes_cli/model_switch.py: when re-resolution returns barecustom, preserve existingapi_key/base_url/api_modefrom the live agent. Add user_providers override for models absent from/v1/modelsbut present in saved provider config.hermes_cli/models.py: custom endpoints where/v1/modelsdoesn't list the requested model now returnaccepted: Truewith a note instead of rejecting — matches real-world server behavior where cloud/aliased models are supported but not enumerated (glm-5:cloud, :latest variants).tui_gateway/server.py: wiresuser_providers/custom_providersintoswitch_model()so TUI gets the same named-custom resolution as CLI.cli.py: pulluser_provs/custom_provsload out of the picker-only branch soswitch_model()always gets them.Root cause
Named custom providers (config
providers: { ollama-launch: { ... } }) resolve through_resolve_named_custom_runtime()returningprovider: "custom". Whenswitch_model()re-resolves credentials withrequested="custom",resolve_runtime_providercan't find the original endpoint and falls through to OpenRouter. Validation then probes OpenRouter's/v1/modelsinstead of the local server.Validation
test_saved_model_still_probes_endpoint) unrelated to this PR./v1/modelsnow returnsaccepted: True; credential preservation guard wired correctly inswitch_model().Credit: @kshitijk4poor (authorship preserved via cherry-pick). Closes #15088. Complements #15823 which fixed the
hermes doctorfalse-positive on the same report.