Fix custom provider model validation when /v1/models endpoint is unavailable#12157
Open
happy5318 wants to merge 1 commit into
Open
Fix custom provider model validation when /v1/models endpoint is unavailable#12157happy5318 wants to merge 1 commit into
happy5318 wants to merge 1 commit into
Conversation
…ailable - Match 'custom:*' provider format (not just 'custom') - Fallback to config's models list when API endpoint fails Some OpenAI-compatible providers have broken /v1/models endpoints but fully functional chat APIs. Users who define models in config should not be blocked by a broken discovery endpoint. Fixes NousResearch#12153
Collaborator
|
Hey, @alt-glitch. Is there any reason why this MR can't be merged? This bug is preventing me from using hermes. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
custom:*provider format (not justcustom)Problem
When using a custom OpenAI-compatible provider whose
/v1/modelsendpoint returns errors (e.g., HTTP 500 or malformed response), Hermes fails to validate and switch to models defined incustom_providersconfig, even though:modelsfield/v1/modelsendpoint is brokenSolution
Changed the condition from
normalized == "custom"tonormalized == "custom" or normalized.startswith("custom:")to match all custom provider formatsAdded fallback logic: when
/v1/modelsis unavailable, validate against the user-defined models list incustom_providersconfigTesting
Tested with a custom provider that has a broken
/v1/modelsendpoint - model switching now works correctly.Fixes #12153
Checklist