Problem or Use Case
Currently, the /model picker relies on a hardcoded curated list (hermes_cli/models.py) for built-in providers (e.g., NVIDIA NIM、Gemini/Google AI Studio/Openrouter). This creates a significant limitation for users who wish to use models offered by the provider that are not included in the static curated list.
The current architecture makes it nearly impossible to extend built-in providers via config.yaml without compromising UX:
- Using providers: (Keyed schema): The entry is silently ignored because the linear loading logic in list_authenticated_providers identifies the slug as already "seen" (due to the built-in provider), causing the user's custom model list to be skipped entirely.
- Using custom_providers: (List schema): Due to the custom: prefix added by custom_provider_slug, the deduplication check is bypassed. This results in duplicate entries in the /model picker (e.g., one "NVIDIA" with curated models and one "Nvidia" with user models), which is confusing and cluttered.
As seen in the context of Issue #9545 and PR #10599, the current "fix" for deduplication only addresses the symptoms (preventing triple entries) but does not solve the fundamental problem: built-in providers are immutable and cannot be extended by the user via configuration.
Proposed Solution
Implement a dynamic model discovery and filtering mechanism for built-in providers.
Instead of relying solely on a hardcoded list, the agent should:
- Dynamic Fetching: Upon successful API key authentication, the agent should optionally query the provider's /v1/models endpoint to retrieve the real-time list of available models.
- User-Defined Filtering: Allow users to define a filter (e.g., regex or a list of keywords) in config.yaml for a built-in provider.
- Merged Model List: The /model picker should merge the curated list with the models that pass the user's filter from the live API response.
Example config entry for NVIDIA:
providers:
nvidia:
filter_models: ["nemotron-4-reasoning", "llama-3.3"] # Only these models from live API will be added to the picker
This approach would allow users to seamlessly integrate new models into the same provider category in the la /model picker, maintaining a clean UI and providing flexibility.
Alternatives Considered
• Adding models manually via custom_providers:: This leads to the "Duplicate Provider" problem analyzed in PR #10599, requiring users to give the provider an alias (e.g., nvidia-custom), which is a poor UX.
• Hardcoding more models in models.py: This is unsustainable for maintainers and cannot keep up with the rapid release cycle of new models.
• Updating list_authenticated_providers to merge results: While possible, it would require a significant refactor of the logic to allow "upward" modification of results that have already been appended to the list.
The proposed solution is better because it shifts the focus from static curated lists to dynamic discovery, which is more aligned with how modern LLM APIs are designed.
Feature Type
Configuration option
Scope
None
Contribution
Debug Report (optional)
Problem or Use Case
Currently, the /model picker relies on a hardcoded curated list (hermes_cli/models.py) for built-in providers (e.g., NVIDIA NIM、Gemini/Google AI Studio/Openrouter). This creates a significant limitation for users who wish to use models offered by the provider that are not included in the static curated list.
The current architecture makes it nearly impossible to extend built-in providers via config.yaml without compromising UX:
As seen in the context of Issue #9545 and PR #10599, the current "fix" for deduplication only addresses the symptoms (preventing triple entries) but does not solve the fundamental problem: built-in providers are immutable and cannot be extended by the user via configuration.
Proposed Solution
Implement a dynamic model discovery and filtering mechanism for built-in providers.
Instead of relying solely on a hardcoded list, the agent should:
Example config entry for NVIDIA:
This approach would allow users to seamlessly integrate new models into the same provider category in the la /model picker, maintaining a clean UI and providing flexibility.
Alternatives Considered
• Adding models manually via custom_providers:: This leads to the "Duplicate Provider" problem analyzed in PR #10599, requiring users to give the provider an alias (e.g., nvidia-custom), which is a poor UX.
• Hardcoding more models in models.py: This is unsustainable for maintainers and cannot keep up with the rapid release cycle of new models.
• Updating list_authenticated_providers to merge results: While possible, it would require a significant refactor of the logic to allow "upward" modification of results that have already been appended to the list.
The proposed solution is better because it shifts the focus from static curated lists to dynamic discovery, which is more aligned with how modern LLM APIs are designed.
Feature Type
Configuration option
Scope
None
Contribution
Debug Report (optional)