Skip to content

fix(cli): show custom providers in /provider and support auto-matching#2580

Closed
francip wants to merge 2 commits into
NousResearch:mainfrom
kortexa-ai:kortexa/coustom-providers
Closed

fix(cli): show custom providers in /provider and support auto-matching#2580
francip wants to merge 2 commits into
NousResearch:mainfrom
kortexa-ai:kortexa/coustom-providers

Conversation

@francip

@francip francip commented Mar 23, 2026

Copy link
Copy Markdown
Contributor

Note: This PR was authored and tested by Avery — a Hermes agent, with guidance from @francip.


What Changed

Custom providers defined in config.yaml custom_providers were not displayed in /model output, and there was no way to switch to them via /model syntax. Additionally, the generic [custom] entry was falsely showing as authenticated using the active provider's base_url. The gateway /provider command had the same gap — custom providers appeared as flat lines with no model listing.

Files: cli.py, hermes_cli/models.py, hermes_cli/runtime_provider.py, gateway/run.py, tests/test_cli_model_command.py, tests/test_runtime_provider_resolution.py

Changes

  • Named custom providers now appear in /model output with their configured model names
  • Named custom providers now appear in /provider (gateway) output with their model names
  • _get_custom_base_url() no longer leaks non-custom provider URLs into the generic [custom] display
  • Named custom providers now report their actual provider name instead of hardcoded "openrouter"
  • Smart auto-matching for /model input:
    • /model sparky-qwen-35b — bare provider name auto-selects sole model
    • /model sparky-qwen-35b:model-name — explicit provider:model selection
    • /model model-name — finds which custom provider owns the model
    • Refuses ambiguous matches when a model exists in multiple custom providers
  • Fixed pre-existing test expecting OpenRouter slug remap for a model already in the current provider's catalog

Why

Users who configure custom providers (local vLLM servers, custom endpoints) had no way to see or switch to them from /model or /provider. The only path was the interactive setup menu in hermes setup. The generic [custom] entry also falsely appeared as authenticated by reading the active provider's (e.g. zai) base_url.

Screenshots

Before/model only shows generic [custom] with wrong endpoint:

Screenshot 2026-03-22 at 18 39 53 Screenshot 2026-03-22 at 18 40 20

After/model shows each named custom provider with its models:

Screenshot 2026-03-22 at 22 45 22 Screenshot 2026-03-22 at 22 38 21

How to Test

Add custom providers to ~/.hermes/config.yaml:

custom_providers:
  - name: my-local-server
    base_url: http://localhost:8000/v1
    models:
      my-model:
        context_length: 32768

Then test each switching method:

/model                          # Should list [custom:my-local-server] with my-model
/model my-local-server          # Should auto-select my-model (sole model)
/model my-local-server:my-model # Should select explicitly
/model my-model                 # Should auto-detect the owning custom provider
/provider                       # Should list custom providers with their models

Validation

Automated:

source venv/bin/activate
python -m pytest tests/test_cli_model_command.py tests/test_runtime_provider_resolution.py tests/hermes_cli/test_models.py -v

Result: 60 passed (1 pre-existing flaky xdist test intermittently fails on main too)

Manual:

  • Verified custom providers display correctly in /model
  • Verified custom providers display correctly in /provider (gateway)
  • Verified all three switching syntaxes work end-to-end
  • Verified the generic [custom] no longer shows non-custom provider URLs
  • Confirmed provider field shows correct name instead of "openrouter"

Platforms Tested

✅ macOS CLI

Backward Compatibility

✅ Backward compatible

  • Generic [custom] endpoint still works for users who configure it directly
  • Built-in provider matching and OpenRouter slug detection unchanged
  • Custom provider model matching only activates when custom_providers is configured

@francip francip force-pushed the kortexa/coustom-providers branch 3 times, most recently from c1a32ea to 8e0f729 Compare March 30, 2026 06:46
@francip francip changed the title fix(cli): show custom providers in /model and support auto-matching fix(cli): show custom providers in /provider and support auto-matching Mar 30, 2026
@francip francip force-pushed the kortexa/coustom-providers branch 7 times, most recently from 8bd0529 to 4edb708 Compare April 9, 2026 00:18
@francip francip force-pushed the kortexa/coustom-providers branch 2 times, most recently from e1c396b to 831083b Compare April 15, 2026 06:45
@francip francip force-pushed the kortexa/coustom-providers branch 4 times, most recently from 5fbcf9b to a87d730 Compare April 21, 2026 17:26
@alt-glitch alt-glitch added type/feature New feature or request comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery labels Apr 21, 2026
@francip francip force-pushed the kortexa/coustom-providers branch from a87d730 to 2386496 Compare April 22, 2026 03:20
francip and others added 2 commits April 22, 2026 19:02
…atching

Custom providers defined in config.yaml custom_providers were not displayed
in /model output, and there was no way to switch to them via /model syntax.

- Display named custom providers with their configured models in /model
- Support /model provider-name (auto-selects sole model)
- Support /model provider-name:model-name (explicit selection)
- Support /model model-name (auto-detects owning custom provider)
- Refuse ambiguous matches when model exists in multiple custom providers
- Fix _get_custom_base_url() leaking non-custom provider URLs into [custom]
- Fix named custom providers reporting provider as "openrouter" at runtime
- Fix pre-existing test expecting OpenRouter slug remap for catalog model
Port the custom provider display logic from cli.py to the gateway
/provider command so named custom providers list their models.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@francip francip force-pushed the kortexa/coustom-providers branch from 2386496 to f0831c8 Compare April 23, 2026 02:02
@francip

francip commented Apr 28, 2026

Copy link
Copy Markdown
Contributor Author

Closing — both commits target /model and /provider slash commands, which were removed in upstream commits 9783c9d (#3080) and b2e124d (#15047). Custom-provider display surface no longer exists in this form. If custom providers need the same fix in the new model/provider UX, will reopen as a fresh PR.

@francip francip closed this Apr 28, 2026
@francip francip deleted the kortexa/coustom-providers branch April 28, 2026 14:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard comp/gateway Gateway runner, session dispatch, delivery type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants