Skip to content

Custom providers in config.providers are misclassified as unknown by hermes model and doctor #14850

@linglichao

Description

@linglichao

Summary

Named user-defined providers configured under config.yaml -> providers: are correctly surfaced in picker/runtime flows, but some validation/entrypoint code paths still treat them as unknown because they call hermes_cli.auth.resolve_provider() (built-in providers only) instead of the user-aware provider resolution path.

This produces false warnings like:

Warning: Unknown provider 'volcengine-plan'. Check 'hermes model' for available providers, or run 'hermes doctor' to diagnose config issues. Falling back to auto provider detection.

and config diagnostics like model.provider points to a provider not found in config, even though the provider is present in config.yaml, appears in the model picker, and can fetch models successfully.

Environment

  • Reproduced on official main
  • Commit: 6fdbf2f2d76cf37393e657bf37ceda3d84589200
  • Remote: https://github.com/NousResearch/hermes-agent.git

Repro

Use a config like:

model:
  default: doubao-seed-2.0-code
  provider: volcengine-plan

providers:
  volcengine-plan:
    name: volcengine-plan
    base_url: https://ark.cn-beijing.volces.com/api/coding/v3
    api_key: ${VOLCENGINE_API_KEY}
    default_model: doubao-seed-2.0-code
    models:
      doubao-seed-2.0-code: {}

Then run:

hermes model

Actual

  1. hermes model first prints:
Warning: Unknown provider 'volcengine-plan'. Check 'hermes model' for available providers, or run 'hermes doctor' to diagnose config issues. Falling back to auto provider detection.
  1. The picker then still shows the provider correctly, for example:
29. volcengine-plan (ark.cn-beijing.volces.com/api/coding/v3) — doubao-seed-2.0-code
  1. Selecting that provider succeeds and fetches models from the endpoint.

  2. hermes doctor / Web UI diagnostics also misclassify model.provider: volcengine-plan as unknown.

Expected

If a provider exists in config.yaml -> providers: (or the compatibility view from get_compatible_custom_providers()), entrypoint validation and diagnostics should accept it consistently. No false warning should be emitted before opening the picker, and doctor should not report the provider as unknown.

Root cause

There are currently two different provider-resolution paths in play:

  • The shared switch/runtime flows correctly support user-defined providers via resolve_provider_full(...) and compatibility helpers.
  • But some validation/display entrypoints still call resolve_provider(...), which only knows built-in providers.

Affected call sites

  1. hermes_cli/main.py

select_provider_and_model() pre-validates the current provider with:

active = resolve_provider(effective_provider)

This is what emits the false Unknown provider warning before the picker opens.

  1. hermes_cli/doctor.py

Doctor validates model.provider with:

canonical_provider = _resolve_provider(provider)

This incorrectly reports user-defined providers: entries as unknown.

Suggested fix

Use the user-aware resolution path for these validation/entrypoint checks, e.g. resolve_provider_full(...) with:

  • cfg.get("providers")
  • get_compatible_custom_providers(cfg)

That should bring hermes model, doctor, and UI diagnostics into alignment with the already-correct picker/runtime behavior.

Notes

This is not a local divergence issue: the reproduction above was verified against official main at 6fdbf2f2d76cf37393e657bf37ceda3d84589200.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium — degraded but workaround existsarea/configConfig system, migrations, profilescomp/cliCLI entry point, hermes_cli/, setup wizardtype/bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions