Skip to content

fix(doctor): accept providers whose catalog id is None#16086

Open
stevenchanin wants to merge 1 commit into
NousResearch:mainfrom
stevenchanin:fix/opencode-zen-doctor-false-positive
Open

fix(doctor): accept providers whose catalog id is None#16086
stevenchanin wants to merge 1 commit into
NousResearch:mainfrom
stevenchanin:fix/opencode-zen-doctor-false-positive

Conversation

@stevenchanin

@stevenchanin stevenchanin commented Apr 26, 2026

Copy link
Copy Markdown
Contributor

What & Why

#17135 (merged) fixed hermes doctor for providers whose Hermes runtime id differs from the models.dev catalog id: ai-gateway, copilot, kilocode, kimi-coding, kimi-coding-cn, opencode-zen. It did not cover local-endpoint aliases — ollama, vllm, llamacpp — which still produce a self-contradicting "unknown provider" error today.

Reproduction on current main (post #17135):

# ~/.hermes/config.yaml
model:
  provider: ollama
  default: dummy-model
  base_url: http://localhost:11434/v1
✗ model.provider 'ollama' is not a recognised provider
  (known: ai-gateway, alibaba, ..., custom, ..., zai)

ollama is not in the printed "known" list, but it's a documented user-facing value — auth.resolve_provider routes it to custom, which is in the list.

Closes #16076 (copilot, fixed by #17135). Closes the still-open part of #16085. Supersedes the earlier scope of this PR; #15159 / #15361 / #15583 / #15778 stay closed.

Root cause (what #17135 missed)

Three providers — ollama, vllm, llamacpp — sit in auth.py:_PROVIDER_ALIASES mapping to "custom", but have no entry in providers.py. After #17135, the validator built up provider_ids_to_accept = {raw, runtime_resolved, catalog_resolved} correctly (including "custom" for these three), but the failure condition still started with:

if catalog_provider is None or (
    known_providers and not (provider_ids_to_accept & valid_provider_ids)
):

resolve_provider_full("ollama") returns None, so catalog_provider is None, and the short-circuit rejects the provider before the intersection check ever runs.

Input auth.resolve_provider(...) resolve_provider_full(...).id Status before this PR
ollama custom None rejected ❌
vllm custom None rejected ❌
llamacpp custom None rejected ❌
bedrock bedrock bedrock (HERMES_OVERLAYS entry now exists) accepted ✅
lmstudio lmstudio (now in PROVIDER_REGISTRY) lmstudio accepted ✅

bedrock and lmstudio were on the original #16085 list but were fixed independently — bedrock by adding a HERMES_OVERLAYS entry, lmstudio by promoting it to PROVIDER_REGISTRY (214ca94, feat(agent): add lmstudio integration). Both are kept in the parametrized test as regression guards.

Change

One behaviour change in hermes_cli/doctor.py: drop the catalog_provider is None or ... short-circuit. The intersection check below already covers the cases that matter — a None catalog id is not a sufficient signal that the provider is unknown when the auth-resolved runtime id lands in PROVIDER_REGISTRY.

Tests

tests/hermes_cli/test_doctor.py adds:

  • test_run_doctor_accepts_provider_when_catalog_resolution_returns_none — parametrized over ollama, vllm, llamacpp (the cases fix(doctor): accept catalog provider aliases #17135 didn't fix) plus bedrock and lmstudio as regression guards.
  • test_run_doctor_validator_invariant_holds_for_every_known_provider — invariant test asserting every name in PROVIDER_REGISTRY (i.e. every name doctor prints as valid) also passes its own validator. Catches forward regressions like the original bedrock shape if a future provider lands in PROVIDER_REGISTRY without a matching catalog or alias entry.
  • test_run_doctor_still_flags_unknown_provider — negative case: provider: completely-made-up is still flagged.
$ scripts/run_tests.sh tests/hermes_cli/test_doctor.py
35 passed in 1.4s

Wider sweep on tests/hermes_cli/: 3471 passed, 9 unrelated failures (test_gateway_service, test_gateway_wsl, test_setup_openclaw_migration, test_tools_config) that reproduce on a clean main without these changes.

Platforms tested

  • macOS 14 (Apple Silicon), Python 3.11.15

Scope

One commit, one file plus tests. Six-line diff in doctor.py (drop one short-circuit + a clarifying comment). No behaviour change for providers that already passed the validator.

NousResearch#17135 fixed doctor for providers whose Hermes runtime id differs from
the models.dev catalog id (ai-gateway, copilot, kilocode, kimi-coding,
kimi-coding-cn, opencode-zen). It missed local-endpoint aliases —
ollama, vllm, llamacpp — which auth.resolve_provider routes to "custom"
but resolve_provider_full has no entry for, so catalog_provider is None.

The validator's failure check started with `if catalog_provider is None
or ...`, which short-circuited and rejected those providers even though
their auth-resolved runtime id ("custom") was already in
provider_ids_to_accept and known_providers.

Drop the short-circuit; rely on the intersection check below. Genuinely
unknown providers (e.g. "completely-made-up") still fail because the
intersection is empty.

Tests: parametrized regression test covering ollama / vllm / llamacpp
(the cases NousResearch#17135 didn't address) plus bedrock and lmstudio (regression
guards — those used to be on the list in NousResearch#16085 but were fixed by
adding a HERMES_OVERLAYS / PROVIDER_REGISTRY entry); an invariant test
asserting every name doctor prints in its "Valid providers" list passes
its own validator; and a negative test asserting a fake provider name
is still rejected.
@stevenchanin stevenchanin force-pushed the fix/opencode-zen-doctor-false-positive branch from 949383c to 4c45156 Compare April 30, 2026 22:20
@stevenchanin stevenchanin changed the title fix(doctor): accept providers whose canonical id differs from PROVIDER_REGISTRY id fix(doctor): accept providers whose catalog id is None Apr 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/cli CLI entry point, hermes_cli/, setup wizard P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug(doctor): reports 'copilot' as unknown provider while listing it as valid

3 participants