fix(doctor): accept providers whose catalog id is None#16086
Open
stevenchanin wants to merge 1 commit into
Open
fix(doctor): accept providers whose catalog id is None#16086stevenchanin wants to merge 1 commit into
stevenchanin wants to merge 1 commit into
Conversation
This was referenced Apr 26, 2026
3 tasks
MitchTalmadge
approved these changes
Apr 26, 2026
19 tasks
NousResearch#17135 fixed doctor for providers whose Hermes runtime id differs from the models.dev catalog id (ai-gateway, copilot, kilocode, kimi-coding, kimi-coding-cn, opencode-zen). It missed local-endpoint aliases — ollama, vllm, llamacpp — which auth.resolve_provider routes to "custom" but resolve_provider_full has no entry for, so catalog_provider is None. The validator's failure check started with `if catalog_provider is None or ...`, which short-circuited and rejected those providers even though their auth-resolved runtime id ("custom") was already in provider_ids_to_accept and known_providers. Drop the short-circuit; rely on the intersection check below. Genuinely unknown providers (e.g. "completely-made-up") still fail because the intersection is empty. Tests: parametrized regression test covering ollama / vllm / llamacpp (the cases NousResearch#17135 didn't address) plus bedrock and lmstudio (regression guards — those used to be on the list in NousResearch#16085 but were fixed by adding a HERMES_OVERLAYS / PROVIDER_REGISTRY entry); an invariant test asserting every name doctor prints in its "Valid providers" list passes its own validator; and a negative test asserting a fake provider name is still rejected.
949383c to
4c45156
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & Why
#17135 (merged) fixed
hermes doctorfor providers whose Hermes runtime id differs from the models.dev catalog id:ai-gateway,copilot,kilocode,kimi-coding,kimi-coding-cn,opencode-zen. It did not cover local-endpoint aliases —ollama,vllm,llamacpp— which still produce a self-contradicting "unknown provider" error today.Reproduction on current
main(post #17135):ollamais not in the printed "known" list, but it's a documented user-facing value —auth.resolve_providerroutes it tocustom, which is in the list.Closes #16076 (
copilot, fixed by #17135). Closes the still-open part of #16085. Supersedes the earlier scope of this PR; #15159 / #15361 / #15583 / #15778 stay closed.Root cause (what #17135 missed)
Three providers —
ollama,vllm,llamacpp— sit inauth.py:_PROVIDER_ALIASESmapping to"custom", but have no entry inproviders.py. After #17135, the validator built upprovider_ids_to_accept = {raw, runtime_resolved, catalog_resolved}correctly (including"custom"for these three), but the failure condition still started with:resolve_provider_full("ollama")returnsNone, socatalog_provider is None, and the short-circuit rejects the provider before the intersection check ever runs.auth.resolve_provider(...)resolve_provider_full(...).idollamacustomNonevllmcustomNonellamacppcustomNonebedrockbedrockbedrock(HERMES_OVERLAYS entry now exists)lmstudiolmstudio(now in PROVIDER_REGISTRY)lmstudiobedrockandlmstudiowere on the original #16085 list but were fixed independently —bedrockby adding aHERMES_OVERLAYSentry,lmstudioby promoting it toPROVIDER_REGISTRY(214ca94,feat(agent): add lmstudio integration). Both are kept in the parametrized test as regression guards.Change
One behaviour change in
hermes_cli/doctor.py: drop thecatalog_provider is None or ...short-circuit. The intersection check below already covers the cases that matter — aNonecatalog id is not a sufficient signal that the provider is unknown when the auth-resolved runtime id lands inPROVIDER_REGISTRY.Tests
tests/hermes_cli/test_doctor.pyadds:test_run_doctor_accepts_provider_when_catalog_resolution_returns_none— parametrized overollama,vllm,llamacpp(the cases fix(doctor): accept catalog provider aliases #17135 didn't fix) plusbedrockandlmstudioas regression guards.test_run_doctor_validator_invariant_holds_for_every_known_provider— invariant test asserting every name inPROVIDER_REGISTRY(i.e. every name doctor prints as valid) also passes its own validator. Catches forward regressions like the originalbedrockshape if a future provider lands inPROVIDER_REGISTRYwithout a matching catalog or alias entry.test_run_doctor_still_flags_unknown_provider— negative case:provider: completely-made-upis still flagged.$ scripts/run_tests.sh tests/hermes_cli/test_doctor.py 35 passed in 1.4sWider sweep on
tests/hermes_cli/: 3471 passed, 9 unrelated failures (test_gateway_service,test_gateway_wsl,test_setup_openclaw_migration,test_tools_config) that reproduce on a cleanmainwithout these changes.Platforms tested
Scope
One commit, one file plus tests. Six-line diff in
doctor.py(drop one short-circuit + a clarifying comment). No behaviour change for providers that already passed the validator.