Skip to content

fix(gateway): honour HERMES_INFERENCE_PROVIDER in _resolve_startup_runtime (#16857)#16873

Closed
Bartok9 wants to merge 1 commit into
NousResearch:mainfrom
Bartok9:fix/16857-new-after-model-switch-custom-provider
Closed

fix(gateway): honour HERMES_INFERENCE_PROVIDER in _resolve_startup_runtime (#16857)#16873
Bartok9 wants to merge 1 commit into
NousResearch:mainfrom
Bartok9:fix/16857-new-after-model-switch-custom-provider

Conversation

@Bartok9

@Bartok9 Bartok9 commented Apr 28, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #16857 — /new after /model switch to a custom provider model fails with "no API key" when the model name also exists in a native provider catalog.

Root Cause

In _resolve_startup_runtime(), HERMES_INFERENCE_PROVIDER (set by /model) was passed as a hint to detect_static_provider_for_model(), but custom providers have no static catalog. The function fell through to matching native catalogs by model name, overriding the explicit custom provider choice with the coincidentally-matching native provider.

Example: /model deepseek-v4-pro via custom:xuanji → /new → detect_static_provider_for_model matches native deepseek catalog → tries to use DEEPSEEK_API_KEY → 401.

Fix

Check HERMES_INFERENCE_PROVIDER immediately after HERMES_TUI_PROVIDER and return early when set. This env var is written by /model and represents the user's explicit provider choice — static catalog detection must not override it.

Also removes the now-redundant HERMES_INFERENCE_PROVIDER read from the current_provider fallback chain (line 651 of the original), since the early return means it can never be reached.

Changes

  • tui_gateway/server.py: Early-return on HERMES_INFERENCE_PROVIDER before static detection
  • tests/test_tui_gateway_server.py: Updated existing test + added 2 new tests

Testing

python3 -m pytest tests/test_tui_gateway_server.py -k "startup_runtime" -x -q -o "addopts="
# 6 passed

Test coverage:

Test Verifies
test_startup_runtime_honours_inference_provider_env Custom provider is returned directly; static detection is NOT called
test_startup_runtime_inference_provider_does_not_override_tui_provider HERMES_TUI_PROVIDER still takes precedence
Existing tests No regression in alias resolution, static detection, TUI provider

How to verify manually

  1. Configure a custom provider with a model name that overlaps a native catalog
  2. /model <overlapping-model-name> → should switch to custom provider
  3. /new → should start new session with the custom provider, not the native one

@Bartok9
fix(gateway): honour HERMES_INFERENCE_PROVIDER in _resolve_startup_ru…
7ef4ab6 
…ntime (NousResearch#16857)

When /model switches to a model served via a custom provider (e.g.
custom:xuanji) and the model name also exists in a native provider's
static catalog (e.g. deepseek-v4-pro in the native deepseek catalog),
/new incorrectly overrides the custom provider with the native one,
causing 'no API key' errors.

Root cause: _resolve_startup_runtime() passed HERMES_INFERENCE_PROVIDER
as the current_provider hint to detect_static_provider_for_model(), but
custom providers have no static catalog, so the function fell through to
matching native catalogs by model name.

Fix: check HERMES_INFERENCE_PROVIDER immediately after HERMES_TUI_PROVIDER
and return early when set. This env var is written by /model and represents
the user's explicit provider choice — it must not be overridden by static
detection.

Also removes the now-redundant HERMES_INFERENCE_PROVIDER read from the
current_provider fallback chain, since it can never be reached.

Tests: updated existing test and added two new tests covering:
- HERMES_INFERENCE_PROVIDER is honoured (static detection skipped)
- HERMES_TUI_PROVIDER still takes precedence over HERMES_INFERENCE_PROVIDER

Fixes NousResearch#16857
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/gateway Gateway runner, session dispatch, delivery comp/tui Terminal UI (ui-tui/ + tui_gateway/) labels Apr 28, 2026
@teknium1 teknium1 mentioned this pull request Apr 28, 2026
Merged
teknium1 added a commit that referenced this pull request Apr 28, 2026
@teknium1
fix(tui): /model writes HERMES_TUI_PROVIDER unconditionally (#16857) (
9e4d79b 
#16897)

`/new` after `/model <custom-provider>:<model>` silently reverted to a
native provider whose static catalog happened to contain the same model
name (e.g. `deepseek-v4-pro` → native `deepseek` → 401).

Root cause at the `/model` writeback site: `HERMES_INFERENCE_PROVIDER`
was set unconditionally but `HERMES_TUI_PROVIDER` was only mirrored when
it was already set. On sessions launched without `--provider`,
`HERMES_TUI_PROVIDER` stayed unset, so `_resolve_startup_runtime()` on
`/new` skipped the explicit-provider early return and fell through to
`detect_static_provider_for_model()`.

Fix: set `HERMES_TUI_PROVIDER` unconditionally alongside
`HERMES_INFERENCE_PROVIDER` when `/model` lands. Keeps #15755's
invariant intact — `HERMES_TUI_PROVIDER` remains the canonical
"explicit this process" carrier, `HERMES_INFERENCE_PROVIDER` remains
ambient and does not short-circuit startup resolution.

Bug report and diagnosis: @Bartok9 in #16857 / #16873.

Fixes #16857
@teknium1

teknium1 commented Apr 28, 2026

Copy link
Copy Markdown
Contributor

Superseded by #16897 — merged to main.

Your diagnosis in #16857 and the initial fix here were spot on. The salvage applies the fix at the /model writeback site (_apply_model_switch, tui_gateway/server.py:850) rather than in _resolve_startup_runtime, which keeps the invariant from #15755 intact (HERMES_TUI_PROVIDER = explicit-this-process carrier, HERMES_INFERENCE_PROVIDER = ambient — must not short-circuit startup resolution). Both behaviors now hold simultaneously.

Credited in the merge commit and PR body. Thanks for the report and the clean reproduction steps!

@teknium1 teknium1 closed this Apr 28, 2026
cluricaun28 referenced this pull request in cluricaun28/Logos Apr 28, 2026
@teknium1
fix(tui): /model writes HERMES_TUI_PROVIDER unconditionally (#16857…
c4050ab 
…) (#16897)

`/new` after `/model <custom-provider>:<model>` silently reverted to a
native provider whose static catalog happened to contain the same model
name (e.g. `deepseek-v4-pro` → native `deepseek` → 401).

Root cause at the `/model` writeback site: `HERMES_INFERENCE_PROVIDER`
was set unconditionally but `HERMES_TUI_PROVIDER` was only mirrored when
it was already set. On sessions launched without `--provider`,
`HERMES_TUI_PROVIDER` stayed unset, so `_resolve_startup_runtime()` on
`/new` skipped the explicit-provider early return and fell through to
`detect_static_provider_for_model()`.

Fix: set `HERMES_TUI_PROVIDER` unconditionally alongside
`HERMES_INFERENCE_PROVIDER` when `/model` lands. Keeps #15755's
invariant intact — `HERMES_TUI_PROVIDER` remains the canonical
"explicit this process" carrier, `HERMES_INFERENCE_PROVIDER` remains
ambient and does not short-circuit startup resolution.

Bug report and diagnosis: @Bartok9 in #16857 / #16873.

Fixes #16857
ulasbilgen pushed a commit to ulasbilgen/hermes-adhd-agent that referenced this pull request May 1, 2026
@teknium1
fix(tui): /model writes HERMES_TUI_PROVIDER unconditionally (NousRe…
7acf88a 
…search#16857) (NousResearch#16897)

`/new` after `/model <custom-provider>:<model>` silently reverted to a
native provider whose static catalog happened to contain the same model
name (e.g. `deepseek-v4-pro` → native `deepseek` → 401).

Root cause at the `/model` writeback site: `HERMES_INFERENCE_PROVIDER`
was set unconditionally but `HERMES_TUI_PROVIDER` was only mirrored when
it was already set. On sessions launched without `--provider`,
`HERMES_TUI_PROVIDER` stayed unset, so `_resolve_startup_runtime()` on
`/new` skipped the explicit-provider early return and fell through to
`detect_static_provider_for_model()`.

Fix: set `HERMES_TUI_PROVIDER` unconditionally alongside
`HERMES_INFERENCE_PROVIDER` when `/model` lands. Keeps NousResearch#15755's
invariant intact — `HERMES_TUI_PROVIDER` remains the canonical
"explicit this process" carrier, `HERMES_INFERENCE_PROVIDER` remains
ambient and does not short-circuit startup resolution.

Bug report and diagnosis: @Bartok9 in NousResearch#16857 / NousResearch#16873.

Fixes NousResearch#16857
02356abc pushed a commit to 02356abc/hermes-agent that referenced this pull request May 14, 2026
@teknium1
fix(tui): /model writes HERMES_TUI_PROVIDER unconditionally (NousRe…
7750d7b 
…search#16857) (NousResearch#16897)

`/new` after `/model <custom-provider>:<model>` silently reverted to a
native provider whose static catalog happened to contain the same model
name (e.g. `deepseek-v4-pro` → native `deepseek` → 401).

Root cause at the `/model` writeback site: `HERMES_INFERENCE_PROVIDER`
was set unconditionally but `HERMES_TUI_PROVIDER` was only mirrored when
it was already set. On sessions launched without `--provider`,
`HERMES_TUI_PROVIDER` stayed unset, so `_resolve_startup_runtime()` on
`/new` skipped the explicit-provider early return and fell through to
`detect_static_provider_for_model()`.

Fix: set `HERMES_TUI_PROVIDER` unconditionally alongside
`HERMES_INFERENCE_PROVIDER` when `/model` lands. Keeps NousResearch#15755's
invariant intact — `HERMES_TUI_PROVIDER` remains the canonical
"explicit this process" carrier, `HERMES_INFERENCE_PROVIDER` remains
ambient and does not short-circuit startup resolution.

Bug report and diagnosis: @Bartok9 in NousResearch#16857 / NousResearch#16873.

Fixes NousResearch#16857
dannyJ848 pushed a commit to dannyJ848/hermes-agent that referenced this pull request May 17, 2026
@teknium1
fix(tui): /model writes HERMES_TUI_PROVIDER unconditionally (NousRe…
983373b 
…search#16857) (NousResearch#16897)

`/new` after `/model <custom-provider>:<model>` silently reverted to a
native provider whose static catalog happened to contain the same model
name (e.g. `deepseek-v4-pro` → native `deepseek` → 401).

Root cause at the `/model` writeback site: `HERMES_INFERENCE_PROVIDER`
was set unconditionally but `HERMES_TUI_PROVIDER` was only mirrored when
it was already set. On sessions launched without `--provider`,
`HERMES_TUI_PROVIDER` stayed unset, so `_resolve_startup_runtime()` on
`/new` skipped the explicit-provider early return and fell through to
`detect_static_provider_for_model()`.

Fix: set `HERMES_TUI_PROVIDER` unconditionally alongside
`HERMES_INFERENCE_PROVIDER` when `/model` lands. Keeps NousResearch#15755's
invariant intact — `HERMES_TUI_PROVIDER` remains the canonical
"explicit this process" carrier, `HERMES_INFERENCE_PROVIDER` remains
ambient and does not short-circuit startup resolution.

Bug report and diagnosis: @Bartok9 in NousResearch#16857 / NousResearch#16873.

Fixes NousResearch#16857
gweeteve pushed a commit to gweeteve/hermes-agent that referenced this pull request Jun 2, 2026
@teknium1
fix(tui): /model writes HERMES_TUI_PROVIDER unconditionally (NousRe…
f3d035d 
…search#16857) (NousResearch#16897)

`/new` after `/model <custom-provider>:<model>` silently reverted to a
native provider whose static catalog happened to contain the same model
name (e.g. `deepseek-v4-pro` → native `deepseek` → 401).

Root cause at the `/model` writeback site: `HERMES_INFERENCE_PROVIDER`
was set unconditionally but `HERMES_TUI_PROVIDER` was only mirrored when
it was already set. On sessions launched without `--provider`,
`HERMES_TUI_PROVIDER` stayed unset, so `_resolve_startup_runtime()` on
`/new` skipped the explicit-provider early return and fell through to
`detect_static_provider_for_model()`.

Fix: set `HERMES_TUI_PROVIDER` unconditionally alongside
`HERMES_INFERENCE_PROVIDER` when `/model` lands. Keeps NousResearch#15755's
invariant intact — `HERMES_TUI_PROVIDER` remains the canonical
"explicit this process" carrier, `HERMES_INFERENCE_PROVIDER` remains
ambient and does not short-circuit startup resolution.

Bug report and diagnosis: @Bartok9 in NousResearch#16857 / NousResearch#16873.

Fixes NousResearch#16857
Egavasyug pushed a commit to Egavasyug/hermes-agent that referenced this pull request Jun 10, 2026
@teknium1
fix(tui): /model writes HERMES_TUI_PROVIDER unconditionally (NousRe…
20bd0f4 
…search#16857) (NousResearch#16897)

`/new` after `/model <custom-provider>:<model>` silently reverted to a
native provider whose static catalog happened to contain the same model
name (e.g. `deepseek-v4-pro` → native `deepseek` → 401).

Root cause at the `/model` writeback site: `HERMES_INFERENCE_PROVIDER`
was set unconditionally but `HERMES_TUI_PROVIDER` was only mirrored when
it was already set. On sessions launched without `--provider`,
`HERMES_TUI_PROVIDER` stayed unset, so `_resolve_startup_runtime()` on
`/new` skipped the explicit-provider early return and fell through to
`detect_static_provider_for_model()`.

Fix: set `HERMES_TUI_PROVIDER` unconditionally alongside
`HERMES_INFERENCE_PROVIDER` when `/model` lands. Keeps NousResearch#15755's
invariant intact — `HERMES_TUI_PROVIDER` remains the canonical
"explicit this process" carrier, `HERMES_INFERENCE_PROVIDER` remains
ambient and does not short-circuit startup resolution.

Bug report and diagnosis: @Bartok9 in NousResearch#16857 / NousResearch#16873.

Fixes NousResearch#16857
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

comp/gateway Gateway runner, session dispatch, delivery comp/tui Terminal UI (ui-tui/ + tui_gateway/) P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

TUI: /new after /model switch to custom provider model fails with 'no API key' when model name exists in native provider catalog

3 participants

@Bartok9 @teknium1 @alt-glitch