fix: honor custom-provider context overrides in session info by Bot1822 · Pull Request #10690 · NousResearch/hermes-agent

Bot1822 · 2026-04-16T03:18:05Z

Summary

make get_model_context_length() honor named custom-provider context overrides even when callers do not pass model.context_length explicitly
reuse that resolution in gateway session info so /reset shows the real context window for legacy custom_providers and v12 providers configs
preserve /api/model/info semantics by continuing to report auto-detected context separately from implicit config-derived overrides

Why

The original fix/session-reset-custom-provider-context branch fixes the immediate gateway banner bug, but only inside GatewayRunner._format_session_info().

That was too narrow and still left other call sites inconsistent with the main runtime path that already treats named custom-provider per-model context_length as authoritative.

This PR makes the fix more robust by centralizing the override lookup in agent/model_metadata.py, then using it from gateway and web UI code without breaking the distinction between:

auto-detected context length
config-derived override
effective context length

Tests

venv/bin/python -m pytest tests/agent/test_model_metadata.py tests/gateway/test_session_info.py tests/hermes_cli/test_web_server.py -q

Bot1822 · 2026-04-16T04:21:04Z

Rebased this branch onto the latest main and force-pushed the updated history.

Local verification after the rebase:

python -m pytest tests/gateway/test_session_info.py -q tests/agent/test_model_metadata.py -q tests/hermes_cli/test_web_server.py -q
all tests passed

This still covers the original reset/session-info bug, plus the broader config-override cases in agent/model_metadata.py and hermes_cli/web_server.py for both legacy custom_providers and v12 providers configs.

Bot1822 · 2026-04-16T04:25:59Z

@teknium1 @austinpickett friendly ping on this one when you have a moment.

This branch is now rebased onto the latest main, the local regression tests pass, and the fix is scoped to the custom-provider context override bug affecting session info / reset display.

I also checked for repo-native AI review bots, but I'm not seeing an obvious configured reviewer/bot handle to ping here, so I'm just tagging human maintainers directly.

…nner The session-reset / info banner in `_format_session_info` resolved the context window only from the top-level `model.context_length` key. When users configured context_length under the new `providers.<name>.models.<model>` dict schema (or the legacy `custom_providers[].models.<model>` list schema), the banner fell through to `get_model_context_length()`'s probe chain. Remote OpenAI-compatible proxies frequently omit `context_length` from `/models` responses, so the probe failed silently and the banner displayed `128K tokens (default — set model.context_length in config to override)` — even though the runtime `ContextCompressor` was already budgeting with the correct value resolved by `AIAgent.__init__`. After the top-level lookup, walk `get_compatible_custom_providers(cfg)`, match the entry by `base_url` against the resolved runtime base_url, and read `entry["models"][model]["context_length"]`. This mirrors the exact resolution `AIAgent.__init__` performs at `run_agent.py:1608-1643` so the banner's displayed value matches what the compressor actually uses. Fixes NousResearch#5089. Relationship to existing open PRs (NousResearch/hermes-agent): - NousResearch#5096, NousResearch#8240, NousResearch#10690: each hand-roll traversal of the legacy `custom_providers:` list only. They do not cover the newer `providers:` dict schema users land on via `hermes model`. - NousResearch#12380: touches `/model` display + gateway fallback paths; overlaps in spirit but still traverses lists manually. Routing through `get_compatible_custom_providers()` — the existing compat shim at `hermes_cli/config.py:2078` — gives a single lookup that covers both schemas and stays consistent with every other runtime caller, eliminating future drift between display and compressor.

alt-glitch · 2026-04-25T21:38:24Z

Multiple open PRs target the same custom-provider context_length display bug: #15668, #15060, #5096, #14382. This PR is the most comprehensive fix (centralizes in model_metadata.py). Consider consolidating.

austinpickett

Fix merge conflicts, and please use .github/PULL_REQUEST_TEMPLATE.md, also address and resolve all Copilot comments.

Copilot

Pull request overview

This PR centralizes and broadens resolution of custom-provider context_length overrides so that context windows shown in gateway session info and /api/model/info reflect implicit per-provider/per-model config overrides even when callers don’t pass model.context_length explicitly.

Changes:

Extend agent.model_metadata.get_model_context_length() to consult custom_providers/providers config-derived overrides early in resolution.
Expand hermes_cli.config.get_custom_provider_context_length() to match by provider identity (name/provider_key) and support provider-level fallback context_length.
Update gateway session info and web /api/model/info to surface implicit config-derived overrides; add regression tests covering legacy custom_providers and v12 providers configs.

Reviewed changes

Copilot reviewed 8 out of 8 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
`agent/model_metadata.py`	Adds config-aware custom-provider override lookup to the core context-length resolver.
`hermes_cli/config.py`	Enhances custom-provider context override lookup (provider-name matching + provider-level fallback).
`gateway/run.py`	Updates gateway `/info` session formatting to reuse implicit override resolution for displayed context length/source.
`hermes_cli/web_server.py`	Makes `/api/model/info` report implicit config override separately from auto-detected context length.
`tests/agent/test_model_metadata.py`	Adds regression tests ensuring overrides are honored even without explicit `config_context_length`.
`tests/gateway/test_session_info.py`	Adds tests asserting gateway session info shows correct config-derived context for both config schemas.
`tests/hermes_cli/test_custom_provider_context_length.py`	Adds unit tests for provider-key matching without base_url and provider-level fallback.
`tests/hermes_cli/test_web_server.py`	Adds endpoint test asserting implicit config override is reported and used as effective context.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

    target_url = (base_url or "").rstrip("/")
-    if not target_url:
+    target_provider = (provider or "").strip().lower()
+    if not target_url and not target_provider:
        return None

    for entry in custom_providers:
        if not isinstance(entry, dict):
            continue
        entry_url = (entry.get("base_url") or "").rstrip("/")
-        if not entry_url or entry_url != target_url:
+        entry_name = str(entry.get("name") or "").strip().lower()
+        entry_provider_key = str(entry.get("provider_key") or "").strip().lower()
+        url_matches = bool(target_url and entry_url and entry_url == target_url)
+        provider_matches = bool(
+            target_provider and target_provider in {entry_name, entry_provider_key}
+        )
+        if not (url_matches or provider_matches):
            continue


+        if config_context_length is None:
+            try:
+                from hermes_cli.config import get_custom_provider_context_length
+                config_context_length = get_custom_provider_context_length(
+                    model=model,
+                    base_url=base_url or "",
+                    provider=provider or "",
+                    custom_providers=custom_provs,
+                    config=data,
+                )


Bot1822 force-pushed the fix/session-info-custom-provider-context-robust branch 2 times, most recently from 9987334 to 4a581ac Compare April 16, 2026 04:16

kshitijk4poor mentioned this pull request Apr 16, 2026

fix: honor custom provider context overrides for display and runtime #10885

Closed

pinch-claw mentioned this pull request Apr 23, 2026

fix(gateway): read nested per-model context_length in session info banner #14382

Open

4 tasks

This was referenced Apr 25, 2026

fix: reapply compression feasibility guard after model switch #15541

Open

fix(cli): sync use_gateway in _reconfigure_provider for tts, browser, and web #15669

Closed

fix: surface provider-scoped context length in gateway session info #15668

Open

alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/gateway Gateway runner, session dispatch, delivery area/config Config system, migrations, profiles labels Apr 25, 2026

fix: honor custom-provider context overrides in session info

42e73e5

Bot1822 force-pushed the fix/session-info-custom-provider-context-robust branch from 4a581ac to 42e73e5 Compare April 26, 2026 15:50

alt-glitch mentioned this pull request Apr 30, 2026

fix(gateway): show per-model custom context in session info #18025

Open

austinpickett requested a review from Copilot May 19, 2026 11:58

Copilot started reviewing on behalf of austinpickett May 19, 2026 11:59 View session

austinpickett requested changes May 19, 2026

View reviewed changes

Copilot AI reviewed May 19, 2026

View reviewed changes

Bot1822 closed this May 20, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix: honor custom-provider context overrides in session info#10690

fix: honor custom-provider context overrides in session info#10690
Bot1822 wants to merge 1 commit into
NousResearch:mainfrom
Bot1822:fix/session-info-custom-provider-context-robust

Bot1822 commented Apr 16, 2026

Uh oh!

Bot1822 commented Apr 16, 2026

Uh oh!

Bot1822 commented Apr 16, 2026

Uh oh!

alt-glitch commented Apr 25, 2026

Uh oh!

austinpickett left a comment •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Conversation

Bot1822 commented Apr 16, 2026

Summary

Why

Tests

Uh oh!

Bot1822 commented Apr 16, 2026

Uh oh!

Bot1822 commented Apr 16, 2026

Uh oh!

alt-glitch commented Apr 25, 2026

Uh oh!

austinpickett left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

austinpickett left a comment •

edited

Loading