Skip to content

Bug: Status banner fails to detect context_length from custom_providers #5089

@chevyphillip

Description

@chevyphillip

Bug Description

The Hermes Agent status banner (and internal context tracking in the gateway) consistently falls back to the default 128K context length display/limit for models defined under custom_providers in ~/.hermes/config.yaml, even when a specific context_length is provided in the provider's model list.

Steps to Reproduce

  1. Configure a custom provider in ~/.hermes/config.yaml:
model:
  default: my-huge-model
  provider: my-custom
custom_providers:
  - name: my-custom
    base_url: https://my-gateway.io/v1
    models:
      - name: my-huge-model
        context_length: 1100000
  1. Start the agent or check the status banner.
  2. Actual Behavior: Shows Context: 128K tokens (default — set model.context_length in config to override).

Expected Behavior

The gateway should traverse custom_providers to match the active model and inherit its context_length before falling back to the 128K default.

Proposed Fix

In gateway/run.py, the _resolve_gateway_info function should be updated to check custom_providers if an explicit model.context_length is missing at the top level.

Potential implementation:

# Inside _resolve_gateway_info
if config_context_length is None:
    custom_providers = data.get("custom_providers") or []
    model_name = model_cfg.get("default") or model
    for cp in custom_providers:
        if (cp.get("name") == provider or cp.get("base_url") == base_url):
            for m in cp.get("models", []):
                if m.get("name") == model_name and "context_length" in m:
                    config_context_length = int(m["context_length"])
                    break

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions