Skip to content

Add per-channel model override (channel_models) — parallel to channel_prompts #17834

@PaulBlackSwan

Description

@PaulBlackSwan

Use case

The gateway today resolves model in this order: per-session runtime override (set via /model or workspace UI, scoped to one session) → model: in config.yaml. For users running Hermes against multiple Telegram/Discord topics or Slack channels with very different needs, the global default forces a compromise and /model doesn't carry over to the next thread.

Concrete example from a current Hermes user (Telegram with topics):

  • #dev / #code-review / #architecture topics → want Claude Opus 4.7 for deep reasoning, willing to pay token costs.
  • #daily / #personal / #journal topics → want a local Ollama model (e.g. gemma4-heretic on the user's Mac mini) for low-stakes chatter, zero token cost, lower latency.
  • Switching globally each time you change topics is impractical and using the workspace model picker doesn't persist across new threads.

Today there's no built-in way to express "this channel/topic uses model X" in config. Workarounds (multiple Hermes profiles per channel set, skills with model directives, smart routing) are either heavy (separate process trees and DBs), brittle, or content-based rather than channel-based.

Existing precedent

Hermes already supports per-channel ephemeral system prompts via channel_prompts (Discord, Slack — docs). The mechanism is clean:

  • Config maps channel_id → prompt string
  • gateway/config.py bridges it from platforms.<plat>.channel_prompts into adapter config.extra
  • gateway/platforms/base.py::resolve_channel_prompt() looks up exact channel ID first, then falls back to parent (forum / thread → parent channel)
  • Adapter calls the resolver when constructing MessageEvent, which carries channel_prompt to the agent run

The same pattern extended to channel_models would solve the use case without inventing new abstractions.

Proposed config

platforms:
  telegram:
    channel_models:
      "-1009999999999:101": anthropic/claude-opus-4.7    # cron-watchdog topic — deep reasoning
      "-1009999999999:102": ollama/gemma4-heretic         # daily-chatter topic — local
      "-1001234567890":     openai/gpt-5.5                # default for whole chat (no thread match)

  discord:
    channel_models:
      "1234567890": anthropic/claude-opus-4.7
      "9876543210": ollama/gemma4-heretic

  slack:
    channel_models:
      C0123456789: anthropic/claude-opus-4.7

Lookup mirrors channel_prompts:

  1. Exact chat_id:thread_id match (if message is in a thread/topic)
  2. Fall back to chat_id (parent chat / channel)
  3. Fall back to model: from top-level config

Proposed code (sketch — full PR can follow if maintainers want)

1. gateway/config.py — bridge config (mirror of channel_prompts block at L710)

if "channel_models" in platform_cfg:
    channel_models = platform_cfg["channel_models"]
    if isinstance(channel_models, dict):
        bridged["channel_models"] = {str(k): str(v).strip() for k, v in channel_models.items() if v}
    # silently ignore non-dict; misconfig surfaces in logs at resolution time

2. gateway/platforms/base.py — resolver helper (mirror of resolve_channel_prompt at L1036)

def resolve_channel_model(
    config_extra: dict,
    channel_id: str,
    parent_id: str | None = None,
) -> str | None:
    """Resolve a per-channel model override from platform config.

    Looks up ``channel_models`` in the adapter's ``config.extra`` dict.
    Prefers an exact match on *channel_id*; falls back to *parent_id*
    (useful for forum threads / topics inheriting a parent channel's model).

    Returns the model identifier string, or None if no match is found.
    """
    models = config_extra.get("channel_models") or {}
    if not isinstance(models, dict):
        return None
    for key in (channel_id, parent_id):
        if not key:
            continue
        model = models.get(key)
        if model is None:
            continue
        model = str(model).strip()
        if model:
            return model
    return None

3. MessageEvent field (parallel to channel_prompt, ~L872)

# Per-channel ephemeral model override (e.g. Telegram channel_models).
# Applied at AIAgent construction time, never persisted to session model state.
channel_model: Optional[str] = None

4. Per-platform wiring — Discord example (mirror of _resolve_channel_prompt at L2700)

def _resolve_channel_model(self, channel_id: str, parent_id: str | None = None) -> str | None:
    return resolve_channel_model(self.config.extra, channel_id, parent_id)

# in MessageEvent construction:
channel_model=self._resolve_channel_model(channel_id, parent_id or None),

Same wiring in telegram.py (using chat_id + message_thread_id) and slack.py (using channel).

5. gateway/run.py — apply override in model resolution

The exact insertion point is wherever _resolve_gateway_model() is consumed before AIAgent(model=...). Sketch (precedence: session runtime override > channel_model > config default):

model = _resolve_gateway_model(user_config)
if event.channel_model:                      # NEW
    model = event.channel_model              # NEW
override = self._session_model_overrides.get(resolved_session_key) if resolved_session_key else None
if override:
    ...  # existing session override logic still wins (explicit user intent)

Threading channel_model through to all AIAgent(...) call sites mirrors how channel_prompt is already threaded.

Precedence (proposed)

Most specific → least specific:

  1. Session-level model override — user explicitly switched model mid-conversation (workspace UI / /model command). Sticky for the session.
  2. channel_models[channel_id:thread_id] — exact topic match.
  3. channel_models[channel_id] — parent fallback.
  4. config.yaml::model — gateway default.
  5. fallback_providers — on auth failure / 429 / etc.

Rationale: the session override is an explicit runtime intent ("for THIS conversation, use X") and should not be silently overridden by config the next turn. channel_models is the new layer for "the default starting point for any new conversation in this channel".

Open design questions

  1. Provider/runtime resolution: a channel_models entry is a model identifier string (anthropic/claude-opus-4.7). The provider/api_key/base_url are still resolved through the existing runtime_provider.resolve_runtime_provider() flow. Should channel_models accept a richer object form ({model: ..., provider: ..., api_key_env: ...}) for cases where the channel needs a non-default provider? My instinct: start with string-only, add object form later if requested.
  2. Reasoning config / toolset overrides: same question — should channel_models also take reasoning_config and enabled_toolsets? Probably out of scope for this issue, but worth mentioning. Could be future channel_overrides: block.
  3. Workspace UI surface: would be nice to expose this in the workspace's Smart Routing UI, but that's outsourc-e/hermes-workspace territory — separate concern.

Alternatives considered

  • Profiles (one Hermes profile per channel set): heavy, separate state DBs, per-channel session continuity lost.
  • Skills with model directives: only works if the skill is invoked, not for free-form chat.
  • Smart Routing (workspace-side content matching): channel-agnostic, harder to predict.
  • A separate chat_routes config layer: more flexible but reinvents the channel_prompts pattern. Reusing the established pattern keeps cognitive load low.

Backward compatibility

  • New optional field; absent config = identical behavior to today.
  • Schema change is additive in gateway/config.py.
  • No DB migration.

I'd be happy to send a PR if maintainers signal interest in this direction. Wanted to align on shape (string-vs-object, precedence) before opening one.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low — cosmetic, nice to havearea/configConfig system, migrations, profilescomp/gatewayGateway runner, session dispatch, deliverytype/featureNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions