Skip to content

feat(gateway): per-platform model overrides via config.yaml#7297

Closed
handsdiff wants to merge 1 commit into
NousResearch:mainfrom
handsdiff:feat/per-platform-model
Closed

feat(gateway): per-platform model overrides via config.yaml#7297
handsdiff wants to merge 1 commit into
NousResearch:mainfrom
handsdiff:feat/per-platform-model

Conversation

@handsdiff

Copy link
Copy Markdown
Contributor

What

Adds optional model.platforms config to override the default model on a per-platform basis. This allows using different models for different messaging platforms while keeping a single inference endpoint.

Why

Multi-platform agents often want a primary model for user-facing conversations (Telegram, Discord) but a cheaper/faster model for high-volume automated conversations (Hub agent-to-agent messaging, webhooks). Currently there's no way to do this without running separate instances.

Config

model:
  provider: custom
  default: claude-sonnet-4-6
  base_url: "https://my-endpoint/v1"
  api_key: "sk-..."
  platforms:
    hub: claude-haiku-4-5
    discord: claude-sonnet-4-6

When model.platforms.{platform} is set, that model is used for conversations on that platform. When not set, falls back to model.default. Internal operations (memory flush, compress, context estimation) always use the default model.

How it works

_resolve_gateway_model() gains an optional platform parameter. The three message-handling call sites (main conversation, background tasks, /btw) pass the platform key. All other call sites (internal operations) continue using the default.

How to test

  1. Set model.platforms.hub: some-model in config.yaml
  2. Send a message via Hub → agent should use some-model
  3. Send a message via Telegram → agent should use model.default
  4. Run /compress or trigger memory flush → should use model.default

Tested on Linux (Ubuntu 24.04).

@handsdiff handsdiff force-pushed the feat/per-platform-model branch 12 times, most recently from 82212f7 to d6ad0bf Compare April 17, 2026 14:20
Add optional `model.platforms` config to override the default model
and provider for specific messaging platforms. Supports both a string
shorthand (model name only) and a dict form with model, base_url, and
api_key for full provider override.

Config example:
  model:
    default: claude-sonnet-4-6
    platforms:
      hub:
        model: claude-haiku-4-5
        base_url: "https://other-endpoint/v1"
        api_key: "sk-..."

Platform-awareness is pushed into the central `_resolve_session_agent_runtime`
helper so every caller that passes a `SessionSource` (regular turns,
background tasks, /btw, hygiene compress, manual /compress) gets overrides
for free. The memory-flush path passes `source=None` and so always uses
the default model — matching the intent that a rolling-memory flush is
an agent-internal operation, not a user-facing turn.

Session /model overrides (set via the /model command) still take absolute
precedence over platform overrides: a complete session bundle short-circuits
platform resolution entirely, and a model-only session override is layered
on top of the platform runtime bundle.

The fallback-eviction check in `_run_agent` also consults the platform-aware
default model, so platform-override sessions aren't evicted from the agent
cache after every run.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@handsdiff

Copy link
Copy Markdown
Contributor Author

Superseded by #12234 (match-based unified routing). That PR generalizes per-platform overrides into a model.routes matcher with compound predicates, and preserves model.platforms.<name> as a legacy shorthand shim — so configs deployed against this PR continue to work without migration. Closing in favor of the unified design.

@handsdiff handsdiff closed this Apr 18, 2026
@handsdiff handsdiff deleted the feat/per-platform-model branch April 18, 2026 17:27
handsdiff added a commit to handsdiff/hermes-agent that referenced this pull request Apr 22, 2026
…er-platform + per-source drafts)

Unify two complementary-but-separate ideas — per-platform model overrides
and per-source-identity model overrides — into one match-based router.
``model.routes`` is a list of ``{match, model, provider, api_key, ...}``
entries; each ``match`` is a subset predicate against a context dict of
``{platform, source_kind, ...}`` built by the caller. First match wins;
no match leaves the base model untouched.

    model:
      default: my-fast-model
      routes:
        - match: { source_kind: owner }        # owner everywhere
          model: my-strong-model
          api_key: sk-owner
        - match: { platform: hub }             # all hub peers
          model: my-fast-model
        - match: { source_kind: cron }
          model: my-fast-model
        - match: { platform: discord, source_kind: stranger }  # compound
          model: some-other

Backwards-compatible shorthand — both forms synthesize ``routes`` entries
internally:

    model.platforms.<name>:       # existing per-platform shorthand
    model.by_source.<kind>:       # per-source shorthand

Explicit ``routes`` always evaluate first; legacy shims run last, so an
explicit route always wins over an equivalent legacy entry.

Hook points:

- ``agent.smart_model_routing.apply_route`` — pure helper, normalizes
  legacy shorthand, iterates routes, returns ``(model, runtime_kwargs)``.
- ``GatewayRuntime._classify_source_kind(source)`` — classifies
  owner / hub_peer / stranger via home-channel match + platform-value
  string. No dependency on a specific ``Platform.HUB`` enum member.
- ``GatewayRuntime._build_routing_context(source)`` — assembles the
  context dict consumed by ``apply_route``.
- ``GatewayRuntime._resolve_session_agent_runtime`` calls ``apply_route``
  as the final layer, below session ``/model`` overrides and above base
  config. Silent fallback on exception — never blocks a turn.
- Cron scheduler applies with context ``{platform: cron, source_kind:
  cron}`` after runtime resolution and before AIAgent construction.
- HermesCLI applies with context ``{platform: cli, source_kind: owner}``
  in ``_resolve_turn_agent_config``.

This supersedes the earlier ``feat/model-routing`` branch that was cut
before upstream NousResearch#12732 wholesale-removed the separate ``smart_model_routing``
cheap-model router. This rewrite drops the obsolete ``resolve_turn_route`` /
``cheap_model`` integration and lands ``apply_route`` as a standalone
feature on a new ``agent/smart_model_routing.py`` file.

Tests: 14 new unit tests covering the matcher (empty context / no routes
/ platform match / source_kind match / compound match / first-match-wins
/ partial override / empty-value rejection / missing context key / null
config) plus legacy shims (platforms string / platforms dict /
by_source). All pass. No regressions in tests/gateway/, tests/cron/,
or tests/agent/ beyond 12 pre-existing upstream-main failures (dingtalk,
matrix, agent_cache) unrelated to this change.

Supersedes NousResearch#7297 (feat/per-platform-model) and the prior draft at NousResearch#12227
(feat/per-source-model). ``model.platforms.*`` configs keep working via
the legacy shim, so deployments on NousResearch#7297 need no migration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
handsdiff added a commit to handsdiff/hermes-agent that referenced this pull request Apr 24, 2026
…g), drop NousResearch#7297, add NousResearch#12207

- Add rows for NousResearch#12234 (match-based model routing, supersedes NousResearch#7297 and draft NousResearch#12227)
  and NousResearch#12207 (compound-background-subshell-leak).
- Move NousResearch#7297 into a new 'Closed / superseded' section; note the branches
  (feat/per-platform-model and feat/per-source-model) are already deleted from origin.
- Update rebase workflow: swap the feat/per-platform-model line for feat/model-routing,
  add fix/compound-background-subshell-leak.
- Update the fork-main rebuild section: document that octopus strategy fails on
  adjacent-region additions and switch the documented command to a sequential-merge
  loop. Note the recurring conflict site (_classify_source_kind vs _is_owner_source)
  and the union-resolve strategy for it.
- Add PR-specific note for NousResearch#12234 covering the helper, classifier, legacy shim, and
  rebase-conflict guidance.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant