feat(gateway): per-platform model overrides via config.yaml#7297
Closed
handsdiff wants to merge 1 commit into
Closed
feat(gateway): per-platform model overrides via config.yaml#7297handsdiff wants to merge 1 commit into
handsdiff wants to merge 1 commit into
Conversation
82212f7 to
d6ad0bf
Compare
Add optional `model.platforms` config to override the default model
and provider for specific messaging platforms. Supports both a string
shorthand (model name only) and a dict form with model, base_url, and
api_key for full provider override.
Config example:
model:
default: claude-sonnet-4-6
platforms:
hub:
model: claude-haiku-4-5
base_url: "https://other-endpoint/v1"
api_key: "sk-..."
Platform-awareness is pushed into the central `_resolve_session_agent_runtime`
helper so every caller that passes a `SessionSource` (regular turns,
background tasks, /btw, hygiene compress, manual /compress) gets overrides
for free. The memory-flush path passes `source=None` and so always uses
the default model — matching the intent that a rolling-memory flush is
an agent-internal operation, not a user-facing turn.
Session /model overrides (set via the /model command) still take absolute
precedence over platform overrides: a complete session bundle short-circuits
platform resolution entirely, and a model-only session override is layered
on top of the platform runtime bundle.
The fallback-eviction check in `_run_agent` also consults the platform-aware
default model, so platform-override sessions aren't evicted from the agent
cache after every run.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
d6ad0bf to
11c2f18
Compare
2 tasks
Contributor
Author
|
Superseded by #12234 (match-based unified routing). That PR generalizes per-platform overrides into a |
handsdiff
added a commit
to handsdiff/hermes-agent
that referenced
this pull request
Apr 22, 2026
…er-platform + per-source drafts)
Unify two complementary-but-separate ideas — per-platform model overrides
and per-source-identity model overrides — into one match-based router.
``model.routes`` is a list of ``{match, model, provider, api_key, ...}``
entries; each ``match`` is a subset predicate against a context dict of
``{platform, source_kind, ...}`` built by the caller. First match wins;
no match leaves the base model untouched.
model:
default: my-fast-model
routes:
- match: { source_kind: owner } # owner everywhere
model: my-strong-model
api_key: sk-owner
- match: { platform: hub } # all hub peers
model: my-fast-model
- match: { source_kind: cron }
model: my-fast-model
- match: { platform: discord, source_kind: stranger } # compound
model: some-other
Backwards-compatible shorthand — both forms synthesize ``routes`` entries
internally:
model.platforms.<name>: # existing per-platform shorthand
model.by_source.<kind>: # per-source shorthand
Explicit ``routes`` always evaluate first; legacy shims run last, so an
explicit route always wins over an equivalent legacy entry.
Hook points:
- ``agent.smart_model_routing.apply_route`` — pure helper, normalizes
legacy shorthand, iterates routes, returns ``(model, runtime_kwargs)``.
- ``GatewayRuntime._classify_source_kind(source)`` — classifies
owner / hub_peer / stranger via home-channel match + platform-value
string. No dependency on a specific ``Platform.HUB`` enum member.
- ``GatewayRuntime._build_routing_context(source)`` — assembles the
context dict consumed by ``apply_route``.
- ``GatewayRuntime._resolve_session_agent_runtime`` calls ``apply_route``
as the final layer, below session ``/model`` overrides and above base
config. Silent fallback on exception — never blocks a turn.
- Cron scheduler applies with context ``{platform: cron, source_kind:
cron}`` after runtime resolution and before AIAgent construction.
- HermesCLI applies with context ``{platform: cli, source_kind: owner}``
in ``_resolve_turn_agent_config``.
This supersedes the earlier ``feat/model-routing`` branch that was cut
before upstream NousResearch#12732 wholesale-removed the separate ``smart_model_routing``
cheap-model router. This rewrite drops the obsolete ``resolve_turn_route`` /
``cheap_model`` integration and lands ``apply_route`` as a standalone
feature on a new ``agent/smart_model_routing.py`` file.
Tests: 14 new unit tests covering the matcher (empty context / no routes
/ platform match / source_kind match / compound match / first-match-wins
/ partial override / empty-value rejection / missing context key / null
config) plus legacy shims (platforms string / platforms dict /
by_source). All pass. No regressions in tests/gateway/, tests/cron/,
or tests/agent/ beyond 12 pre-existing upstream-main failures (dingtalk,
matrix, agent_cache) unrelated to this change.
Supersedes NousResearch#7297 (feat/per-platform-model) and the prior draft at NousResearch#12227
(feat/per-source-model). ``model.platforms.*`` configs keep working via
the legacy shim, so deployments on NousResearch#7297 need no migration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
handsdiff
added a commit
to handsdiff/hermes-agent
that referenced
this pull request
Apr 24, 2026
…g), drop NousResearch#7297, add NousResearch#12207 - Add rows for NousResearch#12234 (match-based model routing, supersedes NousResearch#7297 and draft NousResearch#12227) and NousResearch#12207 (compound-background-subshell-leak). - Move NousResearch#7297 into a new 'Closed / superseded' section; note the branches (feat/per-platform-model and feat/per-source-model) are already deleted from origin. - Update rebase workflow: swap the feat/per-platform-model line for feat/model-routing, add fix/compound-background-subshell-leak. - Update the fork-main rebuild section: document that octopus strategy fails on adjacent-region additions and switch the documented command to a sequential-merge loop. Note the recurring conflict site (_classify_source_kind vs _is_owner_source) and the union-resolve strategy for it. - Add PR-specific note for NousResearch#12234 covering the helper, classifier, legacy shim, and rebase-conflict guidance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds optional
model.platformsconfig to override the default model on a per-platform basis. This allows using different models for different messaging platforms while keeping a single inference endpoint.Why
Multi-platform agents often want a primary model for user-facing conversations (Telegram, Discord) but a cheaper/faster model for high-volume automated conversations (Hub agent-to-agent messaging, webhooks). Currently there's no way to do this without running separate instances.
Config
When
model.platforms.{platform}is set, that model is used for conversations on that platform. When not set, falls back tomodel.default. Internal operations (memory flush, compress, context estimation) always use the default model.How it works
_resolve_gateway_model()gains an optionalplatformparameter. The three message-handling call sites (main conversation, background tasks, /btw) pass the platform key. All other call sites (internal operations) continue using the default.How to test
model.platforms.hub: some-modelin config.yamlsome-modelmodel.default/compressor trigger memory flush → should usemodel.defaultTested on Linux (Ubuntu 24.04).