feat(model): per-source model selection via model.by_source#12227
Closed
handsdiff wants to merge 2 commits into
Closed
feat(model): per-source model selection via model.by_source#12227handsdiff wants to merge 2 commits into
handsdiff wants to merge 2 commits into
Conversation
Add optional `model.platforms` config to override the default model
and provider for specific messaging platforms. Supports both a string
shorthand (model name only) and a dict form with model, base_url, and
api_key for full provider override.
Config example:
model:
default: claude-sonnet-4-6
platforms:
hub:
model: claude-haiku-4-5
base_url: "https://other-endpoint/v1"
api_key: "sk-..."
Platform-awareness is pushed into the central `_resolve_session_agent_runtime`
helper so every caller that passes a `SessionSource` (regular turns,
background tasks, /btw, hygiene compress, manual /compress) gets overrides
for free. The memory-flush path passes `source=None` and so always uses
the default model — matching the intent that a rolling-memory flush is
an agent-internal operation, not a user-facing turn.
Session /model overrides (set via the /model command) still take absolute
precedence over platform overrides: a complete session bundle short-circuits
platform resolution entirely, and a model-only session override is layered
on top of the platform runtime bundle.
The fallback-eviction check in `_run_agent` also consults the platform-aware
default model, so platform-override sessions aren't evicted from the agent
cache after every run.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Allow the agent's model and provider bundle to vary by the *source* that
triggered the turn — owner (DM to a configured home channel), hub_peer
(inbound from a Hub-style agent-to-agent platform), stranger (any other
inbound), or cron (scheduled job).
Configuration (all fields optional per source; missing fields inherit from
the base `model:` block):
model:
default: my-fast-model
provider: custom
api_key: sk-default
base_url: https://example.com/v1
by_source:
owner: { model: my-strong-model, api_key: sk-owner }
hub_peer: { model: my-fast-model }
stranger: { } # empty → use base
cron: { model: my-fast-model }
Motivation: owner interactions (operator or home-channel DM) often benefit
from a stronger model, while high-volume background traffic (cron, peer
coordination) runs fine on a cheaper one. Today the only alternate-model
hook is `smart_model_routing.cheap_model`, which routes by message length
and keyword heuristics — it can't distinguish *who* is talking. This adds
the identity axis without changing any existing behavior: if no
`by_source` block is present, the code path is identical to today.
Implementation:
- `agent/smart_model_routing.apply_source_override(model, runtime,
model_config, source_kind)` is a new pure helper that layers an
override bundle onto `(model, runtime_kwargs)` when
`model.by_source.<source_kind>` is present. Missing fields inherit.
- `GatewayRuntime._classify_source_kind(source)` inlines the classifier
(owner via home-channel match, hub_peer via platform-value string,
stranger otherwise). Self-contained; no new cross-module deps.
- `GatewayRuntime._resolve_session_agent_runtime` applies the override
as the final layer, below session `/model` overrides and above the
base config. Silent fallback on exception — never blocks a turn.
- Cron scheduler and HermesCLI each apply the override with their own
hardcoded source_kind ("cron" and "owner" respectively) before calling
`resolve_turn_route`. The router itself is untouched.
No changes to AIAgent construction signature, session keys, agent
caching, sub-agent inheritance, or the existing cheap/expensive routing
path. The override stacks below session `/model` overrides so
interactive overrides still win.
Tests: 9 new unit tests for `apply_source_override` covering
no-op paths (missing kind, missing config, empty entry, unknown kind),
full overrides, partial overrides (base fields preserved), and
empty-value rejection. All 6 existing `smart_model_routing` tests still
pass.
be6e12f to
f3a18ec
Compare
Contributor
Author
3 tasks
handsdiff
added a commit
to handsdiff/hermes-agent
that referenced
this pull request
Apr 22, 2026
…er-platform + per-source drafts)
Unify two complementary-but-separate ideas — per-platform model overrides
and per-source-identity model overrides — into one match-based router.
``model.routes`` is a list of ``{match, model, provider, api_key, ...}``
entries; each ``match`` is a subset predicate against a context dict of
``{platform, source_kind, ...}`` built by the caller. First match wins;
no match leaves the base model untouched.
model:
default: my-fast-model
routes:
- match: { source_kind: owner } # owner everywhere
model: my-strong-model
api_key: sk-owner
- match: { platform: hub } # all hub peers
model: my-fast-model
- match: { source_kind: cron }
model: my-fast-model
- match: { platform: discord, source_kind: stranger } # compound
model: some-other
Backwards-compatible shorthand — both forms synthesize ``routes`` entries
internally:
model.platforms.<name>: # existing per-platform shorthand
model.by_source.<kind>: # per-source shorthand
Explicit ``routes`` always evaluate first; legacy shims run last, so an
explicit route always wins over an equivalent legacy entry.
Hook points:
- ``agent.smart_model_routing.apply_route`` — pure helper, normalizes
legacy shorthand, iterates routes, returns ``(model, runtime_kwargs)``.
- ``GatewayRuntime._classify_source_kind(source)`` — classifies
owner / hub_peer / stranger via home-channel match + platform-value
string. No dependency on a specific ``Platform.HUB`` enum member.
- ``GatewayRuntime._build_routing_context(source)`` — assembles the
context dict consumed by ``apply_route``.
- ``GatewayRuntime._resolve_session_agent_runtime`` calls ``apply_route``
as the final layer, below session ``/model`` overrides and above base
config. Silent fallback on exception — never blocks a turn.
- Cron scheduler applies with context ``{platform: cron, source_kind:
cron}`` after runtime resolution and before AIAgent construction.
- HermesCLI applies with context ``{platform: cli, source_kind: owner}``
in ``_resolve_turn_agent_config``.
This supersedes the earlier ``feat/model-routing`` branch that was cut
before upstream NousResearch#12732 wholesale-removed the separate ``smart_model_routing``
cheap-model router. This rewrite drops the obsolete ``resolve_turn_route`` /
``cheap_model`` integration and lands ``apply_route`` as a standalone
feature on a new ``agent/smart_model_routing.py`` file.
Tests: 14 new unit tests covering the matcher (empty context / no routes
/ platform match / source_kind match / compound match / first-match-wins
/ partial override / empty-value rejection / missing context key / null
config) plus legacy shims (platforms string / platforms dict /
by_source). All pass. No regressions in tests/gateway/, tests/cron/,
or tests/agent/ beyond 12 pre-existing upstream-main failures (dingtalk,
matrix, agent_cache) unrelated to this change.
Supersedes NousResearch#7297 (feat/per-platform-model) and the prior draft at NousResearch#12227
(feat/per-source-model). ``model.platforms.*`` configs keep working via
the legacy shim, so deployments on NousResearch#7297 need no migration.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
handsdiff
added a commit
to handsdiff/hermes-agent
that referenced
this pull request
Apr 24, 2026
…g), drop NousResearch#7297, add NousResearch#12207 - Add rows for NousResearch#12234 (match-based model routing, supersedes NousResearch#7297 and draft NousResearch#12227) and NousResearch#12207 (compound-background-subshell-leak). - Move NousResearch#7297 into a new 'Closed / superseded' section; note the branches (feat/per-platform-model and feat/per-source-model) are already deleted from origin. - Update rebase workflow: swap the feat/per-platform-model line for feat/model-routing, add fix/compound-background-subshell-leak. - Update the fork-main rebuild section: document that octopus strategy fails on adjacent-region additions and switch the documented command to a sequential-merge loop. Note the recurring conflict site (_classify_source_kind vs _is_owner_source) and the union-resolve strategy for it. - Add PR-specific note for NousResearch#12234 covering the helper, classifier, legacy shim, and rebase-conflict guidance.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add a
model.by_source.<kind>override so the agent's model and provider bundle can vary by who triggered the turn, not just the message text.Supported kinds:
owner(DM to a configured home channel),hub_peer(inbound from a Hub-style agent platform — detected by platform-value string for forward-compat),stranger(any other inbound),cron(scheduled jobs). Unknown kinds are silently ignored.Why
Today the only alternate-model hook is
smart_model_routing.cheap_model, which routes by message length + keyword heuristics. It can't distinguish the operator asking a hard question from a bot pinging/status— they get the same model. For agents that mix high-value owner interactions with high-volume background traffic (cron fires, peer coordination), the identity axis lets you use a stronger model for work that needs it and a cheaper one for routine traffic, without changing the text-based routing.Example config:
Missing sources, missing fields within a source, and empty entries all fall back to the base
model:block. When noby_sourceis present, behavior is identical to today.How it works
agent.smart_model_routing.apply_source_override(model, runtime_kwargs, model_config, source_kind)— pure helper, layersby_source.<kind>over(model, runtime_kwargs). Partial overrides supported; empty values ignored.GatewayRuntime._classify_source_kind(source)— classifies based on home-channel match (owner), platform-value string"hub"(hub_peer), otherwise stranger. No dependency on any specific platform enum value or helper that isn't already upstream.GatewayRuntime._resolve_session_agent_runtimeapplies the override as the final layer, below session/modeloverrides and above base config. Silent fallback on exception — never blocks a turn. Stacks cleanly with feat(gateway): per-platform model overrides via config.yaml #7297's platform-override logic.source_kind="cron"before callingresolve_turn_route.source_kind="owner"in_resolve_turn_agent_config.The router (
resolve_turn_route), AIAgent construction signature, session keys, agent cache, and sub-agent inheritance are all untouched. Existingsmart_model_routing.cheap_modelstill works exactly as before — it's a peer hook, not a replacement.Tests
9 new unit tests for
apply_source_overridecovering:All 6 existing
smart_model_routingtests still pass.Test plan
pytest tests/agent/test_smart_model_routing.py)by_source.ownerpointing to a different model/endpoint correctly swaps both model and base_url in the outbound request; default model path unchanged for auxiliary calls.🤖 Generated with Claude Code