Skip to content

feat(model): per-source model selection via model.by_source#12227

Closed
handsdiff wants to merge 2 commits into
NousResearch:mainfrom
handsdiff:feat/per-source-model
Closed

feat(model): per-source model selection via model.by_source#12227
handsdiff wants to merge 2 commits into
NousResearch:mainfrom
handsdiff:feat/per-source-model

Conversation

@handsdiff

@handsdiff handsdiff commented Apr 18, 2026

Copy link
Copy Markdown
Contributor

Summary

Add a model.by_source.<kind> override so the agent's model and provider bundle can vary by who triggered the turn, not just the message text.

Supported kinds: owner (DM to a configured home channel), hub_peer (inbound from a Hub-style agent platform — detected by platform-value string for forward-compat), stranger (any other inbound), cron (scheduled jobs). Unknown kinds are silently ignored.

Note on stacking: this PR is stacked on top of #7297 (per-platform model overrides) because both hook into _resolve_session_agent_runtime in gateway/run.py. The two features are complementary: #7297 routes by platform (e.g. Hub gets a different model than Telegram), this PR routes by source identity within a platform (e.g. owner on Telegram gets a different model than a stranger on Telegram). When reviewing, diff against #7297's head — the commit-on-top is a single focused change.

Why

Today the only alternate-model hook is smart_model_routing.cheap_model, which routes by message length + keyword heuristics. It can't distinguish the operator asking a hard question from a bot pinging /status — they get the same model. For agents that mix high-value owner interactions with high-volume background traffic (cron fires, peer coordination), the identity axis lets you use a stronger model for work that needs it and a cheaper one for routine traffic, without changing the text-based routing.

Example config:

model:
  default: my-fast-model
  provider: custom
  api_key: sk-default
  base_url: https://example.com/v1
  by_source:
    owner:    { model: my-strong-model, api_key: sk-owner }
    hub_peer: { model: my-fast-model }
    stranger: { }            # empty → use base
    cron:     { model: my-fast-model }

Missing sources, missing fields within a source, and empty entries all fall back to the base model: block. When no by_source is present, behavior is identical to today.

How it works

  • New agent.smart_model_routing.apply_source_override(model, runtime_kwargs, model_config, source_kind) — pure helper, layers by_source.<kind> over (model, runtime_kwargs). Partial overrides supported; empty values ignored.
  • New GatewayRuntime._classify_source_kind(source) — classifies based on home-channel match (owner), platform-value string "hub" (hub_peer), otherwise stranger. No dependency on any specific platform enum value or helper that isn't already upstream.
  • Hook point: GatewayRuntime._resolve_session_agent_runtime applies the override as the final layer, below session /model overrides and above base config. Silent fallback on exception — never blocks a turn. Stacks cleanly with feat(gateway): per-platform model overrides via config.yaml #7297's platform-override logic.
  • Cron scheduler applies override with source_kind="cron" before calling resolve_turn_route.
  • HermesCLI applies override with source_kind="owner" in _resolve_turn_agent_config.

The router (resolve_turn_route), AIAgent construction signature, session keys, agent cache, and sub-agent inheritance are all untouched. Existing smart_model_routing.cheap_model still works exactly as before — it's a peer hook, not a replacement.

Tests

9 new unit tests for apply_source_override covering:

  • no-op paths (missing kind, missing config, empty entry, unknown kind, null values)
  • full overrides, partial overrides (base fields preserved)
  • args-list handling

All 6 existing smart_model_routing tests still pass.

Test plan

  • Unit: 15/15 pass (pytest tests/agent/test_smart_model_routing.py)
  • Live: verified end-to-end on a test VM — CLI turn with by_source.owner pointing to a different model/endpoint correctly swaps both model and base_url in the outbound request; default model path unchanged for auxiliary calls.

🤖 Generated with Claude Code

handsdiff and others added 2 commits April 18, 2026 13:13
Add optional `model.platforms` config to override the default model
and provider for specific messaging platforms. Supports both a string
shorthand (model name only) and a dict form with model, base_url, and
api_key for full provider override.

Config example:
  model:
    default: claude-sonnet-4-6
    platforms:
      hub:
        model: claude-haiku-4-5
        base_url: "https://other-endpoint/v1"
        api_key: "sk-..."

Platform-awareness is pushed into the central `_resolve_session_agent_runtime`
helper so every caller that passes a `SessionSource` (regular turns,
background tasks, /btw, hygiene compress, manual /compress) gets overrides
for free. The memory-flush path passes `source=None` and so always uses
the default model — matching the intent that a rolling-memory flush is
an agent-internal operation, not a user-facing turn.

Session /model overrides (set via the /model command) still take absolute
precedence over platform overrides: a complete session bundle short-circuits
platform resolution entirely, and a model-only session override is layered
on top of the platform runtime bundle.

The fallback-eviction check in `_run_agent` also consults the platform-aware
default model, so platform-override sessions aren't evicted from the agent
cache after every run.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Allow the agent's model and provider bundle to vary by the *source* that
triggered the turn — owner (DM to a configured home channel), hub_peer
(inbound from a Hub-style agent-to-agent platform), stranger (any other
inbound), or cron (scheduled job).

Configuration (all fields optional per source; missing fields inherit from
the base `model:` block):

    model:
      default: my-fast-model
      provider: custom
      api_key: sk-default
      base_url: https://example.com/v1
      by_source:
        owner:    { model: my-strong-model, api_key: sk-owner }
        hub_peer: { model: my-fast-model }
        stranger: { }           # empty → use base
        cron:     { model: my-fast-model }

Motivation: owner interactions (operator or home-channel DM) often benefit
from a stronger model, while high-volume background traffic (cron, peer
coordination) runs fine on a cheaper one. Today the only alternate-model
hook is `smart_model_routing.cheap_model`, which routes by message length
and keyword heuristics — it can't distinguish *who* is talking. This adds
the identity axis without changing any existing behavior: if no
`by_source` block is present, the code path is identical to today.

Implementation:

- `agent/smart_model_routing.apply_source_override(model, runtime,
  model_config, source_kind)` is a new pure helper that layers an
  override bundle onto `(model, runtime_kwargs)` when
  `model.by_source.<source_kind>` is present. Missing fields inherit.

- `GatewayRuntime._classify_source_kind(source)` inlines the classifier
  (owner via home-channel match, hub_peer via platform-value string,
  stranger otherwise). Self-contained; no new cross-module deps.

- `GatewayRuntime._resolve_session_agent_runtime` applies the override
  as the final layer, below session `/model` overrides and above the
  base config. Silent fallback on exception — never blocks a turn.

- Cron scheduler and HermesCLI each apply the override with their own
  hardcoded source_kind ("cron" and "owner" respectively) before calling
  `resolve_turn_route`. The router itself is untouched.

No changes to AIAgent construction signature, session keys, agent
caching, sub-agent inheritance, or the existing cheap/expensive routing
path. The override stacks below session `/model` overrides so
interactive overrides still win.

Tests: 9 new unit tests for `apply_source_override` covering
no-op paths (missing kind, missing config, empty entry, unknown kind),
full overrides, partial overrides (base fields preserved), and
empty-value rejection. All 6 existing `smart_model_routing` tests still
pass.
@handsdiff handsdiff force-pushed the feat/per-source-model branch from be6e12f to f3a18ec Compare April 18, 2026 17:17
@handsdiff

Copy link
Copy Markdown
Contributor Author

Superseded by #12234 (match-based unified routing). That PR subsumes both this (per-source) and #7297 (per-platform) into a single model.routes matcher while keeping both shorthand config forms working via a legacy shim. Closing in favor of the unified design.

@handsdiff handsdiff closed this Apr 18, 2026
@handsdiff handsdiff deleted the feat/per-source-model branch April 18, 2026 17:27
handsdiff added a commit to handsdiff/hermes-agent that referenced this pull request Apr 22, 2026
…er-platform + per-source drafts)

Unify two complementary-but-separate ideas — per-platform model overrides
and per-source-identity model overrides — into one match-based router.
``model.routes`` is a list of ``{match, model, provider, api_key, ...}``
entries; each ``match`` is a subset predicate against a context dict of
``{platform, source_kind, ...}`` built by the caller. First match wins;
no match leaves the base model untouched.

    model:
      default: my-fast-model
      routes:
        - match: { source_kind: owner }        # owner everywhere
          model: my-strong-model
          api_key: sk-owner
        - match: { platform: hub }             # all hub peers
          model: my-fast-model
        - match: { source_kind: cron }
          model: my-fast-model
        - match: { platform: discord, source_kind: stranger }  # compound
          model: some-other

Backwards-compatible shorthand — both forms synthesize ``routes`` entries
internally:

    model.platforms.<name>:       # existing per-platform shorthand
    model.by_source.<kind>:       # per-source shorthand

Explicit ``routes`` always evaluate first; legacy shims run last, so an
explicit route always wins over an equivalent legacy entry.

Hook points:

- ``agent.smart_model_routing.apply_route`` — pure helper, normalizes
  legacy shorthand, iterates routes, returns ``(model, runtime_kwargs)``.
- ``GatewayRuntime._classify_source_kind(source)`` — classifies
  owner / hub_peer / stranger via home-channel match + platform-value
  string. No dependency on a specific ``Platform.HUB`` enum member.
- ``GatewayRuntime._build_routing_context(source)`` — assembles the
  context dict consumed by ``apply_route``.
- ``GatewayRuntime._resolve_session_agent_runtime`` calls ``apply_route``
  as the final layer, below session ``/model`` overrides and above base
  config. Silent fallback on exception — never blocks a turn.
- Cron scheduler applies with context ``{platform: cron, source_kind:
  cron}`` after runtime resolution and before AIAgent construction.
- HermesCLI applies with context ``{platform: cli, source_kind: owner}``
  in ``_resolve_turn_agent_config``.

This supersedes the earlier ``feat/model-routing`` branch that was cut
before upstream NousResearch#12732 wholesale-removed the separate ``smart_model_routing``
cheap-model router. This rewrite drops the obsolete ``resolve_turn_route`` /
``cheap_model`` integration and lands ``apply_route`` as a standalone
feature on a new ``agent/smart_model_routing.py`` file.

Tests: 14 new unit tests covering the matcher (empty context / no routes
/ platform match / source_kind match / compound match / first-match-wins
/ partial override / empty-value rejection / missing context key / null
config) plus legacy shims (platforms string / platforms dict /
by_source). All pass. No regressions in tests/gateway/, tests/cron/,
or tests/agent/ beyond 12 pre-existing upstream-main failures (dingtalk,
matrix, agent_cache) unrelated to this change.

Supersedes NousResearch#7297 (feat/per-platform-model) and the prior draft at NousResearch#12227
(feat/per-source-model). ``model.platforms.*`` configs keep working via
the legacy shim, so deployments on NousResearch#7297 need no migration.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
handsdiff added a commit to handsdiff/hermes-agent that referenced this pull request Apr 24, 2026
…g), drop NousResearch#7297, add NousResearch#12207

- Add rows for NousResearch#12234 (match-based model routing, supersedes NousResearch#7297 and draft NousResearch#12227)
  and NousResearch#12207 (compound-background-subshell-leak).
- Move NousResearch#7297 into a new 'Closed / superseded' section; note the branches
  (feat/per-platform-model and feat/per-source-model) are already deleted from origin.
- Update rebase workflow: swap the feat/per-platform-model line for feat/model-routing,
  add fix/compound-background-subshell-leak.
- Update the fork-main rebuild section: document that octopus strategy fails on
  adjacent-region additions and switch the documented command to a sequential-merge
  loop. Note the recurring conflict site (_classify_source_kind vs _is_owner_source)
  and the union-resolve strategy for it.
- Add PR-specific note for NousResearch#12234 covering the helper, classifier, legacy shim, and
  rebase-conflict guidance.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant