Skip to content

feat(gateway): opt-in per-platform model via platform_models (#34612)#34620

Open
Bartok9 wants to merge 1 commit into
NousResearch:mainfrom
Bartok9:fix-34612-api-server-platform-model
Open

feat(gateway): opt-in per-platform model via platform_models (#34612)#34620
Bartok9 wants to merge 1 commit into
NousResearch:mainfrom
Bartok9:fix-34612-api-server-platform-model

Conversation

@Bartok9

@Bartok9 Bartok9 commented May 29, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Adds an opt-in platform parameter to _resolve_gateway_model() so a single gateway platform can run a different default model.
  • Wires the HTTP API server (api_server._create_agent) to opt in via platform_models.api_server, letting operators run it on a cheaper/faster model than CLI/Discord/Telegram.

Motivation

Closes #34612.

gateway/platforms/api_server.py::APIServerAdapter._create_agent resolved its model with a bare _resolve_gateway_model(), which only ever reads model.default / model.model. There was no platform dimension, so every gateway platform that builds a temporary agent shared one model — no supported knob to put, say, the API server on Sonnet while everything else stays on Opus.

This implements the issue author's proposed design faithfully: platform_models.<platform> (additive, optional) wins over the global default only when a caller passes platform=. The override accepts a bare model string or a {default|model} mapping. Provider/credentials still come from the global runtime config, so the override must name a model compatible with the active provider.

Config shape:

model:
  default: claude-opus-4-8
platform_models:
  api_server:
    default: claude-sonnet-4-6   # or a bare string: api_server: claude-sonnet-4-6

Verification

  • python3 -m pytest tests/test_empty_model_fallback.py — 20 passed (12 existing + 8 new in TestResolveGatewayModelPlatformOverride: opt-in isolation, matching/non-matching platform, bare-string + dict shapes, empty/missing/malformed platform_models).
  • python3 -m pytest tests/gateway/test_api_server.py tests/gateway/test_api_server_toolset.py — 168 passed.
  • python3 -m pytest tests/gateway/test_{fast_command,compress_command,discord_channel_prompts,session_info,compress_plugin_engine,compress_focus}.py — 31 passed. Confirms the other _resolve_gateway_model() consumers (compress/fast/discord/session_info) are unaffected.
  • Backwards compat: every existing call site omits platform= and is byte-for-byte unchanged. One existing monkeypatch in tests/gateway/test_api_server.py was widened from lambda: to lambda *a, **k: to accept the new optional arg.

Note (matches the issue's heads-up): the tests/gateway/test_api_server.py suite leaks file descriptors via aiohttp test apps and can hit OSError: [Errno 24] Too many open files under a low ulimit -n. ulimit -n 4096 makes it green; unrelated to this change.

Differential value (related open PRs)

Two older PRs propose per-platform model routing but neither closes #34612:

This PR is intentionally minimal and surgical to the issue's scope: it adds the opt-in dimension and wires only the API server. If the maintainers prefer the broader provider-aware design, the cleaner consolidation is to land #11439 and add the one-line api_server opt-in from here. Happy to rebase onto whichever direction is chosen.

…earch#34612)

api_server._create_agent always resolved model.default; _resolve_gateway_model() had no platform dimension, so the HTTP API server could not run a cheaper/faster model than CLI/Discord/Telegram.

Add an opt-in platform param to _resolve_gateway_model(): when supplied and platform_models.<platform> exists in config.yaml, that model wins over model.default. Every existing call site omits the arg and is byte-for-byte unchanged; only api_server._create_agent opts in. Override accepts a bare string or a {default|model} mapping; provider/credentials still come from the global runtime config.

Closes NousResearch#34612
@alt-glitch alt-glitch added type/feature New feature or request P3 Low — cosmetic, nice to have comp/gateway Gateway runner, session dispatch, delivery labels May 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/gateway Gateway runner, session dispatch, delivery P3 Low — cosmetic, nice to have type/feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

API server ignores per-platform model config (no way to run api_server on a different model than the global default)

2 participants