Skip to content

fix: read max_tokens from custom_providers per-model config#28988

Closed
zccyman wants to merge 1 commit into
NousResearch:mainfrom
atyou2happy:fix/custom-provider-max-tokens
Closed

fix: read max_tokens from custom_providers per-model config#28988
zccyman wants to merge 1 commit into
NousResearch:mainfrom
atyou2happy:fix/custom-provider-max-tokens

Conversation

@zccyman

@zccyman zccyman commented May 20, 2026

Copy link
Copy Markdown
Contributor

Summary

Fixes #28046max_tokens configured under custom_providers[].models.<model>.max_tokens was silently ignored, always defaulting to 4096.

The codebase already had get_custom_provider_context_length() for reading per-model context_length from custom_providers, but no equivalent for max_tokens.

Changes

  1. Extract get_custom_provider_model_field() — generic lookup helper that searches custom_providers entries for any per-model field. Replaces the inline logic in get_custom_provider_context_length().

  2. Add get_custom_provider_max_tokens() — thin wrapper over the generic helper, symmetric to get_custom_provider_context_length().

  3. Read max_tokens in agent_init.py — after the existing context_length custom_providers lookup, adds a symmetric block that calls get_custom_provider_max_tokens() when agent.max_tokens is None.

  4. 10 regression tests — full coverage for the new lookup: matching, trailing-slash insensitivity, zero/negative rejection, string coercion, coexistence with context_length, first-match-wins, None inputs.

Backward Compatibility

  • get_custom_provider_context_length() signature unchanged — custom_providers remains a positional arg (3 existing tests pass without modification)
  • New code only runs when agent.max_tokens is None (no override from top-level config or constructor) — pure additive fallback
  • 22/22 tests pass (12 existing context_length + 10 new max_tokens)

Root Cause Pattern

This is the same pattern as #28961 (pre_tool_call missing session_id) and #28984 (Typed Plugin Hook Protocol FR): configuration/state flows through call chains without schema enforcement, so adding a new config field requires manually updating every bridge — and omissions are silent.

Previously, only context_length was read from custom_providers per-model
config. max_tokens was silently ignored, always falling back to 4096.

Changes:
- Extract get_custom_provider_model_field() as generic lookup helper
- Add get_custom_provider_max_tokens() symmetric to context_length
- Read custom_providers max_tokens in agent_init.py after context_length
- Add 10 regression tests for max_tokens lookup

Closes NousResearch#28046
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent loop, run_agent.py, prompt builder comp/cli CLI entry point, hermes_cli/, setup wizard area/config Config system, migrations, profiles duplicate This issue or pull request already exists labels May 20, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Duplicate of #28142 — same fix (read max_tokens from custom_providers per-model config in agent_init.py). Also see competing PRs #28154 (already marked dupe of #28142) and #28786.

@zccyman

zccyman commented May 20, 2026

Copy link
Copy Markdown
Contributor Author

Superseded by PR #28995 which includes this fix plus the Config-Runtime Contract Registry (Phase 1).

@zccyman zccyman closed this May 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/config Config system, migrations, profiles comp/agent Core agent loop, run_agent.py, prompt builder comp/cli CLI entry point, hermes_cli/, setup wizard duplicate This issue or pull request already exists P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: max_tokens not read from custom_providers per-model config, always defaults to 4096

2 participants