Skip to content

fix(agent): detect custom provider context length for auto-mode compression#14023

Closed
pbastowski wants to merge 2 commits into
NousResearch:mainfrom
pbastowski:main
Closed

fix(agent): detect custom provider context length for auto-mode compression#14023
pbastowski wants to merge 2 commits into
NousResearch:mainfrom
pbastowski:main

Conversation

@pbastowski

Copy link
Copy Markdown
  • Added _get_custom_provider_context_length() helper to query custom_providers config for context length
  • Integrated step 0b in get_model_context_length() to consult custom_providers before probe tiers
  • Preserves auto flexibility (no hardcoded model names) by matching on base_url

…ession

- Added _get_custom_provider_context_length() helper to query custom_providers config for context length
- Integrated step 0b in get_model_context_length() to consult custom_providers before probe tiers
- Preserves auto flexibility (no hardcoded model names) by matching on base_url
@pbastowski pbastowski force-pushed the main branch 2 times, most recently from 27544fd to 66f1ab4 Compare April 22, 2026 13:50
- test_returns_context_length_from_custom_provider
- test_returns_none_when_base_url_does_not_match
- test_returns_provider_level_context_length
- test_case_insensitive_model_match
- test_returns_none_when_import_fails
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent loop, run_agent.py, prompt builder labels Apr 22, 2026
@alt-glitch

Copy link
Copy Markdown
Collaborator

Related to #13807 / #12977 (same root cause for aux compression model). #13813 is an open PR addressing the compression side; this PR addresses the main model_metadata path.

@alt-glitch

Copy link
Copy Markdown
Collaborator

Related to #13807 / #12977 / #13813.

@teknium1

Copy link
Copy Markdown
Contributor

Thank you for this contribution, @pbastowski! This is a well-scoped fix with good test coverage.

After an automated hermes-sweeper review, the fix proposed here has since been implemented on main by a broader refactor:

  • Commit 125de02 (PR fix(context): honor custom_providers context_length on /model switch + bump probe tier to 256K #15844) adds custom_providers= as an explicit kwarg to get_model_context_length() in agent/model_metadata.py, with a new step 0b that calls get_custom_provider_context_length() from hermes_cli/config.py — the same resolution pattern you proposed here.
  • That commit also wires the lookup through five call sites that previously missed it: run_agent.py startup, AIAgent.switch_model(), the display context-length path, and both gateway /model paths.

The companion issue #13807 / #12977 and PR #13813 remain open — a maintainer may want to verify those are fully resolved by #15844 as well.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent loop, run_agent.py, prompt builder P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants