Skip to content

fix(agent): preserve config context_length when fallback/switch uses same model#32452

Open
xianshishan wants to merge 1 commit into
NousResearch:mainfrom
xianshishan:fix/context-length-fallback-256k
Open

fix(agent): preserve config context_length when fallback/switch uses same model#32452
xianshishan wants to merge 1 commit into
NousResearch:mainfrom
xianshishan:fix/context-length-fallback-256k

Conversation

@xianshishan

Copy link
Copy Markdown

Summary

When a fallback provider is activated or /model switch is triggered, _config_context_length is unconditionally cleared (set to None). The subsequent get_model_context_length() call then resolves the context window from scratch. If the model isn't in any static table (DEFAULT_CONTEXT_LENGTHS) and live probing fails, the resolution falls through all 10 steps to DEFAULT_FALLBACK_CONTEXT (256K).

This causes the status bar to show 97.9K/256K even though the user configured context_length: 1000000 in config.yaml — most visible after a compaction interruption + resume cycle.

Root Cause

In try_activate_fallback() (chat_completion_helpers.py:1013) and switch_model() (agent_runtime_helpers.py:1367):

agent._config_context_length = None  # ← cleared unconditionally

Then get_model_context_length(config_context_length=None, ...) is called. Without the explicit config override, resolution falls through to 256K for custom endpoints that don't advertise context_length via /models.

Fix

Save the original _config_context_length before clearing it. When the fallback/switched model is the same as the primary (common for custom providers where the fallback is the same endpoint accessed via a different credential), restore the saved override so the resolution chain receives it at Step 0:

_effective_config_ctx = getattr(agent, "_config_context_length", None)
if _saved_config_ctx and fb_model == old_model:
    _effective_config_ctx = _saved_config_ctx

When the fallback model is different from the primary, the override is NOT restored — the new model should resolve its own context window.

Changes

File Change
agent/chat_completion_helpers.py Save _config_context_length before fallback activation; restore it for same-model fallback
agent/agent_runtime_helpers.py Same fix for the switch_model() path

Test plan

  • 401 agent tests pass
  • Same-model fallback preserves user's model.context_length
  • Different-model fallback resolves fresh context_length (doesn't inherit override)
  • restore_primary_runtime() continues to work correctly (restores from snapshot)

Closes #32423

…same model

When fallback activation or model switch cleared _config_context_length,
the subsequent get_model_context_length() call could fall through to
DEFAULT_FALLBACK_CONTEXT (256K) if the model wasn't in any static table
and probing failed. This caused the status bar to show 256K after
compaction interruption + resume, even though the user had configured
context_length: 1000000 in config.yaml.

Save the original _config_context_length before clearing it, and restore
it when the fallback/switched model is the same as the primary — the
common case for custom providers where the fallback is the same endpoint
accessed via a different credential.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Context window changes to 256K after interrupted compaction and resume

1 participant