fix(agent): preserve config context_length when fallback/switch uses same model#32452
Open
xianshishan wants to merge 1 commit into
Open
fix(agent): preserve config context_length when fallback/switch uses same model#32452xianshishan wants to merge 1 commit into
xianshishan wants to merge 1 commit into
Conversation
…same model When fallback activation or model switch cleared _config_context_length, the subsequent get_model_context_length() call could fall through to DEFAULT_FALLBACK_CONTEXT (256K) if the model wasn't in any static table and probing failed. This caused the status bar to show 256K after compaction interruption + resume, even though the user had configured context_length: 1000000 in config.yaml. Save the original _config_context_length before clearing it, and restore it when the fallback/switched model is the same as the primary — the common case for custom providers where the fallback is the same endpoint accessed via a different credential. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
When a fallback provider is activated or
/modelswitch is triggered,_config_context_lengthis unconditionally cleared (set toNone). The subsequentget_model_context_length()call then resolves the context window from scratch. If the model isn't in any static table (DEFAULT_CONTEXT_LENGTHS) and live probing fails, the resolution falls through all 10 steps toDEFAULT_FALLBACK_CONTEXT(256K).This causes the status bar to show
97.9K/256Keven though the user configuredcontext_length: 1000000in config.yaml — most visible after a compaction interruption + resume cycle.Root Cause
In
try_activate_fallback()(chat_completion_helpers.py:1013) andswitch_model()(agent_runtime_helpers.py:1367):Then
get_model_context_length(config_context_length=None, ...)is called. Without the explicit config override, resolution falls through to 256K for custom endpoints that don't advertise context_length via/models.Fix
Save the original
_config_context_lengthbefore clearing it. When the fallback/switched model is the same as the primary (common for custom providers where the fallback is the same endpoint accessed via a different credential), restore the saved override so the resolution chain receives it at Step 0:When the fallback model is different from the primary, the override is NOT restored — the new model should resolve its own context window.
Changes
agent/chat_completion_helpers.py_config_context_lengthbefore fallback activation; restore it for same-model fallbackagent/agent_runtime_helpers.pyswitch_model()pathTest plan
model.context_lengthrestore_primary_runtime()continues to work correctly (restores from snapshot)Closes #32423