fix(agent): clear stale config context_length on model switch#22387
fix(agent): clear stale config context_length on model switch#22387AgentArcLab wants to merge 1 commit into
Conversation
When switching models via /model, AIAgent._config_context_length was never cleared, so the new model inherited the previous model's context window instead of auto-detecting the correct one via get_model_context_length(). Clear _config_context_length to None before the runtime field swap so the full resolution chain (custom_providers per-model, endpoint probe, models.dev, etc.) is re-evaluated for the newly selected model. Closes NousResearch#21509
738538b to
e25ac0e
Compare
|
Thanks for the pointer to #11438! You're right that the root cause is the same — stale A couple of notes on where this PR differs:
I noticed that #11438, #15787, and #13052 were all closed without merging, so the bug remains unfixed on main. Happy to adjust the approach if there's a preferred direction — just want to make sure this doesn't fall through the cracks again 🙂 |
|
Merged via PR #24724 (cherry-picked onto current main with your authorship preserved). Thanks for the contribution! |
Problem
When switching models (via
/modelor fallback),AIAgent._config_context_lengthis never cleared, so the new model inherits the previous model's context window instead of auto-detecting the correct one viaget_model_context_length().Root Cause
AIAgent._config_context_lengthis set once during__init__frommodel.context_lengthin config.yaml. This value is never cleared in either:switch_model()— the/modelcommand handler_try_activate_fallback()— the failover path on primary model failureSince
get_model_context_length()checks this value at resolution step 0 and returns it immediately, the new/fallback model inherits the old override instead of going through the full resolution chain (custom_providers per-model, endpoint metadata, models.dev, etc.).Fix
Clear
self._config_context_length = Nonein both code paths, before the runtime field swap. This allowsget_model_context_length()to skip the stale step-0 override and properly resolve the context window for the newly selected model through the standard chain (step 0b:custom_providersper-model, then endpoint probe, models.dev, etc.).Testing
model.context_length: 1048576in config.yaml for model A (1M context)/model— before fix: still shows 1M; after fix: correctly shows 200KCloses #21509