fix(fallback): forward custom_providers to fallback model context-length detection#26043
Closed
PaTTeeL wants to merge 1 commit into
Closed
fix(fallback): forward custom_providers to fallback model context-length detection#26043PaTTeeL wants to merge 1 commit into
PaTTeeL wants to merge 1 commit into
Conversation
…gth detection The same root cause as the auxiliary compression fix (commit 7becb19): get_model_context_length() is called without custom_providers, so per-model context_length overrides are silently skipped. The fallback activation path (_try_activate_fallback) had the same missing parameter. When the agent switches to a fallback provider, the fallback model would use the models.dev value (e.g. 204800 for NVIDIA NIM minimax-m2.7) instead of the user-configured one in custom_providers (e.g. 196608) — a subtle discrepancy that could cause the fallback model to run with an incorrect context window, leading to truncated messages or failed API requests when the model does not support the detected length. Fix: pass self._custom_providers to get_model_context_length() so the fallback path sees the same per-model overrides as the main model path.
Collaborator
|
Duplicate of #25554 — same one-line fix (forward |
Contributor
|
Merged via PR #27247 — your commit was cherry-picked onto current |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a sibling fix to the auxiliary compression fix (PR #25494). The same root cause: get_model_context_length() is called without custom_providers, so per-model context_length overrides in custom_providers are silently skipped. The fallback activation path (_try_activate_fallback) had the same missing parameter.
When the agent switches to a fallback provider, _try_activate_fallback() probes the fallback model's context window via get_model_context_length(). Without custom_providers, it skips step 0b and falls through to models.dev, which may return a different value than what the user configured (e.g. 204800 from models.dev vs 196608 from custom_providers for NVIDIA NIM minimax-m2.7). This can cause the fallback model to run with an incorrect context window, leading to truncated messages or failed API requests.
Fix: added custom_providers=self._custom_providers to the get_model_context_length() call in _try_activate_fallback(), matching what the main model and auxiliary compression paths already do.
Changes: 1 line in run_agent.py
Related to #25494 (same root cause, different call site).