fix(agent): forward custom_providers to context-length probe on fallback activation#25554
Closed
AhmetArif0 wants to merge 1 commit into
Closed
Conversation
…ack activation _try_activate_fallback() updates the context compressor for the fallback model but called get_model_context_length() without custom_providers. Users who pin context_length on a model in custom_providers (e.g. a 1M- context proxy behind a custom endpoint) saw the compressor silently fall back to the 256K default when the primary provider failed over, shifting the compression threshold to the wrong value for the remainder of the session. Fix: mirror the switch_model() pattern — re-read custom_providers from live config and forward the list to get_model_context_length, matching the approach used by _check_compression_model_feasibility() (PR NousResearch#25494).
Collaborator
Contributor
|
This appears to be implemented on current main. Automated hermes-sweeper review evidence:
The prior discussion noting this as the remaining fallback activation gap was useful context; main now has the corresponding fix in the post-refactor location. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Root cause:
_try_activate_fallback()updates the context compressor when switching to the fallback model, but calledget_model_context_length()withoutcustom_providers. Users who pincontext_lengthon a model viacustom_providersinconfig.yaml(e.g. a 1M-context proxy behind a custom endpoint) saw the compressor silently fall through to the 256K default when the primary provider failed over — shifting the compression threshold to the wrong value for the remainder of the session.Fix: Mirror the
switch_model()pattern — re-readcustom_providersfrom live config and forward the list toget_model_context_length(). This is consistent with the approach used by_check_compression_model_feasibility()(PR #25494) andswitch_model()(which both correctly forwardcustom_providers). The fallback activation path was the remaining gap.The three call sites now behave consistently:
switch_model()→ forwards re-read_sm_custom_providers✅_check_compression_model_feasibility()→ forwardsself._custom_providers✅ (PR fix(auxiliary): forward custom_providers to compression model context-length detection (#25359) #25494)_try_activate_fallback()→ forwards re-read_fb_custom_providers✅ (this PR)Related Issue
Closes #8785
Type of Change
Changes Made
run_agent.py: re-readcustom_providersfrom live config and forward toget_model_context_length()in_try_activate_fallback()(+10 lines)tests/run_agent/test_fallback_model.py: regression test verifyingcustom_providersis passed through when a context compressor is present (+50 lines)How to Test
Checklist