Skip to content

fix(run_agent): use aux provider for compression context length lookup#16918

Closed
rxdxxxx wants to merge 1 commit into
NousResearch:mainfrom
rxdxxxx:fix/bedrock-aux-compression-provider-leak
Closed

fix(run_agent): use aux provider for compression context length lookup#16918
rxdxxxx wants to merge 1 commit into
NousResearch:mainfrom
rxdxxxx:fix/bedrock-aux-compression-provider-leak

Conversation

@rxdxxxx

@rxdxxxx rxdxxxx commented Apr 28, 2026

Copy link
Copy Markdown

Problem

When the main model is configured with AWS Bedrock (provider: bedrock),
self.provider == "bedrock" was being passed unconditionally to
get_model_context_length() for the auxiliary compression model.

get_model_context_length has a hard-intercept at step 1b:

if provider == "bedrock" or base_url.startswith("bedrock-runtime."):
    return get_bedrock_context_length(model)  # short-circuits all other resolution

A non-Bedrock aux model (e.g. deepseek/deepseek-v4-flash on OpenRouter) is
not in the Bedrock static table, so it falls back to
BEDROCK_DEFAULT_CONTEXT_LENGTH = 128_000. This triggers a false feasibility
warning every session start:

⚠ Compression model deepseek/deepseek-v4-flash (openrouter) context is 128,000 tokens,
  but the main model's compression threshold was 800,000 tokens.
  Auto-lowered this session's threshold to 128,000 tokens so compression can run.

…even though deepseek/deepseek-v4-flash natively supports 1,048,576 tokens.

Root Cause

_aux_cfg_provider (the aux model's own provider, resolved via
_resolve_task_provider_model("compression")) was already being fetched just
above the call site — but never passed to get_model_context_length(). Instead,
the main model's self.provider was passed unconditionally.

The invariant that provider must belong to the model being queried — not the
caller's model — is not enforced at the call site today, which is why this leak
is silent.

Fix

Pass _aux_cfg_provider when it is explicitly set (not empty or "auto"),
falling back to the main model's provider only when the aux provider is
unspecified.

# Before
provider=getattr(self, "provider", ""),

# After
provider=(_aux_cfg_provider if _aux_cfg_provider and _aux_cfg_provider != "auto" else getattr(self, "provider", "")),

Impact

The Bedrock hard-intercept is the most visible trigger, but the root cause is
general: any provider whose resolution path diverges at step 1b (Bedrock static
table) or later steps will produce wrong results when a foreign provider is
passed for an aux model.

Affected: main model = Bedrock + aux compression model on any non-Bedrock
provider (OpenRouter, OpenAI, Anthropic, etc.). Other combinations are not
affected today because non-Bedrock providers don't short-circuit at step 1b.

Functional impact is minor (threshold auto-corrects for the session), but the
warning is misleading, fires every session, and causes users to misconfigure
their setup to suppress it.

Why This Is Not a Duplicate of #17460

PR #17460 fixes get_model_context_length() to auto-load custom_providers
when the parameter is None — protecting callers that never passed the
argument. This PR fixes a different parameter on the same call: provider
is present at every call site but is being set to the wrong value.

The two fixes are complementary and non-overlapping:

#17460 This PR (#16918)
Parameter fixed custom_providers=None provider=self.provider (wrong value)
Root cause Missing argument threading Provider ownership violation at call site
Affected scenario custom_providers overrides ignored for aux models Bedrock (and future diverging providers) intercepting aux model resolution

Neither PR makes the other redundant. Both should land.

Testing

All 16 existing compression feasibility tests pass unchanged.

Closes #12977
Related: #13807, #17460

@rxdxxxx rxdxxxx force-pushed the fix/bedrock-aux-compression-provider-leak branch from baee64f to 88482fc Compare April 28, 2026 08:39
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/agent Core agent loop, run_agent.py, prompt builder provider/bedrock AWS Bedrock (boto3, IAM) labels Apr 28, 2026
@rxdxxxx rxdxxxx force-pushed the fix/bedrock-aux-compression-provider-leak branch from 76c4057 to e945211 Compare May 5, 2026 02:03
Each auxiliary model must be resolved with its own provider so that
provider-specific paths (e.g. Bedrock static table, OpenRouter API)
are invoked for the correct client, not inherited from the main model.

When the main model is Bedrock, passing self.provider unconditionally
to get_model_context_length() for the aux model caused the Bedrock
static table hard-intercept (step 1b) to fire for non-Bedrock models,
returning BEDROCK_DEFAULT_CONTEXT_LENGTH=128K instead of the model's
real context window — triggering a false compression warning every session.

Fix: pass _aux_cfg_provider when explicitly set, falling back to
self.provider only when the aux provider is unset or "auto".

Closes NousResearch#12977
Related: NousResearch#13807, NousResearch#17460
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent loop, run_agent.py, prompt builder P2 Medium — degraded but workaround exists provider/bedrock AWS Bedrock (boto3, IAM) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: custom_providers.models.context_length not propagated to auxiliary compression feasibility check

2 participants