fix(gateway): read custom_providers context_length in hygiene compression#4044
Closed
teknium1 wants to merge 1 commit into
Closed
fix(gateway): read custom_providers context_length in hygiene compression#4044teknium1 wants to merge 1 commit into
teknium1 wants to merge 1 commit into
Conversation
…sion Gateway hygiene pre-compression only checked model.context_length from the top-level config, missing per-model context_length defined in custom_providers entries. This caused premature compression for custom provider users (e.g. 128K default instead of 200K configured). The AIAgent's own compressor already reads custom_providers correctly (run_agent.py lines 1171-1189). This adds the same fallback to the gateway hygiene path, running after runtime provider resolution so the base_url is available for matching.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Gateway hygiene pre-compression only checked
model.context_lengthfrom config, missing per-model context_length defined incustom_providersentries. Custom provider users got the 128K default instead of their configured value, causing premature compression.Root cause: The agent's own compressor reads
custom_providerscorrectly (run_agent.py lines 1171-1189), but the gateway's hygiene pre-compression block was missing this fallback.Fix: Adds the same
custom_providerslookup to the hygiene block, positioned after runtime provider resolution sobase_urlis available for matching.Example: A user with
custom_providers.models.Qwen3.5-35B-A3B.context_length: 200000was getting hygiene compression at 85% of 128K = 109K instead of 85% of 200K = 170K.Reported by LauraOP in Discord. 61 gateway compression/hygiene tests pass.