fix(run_agent): honor model.context_length override for sub-64K models#11097
Open
c0nSpIc0uS7uRk3r wants to merge 1 commit into
Open
fix(run_agent): honor model.context_length override for sub-64K models#11097c0nSpIc0uS7uRk3r wants to merge 1 commit into
c0nSpIc0uS7uRk3r wants to merge 1 commit into
Conversation
Commit c8aff74 ("prevent agent from stopping mid-task") added a MINIMUM_CONTEXT_LENGTH=64K guard that rejects agent init for any model whose native context window is smaller. The ValueError it raises documents an escape hatch: "or set model.context_length in config.yaml to override" But the guard fires unconditionally — it never checks whether the operator set `model.context_length` in config.yaml. The promised escape hatch is cosmetic. Fix: add `and _config_context_length is None` to the gate condition. `_config_context_length` is already a local variable in the same `__init__` method, populated from the config.yaml value. Effect: operators running against models with sub-64K native context (e.g. Qwen 3.5 35B-A3B at 32K) can now explicitly opt in to the smaller window via `model.context_length: 32000` in config.yaml, matching the behavior the existing error message already promises. Default behavior for operators who don't set the override is unchanged. Repro: 1. In config.yaml: `model.context_length: 32000` 2. Run `hermes chat -q "hello" -Q` against a 32K model 3. Before: ValueError at init. After: runs normally.
8 tasks
Contributor
|
This is an important fix. Can we get this out in the next release? my hermes is bricked until then since i run a local model with less context window. The error message asks to set |
Collaborator
This was referenced Apr 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Commit c8aff74 ("prevent agent from stopping mid-task") added a
MINIMUM_CONTEXT_LENGTH=64Kguard inAIAgent.__init__that rejects agent initialization for any model whose native context window is smaller than 64K. The raisedValueErrorexplicitly documents an escape hatch:But the guard fires unconditionally — it never checks whether the operator actually set
model.context_lengthinconfig.yaml. The promised escape hatch is cosmetic.Fix
Add
and _config_context_length is Noneto the gate condition._config_context_lengthis already a local variable in the same__init__method, populated from the config.yaml value, so no additional plumbing is needed.Behavior change
model.context_length): unchanged — guard still fires, agent still refuses to start against sub-64K models.model.context_lengthin config.yaml, accepting sub-64K risk): agent now starts, matching what the error message already promises.Repro
config.yaml:hermes chat -q "hello" -Qagainst a model with a 32K native context (e.g. Qwen 3.5 35B-A3B).ValueError: Model ... has a context window of 32,000 tokens, which is below the minimum 65,536...Related
Addresses Bug 1 from #11096 (combined issue covering three v0.9.0 rough edges).