fix(cli): reinitialize context compressor on /new to pick up updated context length#31067
Open
gneiss09 wants to merge 1 commit into
Open
fix(cli): reinitialize context compressor on /new to pick up updated context length#31067gneiss09 wants to merge 1 commit into
gneiss09 wants to merge 1 commit into
Conversation
…updated context length The in-memory _endpoint_model_metadata_cache has a 300s TTL, so stale values persisted across /new commands within that window. Clear the cache for LM Studio before re-creating the ContextCompressor so it queries the live server and reads the new context_length.
jsboige
approved these changes
May 23, 2026
jsboige
left a comment
There was a problem hiding this comment.
Targeted fix for stale context window display after /new. The context compressor retained its old context_length across session resets — now reinitialized with fresh provider probing. Also clears LM Studio endpoint metadata cache (300s TTL) to force re-read. Wrapped in try/except so session reset never fails. LGTM.
1 task
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When running Hermes Agent against LM Studio (or similar local providers), the CLI status bar displays stale context window information after
/new. Two root causes:reset_session_state()does not recreatecontext_compressor— the compressor retains its oldcontext_lengthvalue across sessions, so the TUI/CLI shows the wrong context window size.In-memory endpoint metadata cache has a 300s TTL —
_endpoint_model_metadata_cacheinagent/model_metadata.pycaches model info (includingconfig.context_length) per base URL. Within the 5-minute TTL, stale values mask updated context lengths even if the provider changes its configuration.Fix
On
/new, before callingreset_session_state():If the provider is LM Studio, clear the in-memory endpoint metadata cache entry for that base URL (both
_endpoint_model_metadata_cacheand_endpoint_model_metadata_cache_time). This forces a fresh query to/api/v1/modelson next use.Re-create the
ContextCompressorwithconfig_context_length=None, letting it probe the provider fresh instead of reusing the stale value from the old instance.Both operations are wrapped in a try/except — failures are non-fatal and do not block session creation.
Changes
cli.py: Added context compressor reinitialization logic innew_session()(~37 lines)ContextCompressorwith fresh config probeTesting
/newnow correctly shows the new context length in the TUI status bar.lmstudioprovider).