fix(gateway): persist flushed-session set to DB to prevent double flush on restart#3078
Closed
Mibayy wants to merge 1 commit into
Closed
fix(gateway): persist flushed-session set to DB to prevent double flush on restart#3078Mibayy wants to merge 1 commit into
Mibayy wants to merge 1 commit into
Conversation
…sh on restart When the gateway restarted, _pre_flushed_sessions (an in-memory set) was reset to empty, causing the expiry watcher to flush the same expired session a second time — triggering duplicate LLM calls, wasted tokens, and potential memory corruption. Fix: - Add flushed_sessions table to state.db (schema v7 migration) - Load the persisted set into _pre_flushed_sessions on SessionStore init - Persist the marker to DB after each successful proactive flush in run.py - Clean up the DB record when a session is fully reset (discard + remove) Also fix the existing v6 migration that incorrectly bumped schema_version to 5 instead of 6. Additionally, add memory-near-full detection to the flush prompt: when memory is >=90% full, the agent is warned to consolidate or remove stale entries before adding new ones, preventing invalid tool calls at capacity. Fixes NousResearch#3059
thakoreh
added a commit
to thakoreh/hermes-agent
that referenced
this pull request
Mar 26, 2026
The _pre_flushed_sessions in-memory set was lost on gateway restart, causing double-flushing of already-persisted session memories. Persist to a JSON sidecar file and reload on startup. Fixes NousResearch#3078
Contributor
|
Thanks @Mibayy! Your approach of persisting flushed state to SQLite was the most thorough of the four PRs targeting this bug. The fix landed in #4481 using a simpler approach — a |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Fixes #3059.
When the gateway restarted,
_pre_flushed_sessions(an in-memoryset) was reset to empty. The background expiry watcher would then see the same expired session and flush it again, causing:Root cause
Fix
hermes_state.py— Schema v7 migration adds aflushed_sessionstable:add_flushed_session(session_id)— called after each successful proactive flushload_flushed_sessions()— returns the set of already-flushed session IDsremove_flushed_session(session_id)— called when a session is fully resetAlso fixes an existing bug in the v6 migration block that incorrectly bumped
schema_versionto 5 instead of 6.gateway/session.py— OnSessionStoreinit, load the persisted flushed-session set from DB so markers survive restarts. On session reset, callremove_flushed_sessionto clean up the DB record.gateway/run.py— After a successful proactive flush, persist the marker to DB. Also adds memory-near-full detection to the flush prompt: when memory is >=90% full, the agent is warned to consolidate or remove stale entries before adding new ones, preventing invalid tool calls at capacity (the second issue noted in #3059).