Title: holographic memory auto_extract saves raw user messages verbatim instead of extracting preferences
Summary
The auto_extract feature on the holographic memory plugin matches user messages against simple "I prefer / I like / I want" regex patterns at session end and writes the entire matched message (truncated to 400 chars) into fact_store as a fact. There's no extraction, summarization, or synthesis — just a pattern match followed by a raw dump of conversational text.
The result is that fact_store accumulates entries that are user messages verbatim, not facts. Subsequent holographic recall surfaces these conversational snippets as if they were learned preferences, polluting downstream context with chat fragments.
Repro
- Enable
plugins.hermes-memory-store.auto_extract: true in config.yaml with the holographic memory provider configured.
- Send any user message containing a phrase that matches one of the auto-extractor's regexes — e.g. "I like the new cleanup approach better, can we just write to /tmp instead?"
- Let the session end (so
on_session_end fires).
- Inspect
memory_store.db:
sqlite3 ~/.hermes/memory_store.db "SELECT fact_id, category, content FROM facts ORDER BY fact_id DESC LIMIT 5"
Expected
A fact entry should reflect a synthesized preference, e.g. prefers systemd-tmpfiles over alternative cleanup approaches, or no fact should be saved if the matched phrase is conversational filler rather than a preference statement.
Actual
The entire user message body is stored verbatim as category=user_pref:
fact #N | user_pref | I like the new cleanup approach better, can we just write to /tmp instead?
These contaminating entries have empty tags and helpful_count=0, but holographic recall still surfaces them as semantically-related "facts" in subsequent sessions.
Real entries from one test session after auto_extract: true was enabled (synthetic examples representative of the failure mode):
I like that, sounds good
I want you to add tests for the new endpoint
i like the approach, would you set it up on the staging server?
I always check git status before committing
None of these are facts. They're conversational replies that happen to contain the literal substring "I like" / "I want" / "I always."
Suspected cause
plugins/memory/holographic/__init__.py::_auto_extract_facts:
for pattern in _PREF_PATTERNS:
if pattern.search(content):
try:
self._store.add_fact(content[:400], category="user_pref")
extracted += 1
except Exception:
pass
break
content[:400] is the unmodified user message — the regex (.+) capture group is computed but never used; the call falls back to writing the whole message body. There is no extraction step (LLM call, span-extraction, or even a simple match.group(1) substitution) between pattern match and add_fact.
The patterns themselves are also too permissive for a verbatim-dump approach. \bI\s+like\s+(.+) matches every conversational "I like that idea, let's…" reply, which makes every back-and-forth turn a candidate for ingestion.
Possible fixes
In rough order of effort:
- Use the regex capture group:
match.group(1) instead of content[:400], so at minimum only the captured remainder is stored. This is a one-line change but doesn't solve the false-positive rate.
- Tighten the patterns to match clean preference statements only — e.g. require the message to BE a preference statement (start-of-string anchored, no trailing question marks, length cap on the captured span). Reduces noise but still verbatim.
- Replace the regex extractor with a small LLM summarization pass (e.g. via the auxiliary
compression slot) that produces a synthesized fact like User prefers X for Y from the matched message. Highest cost, highest signal.
Option (1) is the smallest fix and would be a clear improvement; (3) is what the feature was likely intended to be.
Workaround
auto_extract: false (the default) disables the behavior entirely. Manual fact_store add calls from the agent still work and produce clean entries. Existing contamination can be cleaned up with a SQL filter on category IN ('user_pref','project') AND tags='' AND helpful_count=0 plus a regex check for conversational openers.
Environment
- Hermes Agent v0.13.0 (v2026.5.7)
- holographic memory provider, default config except
auto_extract: true
- Reproducible with the in-tree code as of
main; no out-of-tree patches required.
Title: holographic memory
auto_extractsaves raw user messages verbatim instead of extracting preferencesSummary
The
auto_extractfeature on the holographic memory plugin matches user messages against simple "I prefer / I like / I want" regex patterns at session end and writes the entire matched message (truncated to 400 chars) intofact_storeas a fact. There's no extraction, summarization, or synthesis — just a pattern match followed by a raw dump of conversational text.The result is that
fact_storeaccumulates entries that are user messages verbatim, not facts. Subsequent holographic recall surfaces these conversational snippets as if they were learned preferences, polluting downstream context with chat fragments.Repro
plugins.hermes-memory-store.auto_extract: trueinconfig.yamlwith the holographic memory provider configured.on_session_endfires).memory_store.db:Expected
A fact entry should reflect a synthesized preference, e.g.
prefers systemd-tmpfiles over alternative cleanup approaches, or no fact should be saved if the matched phrase is conversational filler rather than a preference statement.Actual
The entire user message body is stored verbatim as
category=user_pref:These contaminating entries have empty
tagsandhelpful_count=0, but holographic recall still surfaces them as semantically-related "facts" in subsequent sessions.Real entries from one test session after
auto_extract: truewas enabled (synthetic examples representative of the failure mode):I like that, sounds goodI want you to add tests for the new endpointi like the approach, would you set it up on the staging server?I always check git status before committingNone of these are facts. They're conversational replies that happen to contain the literal substring "I like" / "I want" / "I always."
Suspected cause
plugins/memory/holographic/__init__.py::_auto_extract_facts:content[:400]is the unmodified user message — the regex(.+)capture group is computed but never used; the call falls back to writing the whole message body. There is no extraction step (LLM call, span-extraction, or even a simplematch.group(1)substitution) between pattern match andadd_fact.The patterns themselves are also too permissive for a verbatim-dump approach.
\bI\s+like\s+(.+)matches every conversational "I like that idea, let's…" reply, which makes every back-and-forth turn a candidate for ingestion.Possible fixes
In rough order of effort:
match.group(1)instead ofcontent[:400], so at minimum only the captured remainder is stored. This is a one-line change but doesn't solve the false-positive rate.compressionslot) that produces a synthesized fact likeUser prefers X for Yfrom the matched message. Highest cost, highest signal.Option (1) is the smallest fix and would be a clear improvement; (3) is what the feature was likely intended to be.
Workaround
auto_extract: false(the default) disables the behavior entirely. Manualfact_store addcalls from the agent still work and produce clean entries. Existing contamination can be cleaned up with a SQL filter oncategory IN ('user_pref','project') AND tags='' AND helpful_count=0plus a regex check for conversational openers.Environment
auto_extract: truemain; no out-of-tree patches required.