Background
- Hindsight 0.5.0+ supports
update_mode='append' and update_mode='replace' with same-document-id delete-then-rebuild semantics.
- The Hermes hindsight plugin currently hardcodes
document_id = f"{self._session_id}-{start_ts}" (see plugins/memory/hindsight/__init__.py:931), which guarantees a new document per session, defeating dedup entirely.
- The plugin never passes
update_mode to the retain call — the parameter exists in Hindsight's MCP/API but is unreachable from the Hermes side.
Symptom
Same configuration change (e.g. "nudge_interval changed from 10 to 20") is retained as separate documents across every session that mentions it. Consolidation then produces conflicting facts because the input observations themselves contain stale duplicates. Mitigated by raising retain_every_n_turns from 1 to 3, but the root cause remains.
Example: In a 27-minute window on a single day, the same config change was retained 16+ times across sessions before throttling was applied.
Proposed Changes
1. Expose update_mode parameter in plugin retain call
- Default to
"replace" (Hindsight default).
- Allow configuration via
~/.hermes/hindsight/config.json (e.g. "update_mode": "replace").
2. Add document_id_strategy config option
| Strategy |
document_id format |
Cost |
Dedup scope |
session (current) |
{session_id}-{start_ts} |
Zero |
None — every session is a new doc |
date |
{bank_id}-{date} |
Near-zero |
Same-day merges; cross-day still duplicates |
topic |
{bank_id}-{topic_hash} |
One extra lightweight LLM call per retain batch |
Same-topic convergence across all sessions |
Recommended default: date — it eliminates the majority of same-day duplicates with zero additional cost and no LLM dependency. The topic strategy is for power users who want maximum dedup.
3. For "topic" strategy: lightweight topic extraction
- Use the auxiliary model (or a dedicated cheap model) to extract a short topic key from the retained content.
- Cache the topic→hash mapping to avoid redundant LLM calls.
- Fall back to
"date" on LLM failure.
Workaround Until Fix
No in-plugin solution. Manual cleanup via Hindsight API (DELETE /documents/{id}) is the only option.
Related
Background
update_mode='append'andupdate_mode='replace'with same-document-id delete-then-rebuild semantics.document_id = f"{self._session_id}-{start_ts}"(seeplugins/memory/hindsight/__init__.py:931), which guarantees a new document per session, defeating dedup entirely.update_modeto the retain call — the parameter exists in Hindsight's MCP/API but is unreachable from the Hermes side.Symptom
Same configuration change (e.g. "nudge_interval changed from 10 to 20") is retained as separate documents across every session that mentions it. Consolidation then produces conflicting facts because the input observations themselves contain stale duplicates. Mitigated by raising
retain_every_n_turnsfrom 1 to 3, but the root cause remains.Example: In a 27-minute window on a single day, the same config change was retained 16+ times across sessions before throttling was applied.
Proposed Changes
1. Expose
update_modeparameter in plugin retain call"replace"(Hindsight default).~/.hermes/hindsight/config.json(e.g."update_mode": "replace").2. Add
document_id_strategyconfig optionsession(current){session_id}-{start_ts}date{bank_id}-{date}topic{bank_id}-{topic_hash}Recommended default:
date— it eliminates the majority of same-day duplicates with zero additional cost and no LLM dependency. Thetopicstrategy is for power users who want maximum dedup.3. For
"topic"strategy: lightweight topic extraction"date"on LLM failure.Workaround Until Fix
No in-plugin solution. Manual cleanup via Hindsight API (
DELETE /documents/{id}) is the only option.Related
retainRolesandretainOverlapTurns(other known gaps between Hindsight API and Hermes plugin)update_modeintroduction