feat(hindsight): default recall_types to observation only#33983
Closed
nicoloboschi wants to merge 1 commit into
Closed
feat(hindsight): default recall_types to observation only#33983nicoloboschi wants to merge 1 commit into
nicoloboschi wants to merge 1 commit into
Conversation
Auto-recall used to surface every fact type Hindsight had on the
session — `world`, `experience`, and `observation`. That triple-ships
the same underlying signal in three different framings: observations
are the concrete events the user said/did/asked, while world and
experience facts are aggregate summaries Hindsight derives from those
exact observations. Including all three burns most of
`recall_max_tokens` on rephrasings, crowds out events the model
actually needs to see, and produces effective duplicates in the
prompt — observations themselves are deduplicated by construction
so observation-only recall is denser per token and closer to
conversational ground truth.
Change
------
- Default `_recall_types = ["observation"]` (was `None`, which
delegated to server-side "return everything").
- `initialize()` now treats a missing `recall_types` config the same
way; also accepts comma-separated strings for parity with `recall_tags`.
- An explicit `recall_types=[]` config falls back to the default rather
than disabling the filter (would silently widen recall vs. the new
default).
- Added to `get_config_schema()` so it's discoverable via `hermes config`.
Per-call `hindsight_recall` tool invocations are unaffected — they
already only forward `types` when the caller passes the argument.
Docs / migration
----------------
plugins/memory/hindsight/README.md grows a "Behavior change" callout
explaining the why (no-duplicates, information-efficient) and how to
restore the legacy broad recall:
"recall_types": "observation,world,experience" # or a JSON list
in `~/.hermes/hindsight/config.json`.
Tests
-----
- `test_default_values` updated for the new default.
- New cases: explicit list override, CSV string accepted, empty list
falls back to default (not "wider than default").
4f53457 to
76eb699
Compare
Collaborator
|
Merged via #34079 — your commit was cherry-picked with authorship preserved (it's now Added one small follow-up commit on top: the description/README said the per-call |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Default
recall_typeson auto-recall is now["observation"](was previously unset → server returned every fact type).Why
Per Hindsight's docs, observations are the consolidated knowledge layer Hindsight builds on top of raw facts: deduplicated beliefs grounded in evidence, refined as new facts arrive, with proof counts and freshness signals. Raw
world/experiencefacts are the individual supporting evidence that feeds them.For per-turn context injection, observations are denser per token and avoid feeding the model multiple raw facts that one observation already summarizes.
Change
_recall_typesdefault in__init__None["observation"]initialize()with norecall_typesconfigNone→ all types["observation"]recall_typesaccepted formatsrecall_tags)recall_types: [])["observation"](avoids silently widening recall vs. the new default)get_config_schema()hermes configPer-call
hindsight_recalltool invocations are unaffected — they already only forwardtypeswhen the caller passes the argument explicitly.Migration
The behavior change is called out in
plugins/memory/hindsight/README.md. To restore the broad recall:(JSON list
["observation", "world", "experience"]works too.)Test plan
uv run pytest tests/plugins/memory/test_hindsight_provider.py -k "recall_types or default_values"— 5/5 passed (updated default test + 4 new cases: explicit list override, CSV string, empty-list fallback, default check).test_get_client_passes_idle_timeout_to_hindsight_embedded) reproduces on stock main — unrelated, env issue withhindsight-client==0.6.1.