Skip to content

fix(memory): skip external-provider sync on interrupted turns (#15218)#15249

Closed
briandevans wants to merge 1 commit into
NousResearch:mainfrom
briandevans:fix/memory-sync-skip-interrupted
Closed

fix(memory): skip external-provider sync on interrupted turns (#15218)#15249
briandevans wants to merge 1 commit into
NousResearch:mainfrom
briandevans:fix/memory-sync-skip-interrupted

Conversation

@briandevans

Copy link
Copy Markdown
Contributor

What does this PR do?

Fixes `#15218`. `run_conversation` was calling `memory_manager.sync_all(original_user_message, final_response)` at the end of every turn where both args were present, without considering the `interrupted` local flag. Result: an external memory backend received partial assistant output, aborted tool chains, or mid-stream resets as durable conversational truth — downstream recall then treated the not-yet-real state as if the user had seen it complete, poisoning the trust boundary between "what the user took away from the turn" and "what Hermes was in the middle of producing when the interrupt hit".

Reporter's minimal-fix suggestion was to gate on `not was_interrupted`; I implemented that with a small refactor that also makes the logic testable in isolation.

Fix

Extracted the inline sync block in `run_conversation` into a new private method on `AIAgent`:

```python
def _sync_external_memory_for_turn(
self,
*,
original_user_message: Any,
final_response: Any,
interrupted: bool,
) -> None:
if interrupted:
return
if not (self._memory_manager and final_response and original_user_message):
return
try:
self._memory_manager.sync_all(original_user_message, final_response)
self._memory_manager.queue_prefetch_all(original_user_message)
except Exception:
pass
```

Why a helper instead of a one-line guard at the call site?

  • The interrupt guard becomes a single visible check at the top of the method instead of hidden in a boolean-and at the call site
  • Gives tests a clean seam to assert on — the pre-fix layout buried the logic inside the 3,000-line `run_conversation` function where no focused test could reach it
  • Zero behavior change for the positive path
  • The prefetch side-effect is also gated on the same interrupt flag — the user's next message is almost certainly a retry of the same intent, and a prefetch keyed on the interrupted turn would fire against stale context

Skip conditions, encoded explicitly

Condition Outcome Why
`interrupted=True` skip entirely The #15218 fix. Applies even if both strings happen to be populated — an interrupt may have landed between a streamed reply and the next tool call, so the strings-on-disk lie
No memory manager skip Pre-fix behaviour preserved
No `final_response` skip Pre-fix behaviour — tool-only turns that never resolved
No `original_user_message` skip Pre-fix behaviour — system-initiated refresh, not a user turn
`sync_all` or `queue_prefetch_all` raises swallow Pre-fix behaviour — external memory providers are strictly best-effort; a misconfigured backend must not block the user from seeing their response

Related Issue

Fixes #15218

Type of Change

  • 🐛 Bug fix (non-breaking change that fixes an issue)
  • ♻️ Refactor (no behavior change) — extracted inline sync block into a helper
  • ✅ Tests (adding or improving test coverage)

Test plan

  • 16 new tests in `tests/run_agent/test_memory_sync_interrupted.py` — all green on py3.11 venv
  • Existing `test_memory_provider_init.py` still green — the refactor preserves the positive path exactly
  • Import smoke check on `run_agent` module
  • No pre-existing tests assert on `memory_manager.sync_all` — grep -rn confirmed the seam is new

Test coverage detail

`TestSyncExternalMemoryForTurn` (16 tests):

  • `test_interrupted_turn_does_not_sync` — core fix
  • `test_interrupted_turn_skips_even_when_response_is_full` — a long seemingly-complete response is still partial if `interrupted=True`; the strings-on-disk lie
  • `test_completed_turn_syncs_and_queues_prefetch` — regression guard for the positive path; `sync_all` AND `queue_prefetch_all` both called with the right args
  • `test_no_final_response_skips` / `test_no_original_user_message_skips` / `test_no_memory_manager_is_a_no_op` — pre-fix skip paths preserved
  • `test_sync_exception_is_swallowed` / `test_prefetch_exception_is_swallowed` — best-effort invariant
  • `test_sync_matrix` — 8-case parametrised matrix over (interrupted × final_response × user_message), asserts sync fires iff `interrupted=False` AND both strings are non-empty

Not in scope

  • Exposing a structured turn-status to memory providers so they can make their own skip decisions — the reporter mentioned this as an alternative; bigger API surface, saving it for a follow-up if maintainers want it
  • Add write-origin metadata for external memory provider writes #15219 (write-origin metadata for memory writes) — same neighbourhood, separate change; the reporter filed both at once

…search#15218)

``run_conversation`` was calling ``memory_manager.sync_all(
original_user_message, final_response)`` at the end of every turn
where both args were present.  That gate didn't consider the
``interrupted`` local flag, so an external memory backend received
partial assistant output, aborted tool chains, or mid-stream resets as
durable conversational truth.  Downstream recall then treated the
not-yet-real state as if the user had seen it complete, poisoning the
trust boundary between "what the user took away from the turn" and
"what Hermes was in the middle of producing when the interrupt hit".

Extracted the inline sync block into a new private method
``AIAgent._sync_external_memory_for_turn(original_user_message,
final_response, interrupted)`` so the interrupt guard is a single
visible check at the top of the method instead of hidden in a
boolean-and at the call site.  That also gives tests a clean seam to
assert on — the pre-fix layout buried the logic inside the 3,000-line
``run_conversation`` function where no focused test could reach it.

The new method encodes three independent skip conditions:

  1. ``interrupted`` → skip entirely (the NousResearch#15218 fix).  Applies even
     when ``final_response`` and ``original_user_message`` happen to
     be populated — an interrupt may have landed between a streamed
     reply and the next tool call, so the strings on disk are not
     actually the turn the user took away.
  2. No memory manager / no final_response / no user message →
     preserve existing skip behaviour (nothing new for providerless
     sessions, system-initiated refreshes, tool-only turns that never
     resolved, etc.).
  3. Sync_all / queue_prefetch_all exceptions → swallow.  External
     memory providers are strictly best-effort; a misconfigured or
     offline backend must never block the user from seeing their
     response.

The prefetch side-effect is gated on the same interrupt flag: the
user's next message is almost certainly a retry of the same intent,
and a prefetch keyed on the interrupted turn would fire against stale
context.

### Tests (16 new, all passing on py3.11 venv)

``tests/run_agent/test_memory_sync_interrupted.py`` exercises the
helper directly on a bare ``AIAgent`` (``__new__`` pattern that the
interrupt-propagation tests already use).  Coverage:

- Interrupted turn with full-looking response → no sync (the fix)
- Interrupted turn with long assistant output → no sync (the interrupt
  could have landed mid-stream; strings-on-disk lie)
- Normal completed turn → sync_all + queue_prefetch_all both called
  with the right args (regression guard for the positive path)
- No final_response / no user_message / no memory manager → existing
  pre-fix skip paths still apply
- sync_all raises → exception swallowed, prefetch still attempted
- queue_prefetch_all raises → exception swallowed after sync succeeded
- 8-case parametrised matrix across (interrupted × final_response ×
  original_user_message) asserts sync fires iff interrupted=False AND
  both strings are non-empty

Closes NousResearch#15218

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Copilot AI review requested due to automatic review settings April 24, 2026 16:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes #15218 by preventing external memory providers from persisting interrupted (partial/aborted) turns as durable conversation history.

Changes:

  • Extracts the external-memory sync logic from run_conversation into AIAgent._sync_external_memory_for_turn(...).
  • Gates both sync_all(...) and queue_prefetch_all(...) on interrupted=False while preserving existing skip conditions and best-effort exception swallowing.
  • Adds a focused regression test suite covering interrupted vs completed turns, skip paths, and exception-safety.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
run_agent.py Adds _sync_external_memory_for_turn and routes per-turn external memory sync through it, skipping interrupted turns.
tests/run_agent/test_memory_sync_interrupted.py Adds regression tests ensuring interrupted turns do not sync/prefetch while completed turns still do, including exception-safety.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@alt-glitch alt-glitch added type/bug Something isn't working P1 High — major feature broken, no workaround comp/agent Core agent loop, run_agent.py, prompt builder tool/memory Memory tool and memory providers labels Apr 24, 2026
@teknium1 teknium1 closed this Apr 24, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Salvaged via #15395 — your commit was cherry-picked onto current main with authorship preserved in git log. Thanks @briandevans!

#15395

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent loop, run_agent.py, prompt builder P1 High — major feature broken, no workaround tool/memory Memory tool and memory providers type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

External memory sync should skip interrupted turns

4 participants