Skip to content

fix: persist finalized LLM hook responses#44253

Open
Rivuza wants to merge 1 commit into
NousResearch:mainfrom
Rivuza:fix/persist-transformed-llm-output
Open

fix: persist finalized LLM hook responses#44253
Rivuza wants to merge 1 commit into
NousResearch:mainfrom
Rivuza:fix/persist-transformed-llm-output

Conversation

@Rivuza

@Rivuza Rivuza commented Jun 11, 2026

Copy link
Copy Markdown

Summary

  • Delay turn persistence until after transform_llm_output and post_llm_call finalization
  • Sync the finalized assistant text back into the last assistant message before JSON/SQLite persistence, external memory sync, and result return
  • Honor post_llm_call response override return values such as {"response": "..."}
  • Add regression coverage for transform and post-hook persistence invariants

Test Plan

  • /home/rivuza/hermes-agent/.venv/bin/python -m pytest tests/agent/test_turn_finalizer.py -q
  • /home/rivuza/hermes-agent/.venv/bin/python -m pytest tests/agent/test_turn_finalizer.py tests/agent/test_shell_hooks.py tests/agent/test_plugin_llm.py -q

Closes #44239
Closes #14894

Move turn persistence until after transform_llm_output and post_llm_call finalization, then sync the finalized assistant text back into the durable message history. This keeps result messages, JSON/SQLite session persistence, next-turn replay, and external memory sync aligned with the user-visible response.\n\nAlso honor post_llm_call response override return values and add regression coverage for transform_llm_output and post_llm_call persistence.\n\nCloses NousResearch#44239\nCloses NousResearch#14894
@alt-glitch alt-glitch added type/bug Something isn't working P3 Low — cosmetic, nice to have comp/agent Core agent loop, run_agent.py, prompt builder labels Jun 11, 2026
@liuhao1024

Copy link
Copy Markdown
Contributor

Verification: structural review — clean ✅

Reviewed the full diff (2 files, +202/-6). Checked:

  1. Persist ordering_persist_session moved AFTER transform_llm_output AND post_llm_call hooks, so hook-modified responses are persisted to both session log and SQLite DB. Previously, hooks could override final_response but the pre-hook version was persisted.
  2. _sync_final_response_to_last_assistant — called twice (before post_llm_call + after), ensuring the in-memory assistant message matches the current final_response at each stage. Walks backward to find the last assistant message — safe for multi-turn.
  3. post_llm_call response override — iterates _post_results from invoke_hook, extracts overrides via _extract_post_llm_response_override (handles str, dict with response/override_response/assistant_response keys). Breaks on first override — intentional (first plugin wins).
  4. _response_transformed flag — set in post_llm_call override path, consumed in return value (line 336 in current code). Dead variable concern eliminated.
  5. No double-persist_persist_session is called exactly once, after all hooks complete. The pre-hook call was removed.

No issues found. Test coverage verifies both transform_llm_output and post_llm_call override paths persist correctly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/agent Core agent loop, run_agent.py, prompt builder P3 Low — cosmetic, nice to have type/bug Something isn't working

Projects

None yet

3 participants