Skip to content

fix: cleanly shut down Honcho background workers#11680

Closed
KeroZelvin wants to merge 1 commit into
NousResearch:mainfrom
KeroZelvin:fix/honcho-clean-shutdown
Closed

fix: cleanly shut down Honcho background workers#11680
KeroZelvin wants to merge 1 commit into
NousResearch:mainfrom
KeroZelvin:fix/honcho-clean-shutdown

Conversation

@KeroZelvin

Copy link
Copy Markdown

Bug Description

Hermes could print a valid response and then abort during interpreter teardown in Honcho-enabled CLI runs. This was easiest to reproduce in short-lived one-shot commands like hermes chat -q ....

Root Cause

Hermes's Honcho integration started several daemon background threads for context prefetch, dialectic prefetch, sync, and async writes. During process shutdown, some of those workers could still be doing HTTP/SSL work while Python was already in Py_FinalizeEx, leading to an abort after the response had already been returned.

Fix

  • track Honcho manager-owned background workers and join them during shutdown
  • track provider-owned Honcho workers and stop accepting new background work once shutdown begins
  • route provider shutdown through HonchoSessionManager.shutdown() instead of only flush_all()
  • add regression coverage for manager/provider shutdown behavior

How to Verify

  1. Run ./venv/bin/python -m hermes_cli.main -p 273 chat -q "Reply with exactly OK" repeatedly in a Honcho-enabled profile.
  2. Confirm the command returns RC:0 after printing the response instead of intermittently aborting with a coredump.
  3. Run the targeted Honcho tests below.

Test Plan

  • Added regression test for this bug
  • Existing tests still pass
  • Manual verification of the fix

Commands run:

  • ./venv/bin/python -m pytest tests/honcho_plugin/test_async_memory.py tests/honcho_plugin/test_session.py -q
  • ./venv/bin/python -m py_compile plugins/memory/honcho/session.py plugins/memory/honcho/__init__.py tests/honcho_plugin/test_async_memory.py
  • repeated manual one-shot CLI smoke tests on profile 273

Risk Assessment

Medium — this changes Honcho shutdown/lifecycle behavior, but the blast radius is limited to Hermes's Honcho integration and is covered by targeted regression tests.

@KeroZelvin KeroZelvin marked this pull request as ready for review April 17, 2026 16:29
@alt-glitch alt-glitch added type/bug Something isn't working P2 Medium — degraded but workaround exists comp/plugins Plugin system and bundled plugins labels Apr 24, 2026
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the fix, @KeroZelvin! After reviewing against the current main branch, this fix is already implemented.

This is an automated hermes-sweeper review.

Evidence:

  • plugins/memory/honcho/__init__.py line 1235 — HonchoMemoryProvider.shutdown() already joins _prefetch_thread and _sync_thread (timeout=5.0) and calls self._manager.flush_all()
  • plugins/memory/honcho/session.py line 453 — HonchoSessionManager.shutdown() already uses the _ASYNC_SHUTDOWN sentinel (line 21), drains the queue via flush_all(), and joins the async writer thread with timeout=10
  • Both were introduced in commit 924bc67ee (feat(memory): pluggable memory provider interface..., 2026-04-02) — two weeks before this PR was opened

The shutdown lifecycle this PR describes is fully operational on main. Closing as already implemented.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp/plugins Plugin system and bundled plugins P2 Medium — degraded but workaround exists type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants