fix(tui): lazy MCP connection pool — defer server spawning to first tool use#15797
fix(tui): lazy MCP connection pool — defer server spawning to first tool use#15797thapecroth wants to merge 2 commits into
Conversation
…ool use (NousResearch#15275) Architectural fix inspired by Claude Code's SocketPool pattern. Replaces eager import-time MCP server spawning with lazy initialization: - discover_mcp_tools(lazy=True): registers tool stubs from cache without connecting to MCP servers - ensure_mcp_initialized(): thread-safe singleton that connects all configured MCP servers on first actual tool call (no-op thereafter) - _register_mcp_tool_stubs(): loads cached tool schemas from ~/.hermes/cache/mcp_tools/ so the model sees MCP tools at session start without subprocess overhead - _save_cached_tool_definitions(): persists tool schemas after successful discovery for future lazy loads Key difference from env var guard (PR NousResearch#15796): this prevents duplicates by design — no consumer needs to remember to set an env var. Multiple importers of model_tools in the same process share the same lazy pool. MCP servers only spawn when a tool is actually invoked. 11 new tests cover: lazy discovery, ensure_mcp_initialized idempotency, cache persistence, and slash_worker import safety.
Add real subprocess test that verifies importing model_tools does NOT call register_mcp_servers at import time (the core NousResearch#15275 fix). This catches regressions even if the mock-based tests pass.
|
Closing — the duplicate-spawn bug this targets no longer exists on The eager module-level Separately: the lazy connect-on-first-tool-use idea here is genuinely interesting, but it's an optimization rather than a bugfix, and it conflicts with the design #16856 deliberately landed — discover in the background at entry, then briefly join ( Thanks for the thorough write-up and the SocketPool-inspired design — appreciated even though we're not taking it. |
Summary
Fixes #15275
Problem: A single TUI session spawns two sets of
hermes mcp servechild processes — one fromtui_gateway.entryand one fromtui_gateway.slash_worker. Both paths independently importmodel_tools.py, which eagerly callsdiscover_mcp_tools()at module level.Approach: Lazy connection pool (architectural fix)
Inspired by Claude Code's SocketPool pattern. Instead of eagerly spawning MCP servers at import time, defer connections to first actual tool use:
discover_mcp_tools(lazy=True): Registers tool stubs from a local cache (~/.hermes/cache/mcp_tools/) so the model sees MCP tools at session start — but no subprocesses spawnensure_mcp_initialized(): Thread-safe singleton. On first tool call, connects all configured MCP servers. Subsequent calls are no-ops. Equivalent to Claude Code'sensureConnected()→refreshClients()flowWhy this is better than the env var guard
os.environ[...]before importChanges
tools/mcp_tool.py_mcp_initializedflag,ensure_mcp_initialized(),_register_mcp_tool_stubs(),_load/save_cached_tool_definitions(),_make_lazy_handler()model_tools.pydiscover_mcp_tools(lazy=True)instead ofdiscover_mcp_tools()tests/tui_gateway/test_slash_worker_mcp.pyFlow diagram
Alternative: Env var guard
See #15796 for a simpler tactical fix using
HERMES_SKIP_MCP_DISCOVERY. Both approaches fix the same bug. The env var guard is minimal and low-risk; this PR is the strategic long-term solution.Test results