Skip to content

feat(bridge): add Hermes JSONL bridge with native/synthetic streaming…#287

Closed
zayr0-9 wants to merge 1 commit into
NousResearch:mainfrom
zayr0-9:native-streaming-headless-bridge
Closed

feat(bridge): add Hermes JSONL bridge with native/synthetic streaming…#287
zayr0-9 wants to merge 1 commit into
NousResearch:mainfrom
zayr0-9:native-streaming-headless-bridge

Conversation

@zayr0-9

@zayr0-9 zayr0-9 commented Mar 2, 2026

Copy link
Copy Markdown

Title

feat(bridge): add JSONL stdio bridge with native/synthetic streaming support, docs, and tests

Summary

This PR adds a new hermes bridge mode for embedding Hermes in external hosts (Electron apps, local servers, custom frontends, etc.) using a deterministic JSONL protocol over stdio.

It supports both streaming and non-streaming consumers:

  • Native streaming path (native_stream=true + native_assistant_delta=true)
  • Synthetic streaming fallback (bridge chunks full assistant turn into assistant_delta)
  • Non-streaming path (full-turn assistant_message + final events)

Why

Host integrations currently need a stable, machine-readable transport.
This introduces a first-class bridge protocol so frontends can consume Hermes output without scraping TUI text.

What’s included

New

  • hermes_cli/bridge.py
    • JSONL stdin request loop / stdout event emitter
    • Request types:
      • run
      • ping → pong
      • interrupt → interrupt_ack
      • shutdown / exit / quit → shutdown_ack
    • Session tracking + resume support
    • Active-run protection per session_id
    • Optional per-request/default cwd handling
    • JSON-safe event serialization

Updated

  • hermes_cli/main.py

    • Added hermes bridge command + args:
      • --model, --base-url, --api-key, --max-iterations
      • --toolsets, --disabled-toolsets
      • --cwd, --once, --verbose
      • --assistant-delta, --assistant-delta-chunk-size
      • --native-assistant-delta, --native-stream
      • stream timeout knobs

  • run_agent.py

    • Bridge-facing event flow wired for tool/lifecycle/finalization + assistant delta streaming path

  • README.md

    • Added bridge command docs and JSONL quick reference
    • Documented native vs synthetic assistant_delta

Tests

  • tests/hermes_cli/test_bridge.py
    • command delegation
    • ping/pong
    • request validation errors
    • interrupt behavior + ack
    • native delta passthrough
    • synthetic fallback behavior
    • event ordering guarantees
  • tests/test_run_agent.py
    • additional native streaming and tool-call loop coverage

Protocol behavior notes

  • Deterministic turn ordering is enforced.
  • With native deltas, clients receive deltas + end marker + full assistant message.
  • With synthetic fallback, deltas are emitted from completed assistant content (to avoid duplication behavior is tested and documented).

Backward compatibility

  • No behavior changes to existing hermes / hermes chat flows.
  • Bridge mode is additive (hermes bridge only).

How to test

python -m pytest tests/hermes_cli/test_bridge.py -v
python -m pytest tests/test_run_agent.py -v

Manual smoke:

{"type":"run","request_id":"req-1","session_id":"sess-1","message":"Summarize README.md","assistant_delta":true,"native_stream":true,"native_assistant_delta":true}
{"type":"interrupt","request_id":"req-2","session_id":"sess-1","message":"stop now"}
{"type":"shutdown","request_id":"req-3"}

Branch may drift due to active upstream updates; happy to rebase/sync once review feedback is in.

@teknium1

teknium1 commented Mar 5, 2026

Copy link
Copy Markdown
Contributor

Hey @zayr0-9, thanks for the effort here — the bridge concept is genuinely useful and the protocol design (JSONL over stdio, ping/pong, interrupt, shutdown) is well thought out.

However, we have a concern with the scope of the run_agent.py changes. This PR adds ~400 lines to an already complex file — event emission at every return point, a full streaming code path, tool call accumulation, etc. It makes the core agent loop significantly harder to read and maintain.

After reviewing, we believe the bridge can deliver its full value without touching run_agent.py at all:

  • Final result: run_conversation() already returns the complete response, messages list, api_calls count, and status flags (completed/interrupted/partial/failed).
  • Tool progress: tool_progress_callback already fires on every tool invocation — the bridge can use this for real-time "agent is calling X" notifications.
  • Interrupt: agent.interrupt() already exists and works.
  • Streaming feel: The synthetic delta chunking you already implemented in bridge.py works entirely from the return value — no internal hooks needed.

The only thing that genuinely requires run_agent internals is native token-by-token streaming (stream=True on the API call). But that's a separate feature with its own complexity, and shouldn't be bundled into the bridge transport layer.

What we'd like to see:

A refactored PR where bridge.py is a clean wrapper around the existing AIAgent API:

  1. bridge.py — JSONL stdin/stdout loop, creates AIAgent, calls run_conversation(), uses tool_progress_callback for real-time tool events, emits structured JSONL from the return value, synthetic delta chunking for streaming UX
  2. main.py — the hermes bridge subcommand (as-is, this part is clean)
  3. Tests
  4. Zero changes to run_agent.py

The type hint cleanups and tool_calls parsing improvements in run_agent.py are welcome but should be a separate small PR.

If we later want an event_callback hook or native streaming support in the agent core, those can be focused follow-up PRs with much smaller surface area.

This would take the PR from ~1800 lines touching the agent core to a ~700-line additive-only change, which is much easier to review and merge confidently. Let us know if you have questions or want to discuss the approach!

@teknium1 teknium1 closed this Mar 9, 2026
@zayr0-9

zayr0-9 commented Mar 9, 2026

Copy link
Copy Markdown
Author

All right I will trim it down to just have a simple headless server interface without touching run_agent.py 😁

@PowerCreek PowerCreek mentioned this pull request May 26, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@zayr0-9 @teknium1