Problem

Tool progress markers (e.g. ⏰ list, 🔍 web_search) were injected directly into SSE delta.content chunks in the /v1/chat/completions endpoint. OpenAI-compatible frontends (Open WebUI, LobeChat, LibreChat, etc.) store delta.content verbatim as the assistant message and send it back on subsequent requests.

After enough turns (~10-15), the model learns to emit these markers as plain text instead of issuing real tool calls — silently hallucinating tool results. The user sees what looks like a working tool response, but no tool ever runs and underlying state is never modified. Smaller/local models (Gemma 26B, etc.) are especially vulnerable.

See #6972 for an excellent detailed reproduction and root cause analysis by @Swift42.

Root Cause

_on_tool_progress() in api_server.py pushed formatted marker strings directly onto _stream_q — the same queue that feeds delta.content SSE chunks:

# Before (broken)
_stream_q.put(f"\n\`{emoji} {label}\`\n")

Fix

Send tool progress as a custom SSE event type (event: hermes.tool.progress) instead of mixing it into delta.content:

event: hermes.tool.progress
data: {"tool": "terminal", "emoji": "💻", "label": "ls -la"}

Per the SSE specification, clients that don't understand a custom event type silently ignore it. This means:

• Open WebUI / LobeChat / LibreChat: Won't store the markers in conversation history (they only listen for default data: events). The corruption loop is broken.
• Future-compatible frontends: Can listen for hermes.tool.progress events to render progress indicators without persisting them.
• The /v1/runs endpoint: Already uses structured events for tool progress — this aligns /v1/chat/completions with the same principle.

Implementation

1. _on_tool_progress() now pushes a tagged tuple ("__tool_progress__", payload) instead of a plain string
2. New _emit() helper in the SSE writer routes tagged tuples to event: hermes.tool.progress and plain strings to normal delta.content chunks
3. Updated tests verify that tool progress appears as custom events and does NOT appear inside delta.content

Testing

• Updated test_stream_includes_tool_progress: Verifies markers appear as event: hermes.tool.progress and are absent from any chat.completion.chunk content delta
• Updated test_stream_tool_progress_skips_internal_events: Verifies internal events still suppressed, real progress uses custom event type

Backward Compatibility

• No breaking changes — frontends that don't handle the custom event just don't see progress indicators (same as if tool_progress_callback were disabled)
• Non-streaming /v1/chat/completions is unaffected
• /v1/runs endpoint is unaffected (already uses structured events)

Closes #6972