You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Hermes retries, hits same 400, falls through to (empty) user-facing
Reproducible from sandbox session on duplex container (DEVAGENTIC_USER_ID=sandbox, HERMES_DEFER_PERSONA=1, v0.17.1).
The recovery currently makes the empty-content problem worse than no recovery — at least the original (empty) was a clean failure; now we get noisy 400s + cascading retries.
Root cause
conversation_loop.py builds the API payload from each in-memory message by api_msg = msg.copy(), then strips individual known internal fields (reasoning, finish_reason, _thinking_prefill). But other hermes-side bookkeeping markers — _empty_recovery_synthetic (PR #69 / #108), _empty_terminal_sentinel (PR #69) — are NOT in the strip list. They round-trip to upstream.
Strict OpenAI validators (Groq, Mistral, Fireworks) reject any unknown property. Permissive ones (OpenRouter, Anthropic) silently ignore — which is why the bug didn't surface until a Groq deployment exercised the new #108 path.
This is a pre-existing bug since v0.16.0 (PR #69 introduced _empty_recovery_synthetic), but #108 makes it fire much more often (any finish_reason=tool_calls with empty list, not just finish_reason=stop with no tool_calls and no prior tool turn).
Fix
Replace the individual _thinking_prefill strip with a generic leading-underscore strip in the API-message preparation loop:
# Strip ALL internal markers — keys starting with `_` are hermes-side# bookkeeping and MUST NOT round-trip to upstream APIs.for_kin [kforkinapi_msgifisinstance(k, str) andk.startswith("_")]:
api_msg.pop(_k, None)
The leading-underscore convention is stable; anything that genuinely needs to ride the API has a documented OpenAI shape. Future internal markers added with _ prefix get auto-stripped.
Acceptance
Groq-routed sessions with HERMES_DEFER_PERSONA=1 + tool-bearing prompts no longer cascade HTTP 400s on synthetic recovery.
Existing tests pass (no regression on Mistral / OpenRouter / Anthropic paths).
New unit test asserts each known marker (_empty_recovery_synthetic / _empty_terminal_sentinel / _thinking_prefill) is stripped + standard OpenAI fields are preserved.
Symptom
In v0.17.1 (post #108), running through Groq with HERMES_DEFER_PERSONA=1 + a tool-bearing prompt causes a cascade:
finish_reason=tool_callswith empty content + empty tool_calls list_empty_recovery_synthetic: Truemarkersmessages.2: property _empty_recovery_synthe...→ HTTP 400(empty)user-facingReproducible from sandbox session on duplex container (DEVAGENTIC_USER_ID=sandbox, HERMES_DEFER_PERSONA=1, v0.17.1).
The recovery currently makes the empty-content problem worse than no recovery — at least the original
(empty)was a clean failure; now we get noisy 400s + cascading retries.Root cause
conversation_loop.pybuilds the API payload from each in-memory message byapi_msg = msg.copy(), then strips individual known internal fields (reasoning,finish_reason,_thinking_prefill). But other hermes-side bookkeeping markers —_empty_recovery_synthetic(PR #69 / #108),_empty_terminal_sentinel(PR #69) — are NOT in the strip list. They round-trip to upstream.Strict OpenAI validators (Groq, Mistral, Fireworks) reject any unknown property. Permissive ones (OpenRouter, Anthropic) silently ignore — which is why the bug didn't surface until a Groq deployment exercised the new #108 path.
This is a pre-existing bug since v0.16.0 (PR #69 introduced
_empty_recovery_synthetic), but #108 makes it fire much more often (anyfinish_reason=tool_callswith empty list, not justfinish_reason=stopwith no tool_calls and no prior tool turn).Fix
Replace the individual
_thinking_prefillstrip with a generic leading-underscore strip in the API-message preparation loop:The leading-underscore convention is stable; anything that genuinely needs to ride the API has a documented OpenAI shape. Future internal markers added with
_prefix get auto-stripped.Acceptance
_empty_recovery_synthetic/_empty_terminal_sentinel/_thinking_prefill) is stripped + standard OpenAI fields are preserved.Composition
_empty_recovery_synthetic(latent bug for Groq paths since then)finish_reason=tool_calls(surfaced the bug)Devagentic#295 (tools-strip workaround) should NOT be lifted until this fix deploys.