Discovered via devagentic-side multi-rotation diagnosis
Sandbox tool-execution path has been broken across multiple rotations of devagentic-side cascade work. Root cause finally pinned via devagentic#334 diagnostic:
- mistral-large emits a structurally valid tool_call response:
content="" + tool_calls=[{...}] with finish_reason="tool_calls"
- devagentic correctly treats this as success (per OpenAI contract — populated tool_calls is success)
- hermes-cli rejects the response as "empty" and retries with synthesis prose, never executing the actual tool
Exact structure mistral returns
{
"id": "5oTo1Uia0",
"function": {
"name": "write_file",
"arguments": "{\"path\": \"/tmp/random_test.py\", ...}"
},
"index": 0
}
Missing type: "function" field. OpenAI spec lists it as required, but:
- The field has only ONE valid value (
"function") — it's a marker for future polymorphism, not a discriminator with multiple cases
- Multiple providers (mistral observed; possibly others) omit it
- Strict-spec response handlers filter on
tc.type == "function" and skip type-less entries
Bug
hermes-cli's response handler likely:
# pseudocode
for tc in tool_calls:
if tc.type == "function": # ← filters out type-less entries
execute(tc.function.name, tc.function.arguments)
# ... falls through to "no tool_calls processed" handling → treats as empty
For mistral-shaped tool_calls (no type field), the loop's filter rejects everything, and hermes treats the response as content-less → retries.
This is the next-uncovered case after hermes-agent#99 (tool_calls=[] with finish_reason=tool_calls) and #108/#111 (related shape fixes). Same family of bugs — handler is over-strict on the OpenAI spec.
Proposed fix
Treat missing type as type="function" per OpenAI's spec semantics (function is the only valid value; spec mandates it but doesn't actually disambiguate). Either:
(a) Default-at-parse: in the OpenAI-response parsing path, default tc.type = "function" when absent.
(b) Filter-by-elimination: when iterating tool_calls, treat tc.type in {"function", None} (or tc.type absent) as the function case.
Lean (a) — single normalization point catches every code path that later iterates tool_calls.
Repro
Send any chat-completion request to devagentic's OAI-compat shim (/v1/chat/completions) with a tools array that mistral can answer with a tool_call. Mistral consistently returns the response without type field. hermes-cli rejects it.
Verified via devagentic in-process probe: service._real_completion(model="mistral-large", messages=[{"role":"user","content":"Write a python script and execute it"}], tools=[...]) returns the shape above.
Companion (devagentic-side defense-in-depth)
Filed devagentic#NN to normalize type="function" at the devagentic OAI-shim boundary as a SHORT-TERM workaround. Helps any other strict-spec client too. Ships independently of this hermes fix.
Once this issue lands, the devagentic-side normalization becomes redundant defense (still harmless to keep).
Owner
hermes-maintainer (per devagentic#203 §1.3 — hermes-side response handler + tool-call processing).
DoD
- hermes-cli's response handler defaults
tool_call.type = "function" when the field is absent on the upstream response
- Test against the exact mistral-shaped tool_calls (id + function + index, no type)
- Verify against the sandbox prompt end-to-end: model returns tool_call → hermes executes it → file actually written
Related
🤖 Generated with Claude Code
Discovered via devagentic-side multi-rotation diagnosis
Sandbox tool-execution path has been broken across multiple rotations of devagentic-side cascade work. Root cause finally pinned via devagentic#334 diagnostic:
content=""+tool_calls=[{...}]withfinish_reason="tool_calls"Exact structure mistral returns
{ "id": "5oTo1Uia0", "function": { "name": "write_file", "arguments": "{\"path\": \"/tmp/random_test.py\", ...}" }, "index": 0 }Missing
type: "function"field. OpenAI spec lists it as required, but:"function") — it's a marker for future polymorphism, not a discriminator with multiple casestc.type == "function"and skip type-less entriesBug
hermes-cli's response handler likely:For mistral-shaped tool_calls (no
typefield), the loop's filter rejects everything, and hermes treats the response as content-less → retries.This is the next-uncovered case after hermes-agent#99 (
tool_calls=[]withfinish_reason=tool_calls) and #108/#111 (related shape fixes). Same family of bugs — handler is over-strict on the OpenAI spec.Proposed fix
Treat missing
typeastype="function"per OpenAI's spec semantics (function is the only valid value; spec mandates it but doesn't actually disambiguate). Either:(a) Default-at-parse: in the OpenAI-response parsing path, default
tc.type = "function"when absent.(b) Filter-by-elimination: when iterating tool_calls, treat
tc.type in {"function", None}(ortc.typeabsent) as the function case.Lean (a) — single normalization point catches every code path that later iterates
tool_calls.Repro
Send any chat-completion request to devagentic's OAI-compat shim (
/v1/chat/completions) with atoolsarray that mistral can answer with a tool_call. Mistral consistently returns the response withouttypefield. hermes-cli rejects it.Verified via devagentic in-process probe:
service._real_completion(model="mistral-large", messages=[{"role":"user","content":"Write a python script and execute it"}], tools=[...])returns the shape above.Companion (devagentic-side defense-in-depth)
Filed devagentic#NN to normalize
type="function"at the devagentic OAI-shim boundary as a SHORT-TERM workaround. Helps any other strict-spec client too. Ships independently of this hermes fix.Once this issue lands, the devagentic-side normalization becomes redundant defense (still harmless to keep).
Owner
hermes-maintainer (per devagentic#203 §1.3 — hermes-side response handler + tool-call processing).
DoD
tool_call.type = "function"when the field is absent on the upstream responseRelated
🤖 Generated with Claude Code