Summary

When using Gemma 4 served via vLLM (NaN Builders, and others), the harmony parser intermittently structures the tool call cleanly — tool_calls is non-empty and the upstream returns finish_reason='tool_calls' — but lets the model's chat-template quote markers (<|"|> in its full and partial forms) escape into the JSON of tool_call.arguments as literal characters.

Hermes' downstream json.loads(arguments) then fails. The retry sees the same deterministic corruption, fails again, and the run gets surfaced to the user as Response truncated (finish_reason='length') — model hit max output tokens despite no token cap being hit and the upstream having returned successfully.

This is a complement to the existing content-side recovery path proposed in #7449 — that one fires on if not tool_calls and content:, assuming the structured channel is safe. This PR handles the case where the structured channel is taken but the JSON inside arguments is corrupted.

Symptom

Captured payload (kanban_complete from a worker, gemma4 via NaN Builders, full 1658-byte arg string, captured via SSE pass-through proxy):

{"metadata": {"findings": [{"date": "<|\"|"2023-09-18<|\", "identifier": "<|\"|"BOE-A-2023-19616<|\", "title": "<|Resolución de 28 de julio de 2023, ...<|"}, ... ]}, "summary": "...", "task_id": "t_8345a55c"}

json.loads chokes at column 43 because "<|\"|" parses as a closed string <|"| followed by a bare 2023-09-18 token. Same payload retried twice (deterministic), same parse error twice, session aborts.

Four marker shapes turn up in the wild:

Approach

agent/transports/chat_completions.py::normalize_response already populates tool_calls. After that loop, if any tool_call.arguments contains <| and fails json.loads, run the marker-stripping substitution and accept the result only if the cleaned variant parses. Concretely:

if tool_calls:
    import json as _json_for_validation

    def _strip_gemma4_quote_markers(s: str) -> str:
        if not s or "<|" not in s:
            return s
        return (s
                .replace('"<|\\"|"', '"')
                .replace('<|\\"', '"')
                .replace('"<|', '"')
                .replace('<|"', '"'))

    for tc in tool_calls:
        args_str = tc.arguments
        if not args_str or "<|" not in args_str:
            continue
        try:
            _json_for_validation.loads(args_str)
            continue  # already valid
        except (ValueError, TypeError):
            pass
        cleaned = _strip_gemma4_quote_markers(args_str)
        if cleaned == args_str:
            continue
        try:
            _json_for_validation.loads(cleaned)
        except (ValueError, TypeError):
            continue  # cleaned still broken
        tc.arguments = cleaned

The double guard (original unparseable AND cleaned parseable) means already-valid args are never touched, even if they cosmetically contain <| substrings — legitimate user content (regex with alternation, certain code patterns) can do that.

Tests

TestChatCompletionsGemma4ArgsSanitization (6 cases) added to tests/agent/transports/test_chat_completions.py:

• ✅ Real-world full-marker payload (the dump shape above) → recovered.
• ✅ Full open marker + bare close on the same value → recovered.
• ✅ Bare-only markers that DON'T break JSON → arguments left intact (strict guard).
• ✅ Already-valid arguments → not mutated.
• ✅ <| present but cleaned variant still unparseable → original left for downstream error visibility.
• ✅ Multiple tool_calls in a single response, mixed clean/dirty → each handled independently.

Full test file: 68 passed in 4.11s (61 existing + 7 new).

Validation in production

Validated against 14 tool_calls captured from a failing gemma4 + NaN Builders session: 12 already-valid args untouched, 2 broken kanban_complete payloads (with nested metadata.findings) parse cleanly post-sanitization, 0 collateral damage. Worker resumes normally.

Relationship to #7449

Independent. #7449 (and the three follow-up issues in its thread) addresses Gemma 4's tool calls landing in message.content instead of tool_calls. This PR addresses the orthogonal case where tool_calls IS populated but its arguments JSON is corrupted. Either fix can land first; both are needed for full coverage.