Skip to content

kanban_complete + patch tool schemas advertise weaker required than handler enforces #2

@jarvis-stark-ops

Description

@jarvis-stark-ops

Symptom

Tool-using LLMs that honor JSON Schema literally (observed with xAI grok-4.3 via xai-oauth) call kanban_complete and patch without the fields the handler requires, then loop on a tool_error from the Python validator. The LLM never recovers because nothing in the schema told it which fields to provide.

Anthropic-tuned models work because they read the prose hints in the tool description — but the prose-only contract is fragile.

Reproduction

Spawn JARVIS profile (~/.hermes/profiles/jarvis/) configured with provider: xai-oauth, default: grok-4.3 on any kanban task that ends in completion. Log shows:

```
┊ ⚡ kanban_co 0.0s [provide at least one of: summary (preferred), result]
┊ ⚡ kanban_co 0.0s [error]
… looping …
```

Same pattern for the patch tool — grok emits `patch(mode="replace", path="x")` without `old_string`/`new_string`.

Root cause

In tools/kanban_tools.py:

  • KANBAN_COMPLETE_SCHEMA.parameters.required = []
  • _handle_complete (line ~543) enforces `if not (summary or result): return tool_error("provide at least one of: summary (preferred), result")`

In tools/file_tools.py:

  • PATCH_SCHEMA.parameters.required = [\"mode\"]
  • _handle_patch enforces conditional requirements (mode=replace needs path/old_string/new_string; mode=patch needs patch content)

The handler-only enforcement means the schema sent to the model doesn't reflect the real contract.

Fix direction

Express constraints in JSON Schema 2020-12:

  • `kanban_complete`: `anyOf: [{required: ["summary"]}, {required: ["result"]}]`
  • `patch`: `oneOf: [{properties.mode.const: "replace", required: ["mode", "path", "old_string", "new_string"]}, {properties.mode.const: "patch", required: ["mode", "patch"]}]`

Handlers stay unchanged for CLI legacy compatibility.

Status

Fixed in PR #1 (merged 2026-06-06, commit `928552a5d`). 11/11 jsonschema 2020-12 validation tests pass. End-to-end verified: JARVIS on grok-4.3 successfully calls `kanban_complete` + iterates the full 4-agent loop.

Validation context

  • Discovered during morning-brief-dashboard end-to-end test of the 4-agent loop on grok-4.3.
  • Learnings: `~/wiki/docs/learnings-compound/2026-06-06-jarvis-model-routing-and-loop-validation.md`
  • Related hermes-jarvis commits: `88ae91b` (interim codex routing), `fd373ae` (return to grok-4.3 after patches).

Followups

  • Audit remaining `registry.register` schemas (terminal, write_file, etc.) for the same prose-only-required pattern.
  • Consider upstream PR to NousResearch/hermes-agent once patches sit in production for a few days.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions