Skip to content

Model capability pre-flight validation #5437

@kshitijk4poor

Description

@kshitijk4poor

Problem

Hermes sends tool schemas, vision content, and reasoning params to all models regardless of whether they support those features. Failures are discovered only at API call time as cryptic provider errors like 400: unsupported parameter.

Proposed Solution

Add a ModelCapabilities structure populated from the static model catalog in hermes_cli/models.py and from models.dev metadata:

  • supports_tools, supports_vision, supports_reasoning, supports_streaming
  • max_output_tokens, context_window

Use it in run_conversation() pre-flight to emit warnings when:

  • Tool schemas are sent to non-tool models
  • Vision payloads are sent to non-vision models
  • Reasoning params are sent to non-reasoning models

Advisory only — never block requests. Warn the user early rather than letting them discover incompatibility via a cryptic API error.

Benefits

  • Better UX: users see "this model doesn't support tools" before wasting an API call
  • Foundation for smarter model selection and routing
  • Lightweight: pure pattern matching, no API calls

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low — cosmetic, nice to havecomp/agentCore agent loop, run_agent.py, prompt buildertype/featureNew feature or request

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions