Skip to content

bug(vision): image analysis should recover gracefully when primary vision path fails #1034

@atuida

Description

@atuida

Summary

Image understanding in Hermes can fail with a generic message like:

"There was a problem with the request and the image could not be analyzed."

In practice this leaves the agent unable to inspect user-sent images, even when the rest of the system is otherwise working.

Why this matters

For chat/gateway use, image handling is not optional. If a user sends an image, Hermes should either:

  • analyze it successfully, or
  • degrade gracefully with a useful fallback path

A generic failure with no retry/fallback makes image support feel unreliable.

Observed behavior

  • User sends image in chat
  • vision_analyze can fail with a generic request error
  • browser_vision may also be unavailable or fail depending on runtime
  • Agent is left without a reliable way to inspect the image

Expected behavior

When primary image analysis fails, Hermes should have a clear recovery path, for example:

  1. retry with auxiliary vision client
  2. fall back to another available vision backend/tool
  3. surface a precise error (missing dependency / backend unavailable / unsupported image path)
  4. avoid vague "could not be analyzed" messaging when the failure reason is knowable

Requested improvement

  • Add robust fallback/retry behavior for image analysis
  • Improve diagnostics so the agent can tell whether the issue is backend, dependency, path, or provider support
  • Prefer graceful degradation over generic vision failure

User impact

This is especially visible on messaging platforms like Telegram, where image understanding is a core expectation. Hermes should be able to reliably inspect user-sent images or fail in a much more actionable way.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions