Skip to content

hermes doctor: generic memory provider check overstates is_available() — "X provider active" implies reachability that wasn't probed #36

@PowerCreek

Description

@PowerCreek

Context

Continuing the diagnostic-loudness audit after #34/#35 (Mem0 probe). Looked at every other memory provider's is_available() implementation:

  • openvikingbool(os.environ.get("OPENVIKING_ENDPOINT"))
  • supermemory — checks SUPERMEMORY_API_KEY + tries __import__("supermemory")
  • hindsight — config + env-var check
  • byterover — checks brv CLI is on PATH
  • retaindbbool(os.environ.get("RETAINDB_API_KEY"))
  • holographic — always returns True (local SQLite)

None of these probe the backend. So the doctor message:

✓ openviking provider active

is misleading — OPENVIKING_ENDPOINT could point at an unreachable host, SUPERMEMORY_API_KEY could be expired, brv CLI could be installed but not logged in. The operator gets a false-positive green check.

The bigger picture

Honcho probes via get_honcho_client(hcfg) (#34 noted this). Mem0 probes via MemoryClient.get_all() (just shipped in #35). The other 6 providers don't probe at all — they get an asymmetric diagnostic.

The right long-term shape: extend agent/memory_provider.MemoryProvider with a health_check() -> tuple[bool, str] ABC method. Default implementation delegates to is_available(). Plugins that can do a real probe override it. Doctor's generic branch then calls health_check() and treats the (False, reason) case as a failure with the specific reason.

That's a design-level change; would touch every memory plugin's __init__.py. Worth a focused proposal.

Scope of this issue / PR

Short-term mitigation only:

  1. Soften the generic branch's check_ok wording from "X provider active" to "X provider configured" (the truthful claim — config is detected, reachability is not).
  2. Add a check_info row pointing at the gap so the operator knows reachability wasn't probed: "is_available() reports True (env vars / CLI present) — backend reachability not probed by doctor. Run a real session to confirm; see #N."

The health_check() ABC design ships in a follow-up issue.

Out of scope (this PR)

  • Adding health_check() to the ABC. Bigger surface.
  • Per-provider probe blocks like Mem0/Honcho. Would require deep knowledge of each provider's SDK; high upfront cost.

Filed by hermes-maintainer (PowerCreek). PR incoming for the message tightening.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions