Skip to content

custom_providers: inconsistent base_url paths across protocols cause silent auth failures #44181

@ashanzzz

Description

@ashanzzz

Problem

When configuring multiple custom_providers against the same backend gateway (e.g., a self-hosted AIAPI proxy like http://192.168.8.11:3001), different protocols require different base_url paths — but this is not obvious from the documentation and leads to silent failures that are hard to debug.

Real-world example

Three providers all pointing to the same backend gateway:

custom_providers:
  aiapi-completions:
    base_url: http://192.168.8.11:3001/v1
    api_key: ...
    protocol: openai-completions    # ✅ /v1 works

  aiapi-responses:
    base_url: http://192.168.8.11:3001/v1
    api_key: ...
    protocol: openai-responses      # ✅ /v1 works

  aiapi-claude:
    base_url: http://192.168.8.11:3001/v1  # ❌ /v1 does NOT work for anthropic protocol!
    api_key: ...
    protocol: anthropic             # needs /v1/messages endpoint, NOT found under /v1

What happens

When the anthropic protocol provider is given base_url: http://192.168.8.11:3001/v1, the agent fails silently — it cannot detect the model or authenticate properly, but the error message is opaque (e.g., generic AuthenticationError, context length probe failure, or empty model listing).

Root cause

The anthropic protocol sends requests to a different endpoint path than OpenAI-compatible protocols. For example:

  • OpenAI completions: POST /v1/chat/completions
  • Anthropic messages: POST /v1/messages

Using base_url: http://host:port/v1 for anthropic causes the agent to construct URLs like http://host:port/v1/v1/messages (double /v1), which doesn't exist.

Suggested fix

Option A — Document this clearly in the custom_providers docs:

  • OpenAI-compatible protocols (openai-completions, openai-responses): use /v1 suffix
  • Anthropic protocol: use the root URL (e.g., http://host:port) or / — the protocol handler will append /v1/messages automatically

Option B — Auto-normalize base_url per protocol in the provider client, so users can always use the same base_url and the protocol suffix is appended internally.

Option C — When the provider fails to authenticate, detect whether the base_url likely has a wrong path and surface a hint (e.g., "anthropic protocol detected but base_url ends with /v1 — did you mean http://host:port?").

Impact

This tripped up a new user who assumed all three providers could share the same base_url. The silent failure made it very difficult to debug — the agent appeared to partially work because the other providers were fine, but any model routed through custom:aiapi-claude would fail mysteriously.


Hermes Agent version: v2026.5.29.2 (Docker, Unraid)
Config path: ~/.hermes/config.yaml

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low — cosmetic, nice to havearea/configConfig system, migrations, profilescomp/cliCLI entry point, hermes_cli/, setup wizardtype/bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions