Skip to content

[Bug]: Hermes v0.10.0 Discord /model command fails for Copilot model selection after upgrade #12086

@Nicecsh

Description

@Nicecsh

Bug Description

After upgrading Hermes Agent to v0.10.0, using the Discord gateway chat command /model to switch to a specific GitHub Copilot model fails, even though direct Copilot-backed chat requests still work.

In my environment, this regression appears in Discord conversations after the upgrade:

  • /model fails when selecting a concrete Copilot model such as gpt-5.4, gpt-5-mini, or claude-sonnet-4.6
  • but direct CLI calls like hermes chat --provider copilot -m gpt-5.4 -q '只回复OK' still succeed

This makes the issue look like a /model validation-path regression, not a general Copilot outage.

Steps to Reproduce

  1. Upgrade Hermes Agent to v0.10.0
  2. Configure Hermes to use provider: copilot
  3. In a Discord chat connected to Hermes gateway, type /model
  4. Select or enter a concrete Copilot model such as:
    • gpt-5.4
    • gpt-5-mini
    • claude-sonnet-4.6
  5. Observe that model switching fails

Expected Behavior

If the Copilot-backed model is actually usable for inference, /model should allow the switch (or at least degrade with a warning instead of hard-failing during pre-validation).

Actual Behavior

/model fails before switching, with an error equivalent to:

Could not reach the GitHub Copilot API to validate `<model>`.
If the service isn't down, this model may not be valid.

What I Verified

1. Direct Copilot-backed chats still work

These succeed in the same environment:

hermes chat -q '只回复OK' --provider copilot -m gpt-5-mini --quiet
hermes chat -q '只回复OK' --provider copilot -m gpt-5.4 --quiet

So this is not a simple “Copilot is broken” situation.

2. /model switch path fails in switch_model()

Calling the shared model switch logic for Copilot fails for known-good models:

  • gpt-5.4
  • gpt-5-mini
  • claude-sonnet-4.6

The failure happens in the validation path, not in actual inference.

3. Root cause appears to be Copilot /models validation

I traced the failure to the /model validation pipeline:

  • /modelswitch_model()
  • switch_model()validate_requested_model()
  • for Copilot, validation probes https://api.githubcopilot.com/models

In this environment, that endpoint returns:

HTTP 403
unauthorized: not authorized to use this Copilot feature

However, actual chat inference is still usable for concrete models.

So the behavior is:

  • chat completion works
  • Copilot /models probe returns 403
  • /model hard-fails because it requires the probe to succeed

Why this is a bug

The current /model implementation appears to treat failure to access Copilot /models as a hard validation failure, even when the selected model is still usable in real requests.

That creates a broken UX in Discord/gateway usage:

  • users cannot switch to models that are actually usable
  • /model becomes stricter than real inference

Suggested Fix

For provider=copilot, when the /models probe returns 403 / unavailable:

  1. do not immediately hard-fail /model
  2. fall back to Hermes’ local curated/known Copilot model list plus normalization
  3. if the requested model is a known Copilot-supported model, allow the switch with a warning such as:
    • Could not validate against Copilot /models (403); allowing switch based on known model catalog. Actual availability will be verified on first request.

This preserves safety while avoiding false negatives.

Environment

  • Hermes Agent: v0.10.0 (2026.4.16)
  • Python: 3.11.15
  • OS: Ubuntu 24.04.2 LTS
  • Provider: copilot
  • Gateway: Discord

Related Context

This seems related to Copilot provider integration and validation behavior, but is distinct from tool-flow 403 issues such as #11035.

This report is specifically about:

  • Discord gateway /model command
  • post-upgrade behavior on v0.10.0
  • model switch failing during validation even though actual Copilot inference works

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium — degraded but workaround existscomp/gatewayGateway runner, session dispatch, deliveryplatform/discordDiscord bot adapterprovider/copilotGitHub Copilot (ACP + Chat)type/bugSomething isn't working

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions