Bug Description
After upgrading Hermes Agent to v0.10.0, using the Discord gateway chat command /model to switch to a specific GitHub Copilot model fails, even though direct Copilot-backed chat requests still work.
In my environment, this regression appears in Discord conversations after the upgrade:
/model fails when selecting a concrete Copilot model such as gpt-5.4, gpt-5-mini, or claude-sonnet-4.6
- but direct CLI calls like
hermes chat --provider copilot -m gpt-5.4 -q '只回复OK' still succeed
This makes the issue look like a /model validation-path regression, not a general Copilot outage.
Steps to Reproduce
- Upgrade Hermes Agent to
v0.10.0
- Configure Hermes to use
provider: copilot
- In a Discord chat connected to Hermes gateway, type
/model
- Select or enter a concrete Copilot model such as:
gpt-5.4
gpt-5-mini
claude-sonnet-4.6
- Observe that model switching fails
Expected Behavior
If the Copilot-backed model is actually usable for inference, /model should allow the switch (or at least degrade with a warning instead of hard-failing during pre-validation).
Actual Behavior
/model fails before switching, with an error equivalent to:
Could not reach the GitHub Copilot API to validate `<model>`.
If the service isn't down, this model may not be valid.
What I Verified
1. Direct Copilot-backed chats still work
These succeed in the same environment:
hermes chat -q '只回复OK' --provider copilot -m gpt-5-mini --quiet
hermes chat -q '只回复OK' --provider copilot -m gpt-5.4 --quiet
So this is not a simple “Copilot is broken” situation.
2. /model switch path fails in switch_model()
Calling the shared model switch logic for Copilot fails for known-good models:
gpt-5.4
gpt-5-mini
claude-sonnet-4.6
The failure happens in the validation path, not in actual inference.
3. Root cause appears to be Copilot /models validation
I traced the failure to the /model validation pipeline:
/model → switch_model()
switch_model() → validate_requested_model()
- for Copilot, validation probes
https://api.githubcopilot.com/models
In this environment, that endpoint returns:
HTTP 403
unauthorized: not authorized to use this Copilot feature
However, actual chat inference is still usable for concrete models.
So the behavior is:
- chat completion works
- Copilot
/models probe returns 403
- /model hard-fails because it requires the probe to succeed
Why this is a bug
The current /model implementation appears to treat failure to access Copilot /models as a hard validation failure, even when the selected model is still usable in real requests.
That creates a broken UX in Discord/gateway usage:
- users cannot switch to models that are actually usable
/model becomes stricter than real inference
Suggested Fix
For provider=copilot, when the /models probe returns 403 / unavailable:
- do not immediately hard-fail
/model
- fall back to Hermes’ local curated/known Copilot model list plus normalization
- if the requested model is a known Copilot-supported model, allow the switch with a warning such as:
Could not validate against Copilot /models (403); allowing switch based on known model catalog. Actual availability will be verified on first request.
This preserves safety while avoiding false negatives.
Environment
- Hermes Agent:
v0.10.0 (2026.4.16)
- Python:
3.11.15
- OS:
Ubuntu 24.04.2 LTS
- Provider:
copilot
- Gateway:
Discord
Related Context
This seems related to Copilot provider integration and validation behavior, but is distinct from tool-flow 403 issues such as #11035.
This report is specifically about:
- Discord gateway /model command
- post-upgrade behavior on v0.10.0
- model switch failing during validation even though actual Copilot inference works
Bug Description
After upgrading Hermes Agent to v0.10.0, using the Discord gateway chat command
/modelto switch to a specific GitHub Copilot model fails, even though direct Copilot-backed chat requests still work.In my environment, this regression appears in Discord conversations after the upgrade:
/modelfails when selecting a concrete Copilot model such asgpt-5.4,gpt-5-mini, orclaude-sonnet-4.6hermes chat --provider copilot -m gpt-5.4 -q '只回复OK'still succeedThis makes the issue look like a /model validation-path regression, not a general Copilot outage.
Steps to Reproduce
v0.10.0provider: copilot/modelgpt-5.4gpt-5-miniclaude-sonnet-4.6Expected Behavior
If the Copilot-backed model is actually usable for inference,
/modelshould allow the switch (or at least degrade with a warning instead of hard-failing during pre-validation).Actual Behavior
/modelfails before switching, with an error equivalent to:What I Verified
1. Direct Copilot-backed chats still work
These succeed in the same environment:
So this is not a simple “Copilot is broken” situation.
2.
/modelswitch path fails inswitch_model()Calling the shared model switch logic for Copilot fails for known-good models:
gpt-5.4gpt-5-miniclaude-sonnet-4.6The failure happens in the validation path, not in actual inference.
3. Root cause appears to be Copilot
/modelsvalidationI traced the failure to the
/modelvalidation pipeline:/model→switch_model()switch_model()→validate_requested_model()https://api.githubcopilot.com/modelsIn this environment, that endpoint returns:
However, actual chat inference is still usable for concrete models.
So the behavior is:
/modelsprobe returns 403Why this is a bug
The current
/modelimplementation appears to treat failure to access Copilot/modelsas a hard validation failure, even when the selected model is still usable in real requests.That creates a broken UX in Discord/gateway usage:
/modelbecomes stricter than real inferenceSuggested Fix
For
provider=copilot, when the/modelsprobe returns403/ unavailable:/modelCould not validate against Copilot /models (403); allowing switch based on known model catalog. Actual availability will be verified on first request.This preserves safety while avoiding false negatives.
Environment
v0.10.0 (2026.4.16)3.11.15Ubuntu 24.04.2 LTScopilotDiscordRelated Context
This seems related to Copilot provider integration and validation behavior, but is distinct from tool-flow 403 issues such as #11035.
This report is specifically about: