Problem
When running agentic workflows with engine: copilot on GitHub Enterprise Server (GHES), failures are opaque and difficult to diagnose. The Copilot CLI fails with Error loading models: 400 Bad Request when the actual root cause is a 403 from the token exchange endpoint — typically because Copilot is not licensed for the enterprise/org.
Users currently have no way to distinguish between:
- Copilot not licensed on the GHES instance
- API proxy routing issues (gh-aw-firewall)
- Firewall blocking required domains
- Invalid or wrong token type
Proposal
When engine: copilot is detected, automatically inject a pre-flight diagnostic step before the Copilot CLI execution that:
-
Token exchange test: curl to ${GITHUB_API_URL}/copilot_internal/v2/token with COPILOT_GITHUB_TOKEN
- 200 → proceed normally
- 403 "not licensed to use Copilot" → fail fast with: "Copilot is not licensed for this user/org. Ask your GHES admin to enable Copilot at the enterprise level and assign a seat."
- 403 "not accessible by personal access token" → suggest correct token type/permissions
- 401 → token invalid or expired
- 404 → GHES version may not support Copilot
-
GHES detection: If GITHUB_SERVER_URL ≠ https://github.com, run GHES-specific checks:
- Verify
engine.api-target is set and matches GITHUB_API_URL
- Verify GHES API domain is in firewall allowed domains
- Check if Copilot endpoints exist on the instance
-
Direct CLI smoke test (optional, behind a flag): Run copilot directly on the host (no awf/proxy) to distinguish proxy issues from licensing issues
Evidence
We manually added these diagnostic steps to our workflow on contoso-aw.ghe.com and they immediately identified:
- Token exchange returning
403 "unauthorized: not licensed to use Copilot" — but this was surfaced to users as Error loading models: 400 Bad Request
- The Copilot CLI on the host (no proxy) showed
Error: Access denied by policy settings — confirming enterprise licensing, not a proxy issue
- The GHES
/meta endpoint showed Copilot infrastructure present, but /orgs/<org>/copilot returned 404 (Copilot not configured at org level)
Without these diagnostics, we spent multiple hours debugging what turned out to be a straightforward licensing issue.
Related
Problem
When running agentic workflows with
engine: copiloton GitHub Enterprise Server (GHES), failures are opaque and difficult to diagnose. The Copilot CLI fails withError loading models: 400 Bad Requestwhen the actual root cause is a 403 from the token exchange endpoint — typically because Copilot is not licensed for the enterprise/org.Users currently have no way to distinguish between:
Proposal
When
engine: copilotis detected, automatically inject a pre-flight diagnostic step before the Copilot CLI execution that:Token exchange test:
curlto${GITHUB_API_URL}/copilot_internal/v2/tokenwithCOPILOT_GITHUB_TOKENGHES detection: If
GITHUB_SERVER_URL≠https://github.com, run GHES-specific checks:engine.api-targetis set and matchesGITHUB_API_URLDirect CLI smoke test (optional, behind a flag): Run
copilotdirectly on the host (no awf/proxy) to distinguish proxy issues from licensing issuesEvidence
We manually added these diagnostic steps to our workflow on
contoso-aw.ghe.comand they immediately identified:403 "unauthorized: not licensed to use Copilot"— but this was surfaced to users asError loading models: 400 Bad RequestError: Access denied by policy settings— confirming enterprise licensing, not a proxy issue/metaendpoint showed Copilot infrastructure present, but/orgs/<org>/copilotreturned 404 (Copilot not configured at org level)Without these diagnostics, we spent multiple hours debugging what turned out to be a straightforward licensing issue.
Related