-
Notifications
You must be signed in to change notification settings - Fork 18
GHEC: API proxy does not forward GHES token exchange to enterprise Copilot endpoints #1315
Description
Problem
When running gh-aw workflows on GitHub Enterprise Cloud (GHEC) tenants (e.g., company.ghe.com), the Copilot CLI fails to authenticate because the API proxy / firewall does not correctly handle the token exchange flow for enterprise environments.
Relates to: github/gh-aw#18480
Symptoms
Two independent GHEC users (@kbreit-insight on myorg.ghe.com, @charleshancoder on xyz.ghe.com) report the same failure pattern:
-
Token exchange returns 403 on GHES API:
POST https://api.{tenant}.ghe.com/copilot_internal/v2/tokenreturns"Resource not accessible by personal access token"(403) when using a fine-grained PAT with Copilot Requests permission. -
Copilot CLI sees "Error loading models: 400 Bad Request" when running inside the awf container, even though
COPILOT_API_URLis set to the api-proxy address (http://172.30.0.30:10002). -
GHEC domains not injected into
--allowed-domains: The firewall blocks connections to{tenant}.ghe.combecause the domain is not being automatically added to the allowed list. User reports: "When the firewall blocked a connection to GHE it showed a list of blocked and allowed domains. This also means the domain isn't being injected into the--allowed-domainsargument." -
Direct host test works differently: Running
copilot --promptdirectly on the host (bypassing awf/api-proxy) also fails with a 401 on user login, but the error path is different, suggesting the api-proxy is transforming or misrouting the request.
Diagnostic Evidence
Both users ran the same diagnostic script. Key findings:
Test 1: COPILOT_GITHUB_TOKEN → GHES token exchange
URL: https://api.{tenant}.ghe.com/copilot_internal/v2/token
Status: 403 ("Resource not accessible by personal access token")
Test 2: github.token (Actions) → GHES token exchange
URL: https://api.{tenant}.ghe.com/copilot_internal/v2/token
Status: 403 ("Resource not accessible by integration")
Test 3: COPILOT_GITHUB_TOKEN → api.github.com token exchange
Status: 401 ("Bad credentials") ← expected: GHEC PAT invalid on github.com
Test 4: GHES endpoint availability
/copilot_internal/v2/token → 403
/copilot_internal/v2/models → 404
/models → 404
/meta → 200
Agent logs inside awf container:
Using COPILOT_API_URL from environment: http://172.30.0.30:10002
Error loading models: Error: Failed to list models: 400 Bad Request
Root Causes (suspected)
There appear to be two distinct issues:
Issue 1: GHEC domains not added to firewall allowed list
When GITHUB_SERVER_URL is a GHEC tenant (e.g., https://company.ghe.com), the firewall's --allowed-domains argument does not automatically include:
company.ghe.comapi.company.ghe.com*.company.ghe.com(for any subdomains the GHEC tenant uses)
Expected: The firewall should detect non-github.com GITHUB_SERVER_URL values and automatically add the GHEC domain and its API subdomain to the allowed list.
Issue 2: API proxy does not route Copilot token exchange to GHES endpoint
The api-proxy intercepts Copilot CLI's token exchange request but may be routing it to api.github.com/copilot_internal/v2/token instead of api.{tenant}.ghe.com/copilot_internal/v2/token. This would explain:
- The 400 Bad Request from the api-proxy (it proxied to the wrong host)
- The fact that previously (v0.45.0) users could work around this by manually setting
GITHUB_SERVER_URLandGITHUB_API_URLenv vars in the lock file
The api-proxy needs to respect GITHUB_API_URL / GITHUB_SERVER_URL when determining the upstream Copilot API endpoint for token exchange and model listing.
Proposed Fix
-
Firewall: Auto-detect GHEC domains from
GITHUB_SERVER_URLenvironment variable and add them to--allowed-domains. WhenGITHUB_SERVER_URLis nothttps://github.com, extract the host and add both{host}andapi.{host}(or derive fromGITHUB_API_URL). -
API Proxy: When proxying Copilot API requests (token exchange, model listing), use
GITHUB_API_URLas the upstream base URL instead of hardcodingapi.github.com. The environment variablesGITHUB_SERVER_URLandGITHUB_API_URLare already passed through to the awf container by gh-aw.
Environment Details
- gh-aw version: v0.58.0 (latest at time of testing)
- Firewall version: 0.24.1 (also tested with latest)
- GHEC tenants: Two independent organizations on
*.ghe.com - PAT type: Fine-grained with Copilot Requests permission
- Previous working version: gh-aw v0.45.0 (with manual lock file edits)