Skip to content

fix: exchange GitHub token for Copilot session token before API calls#7730

Closed
emma-clawdbot wants to merge 6 commits into
NousResearch:mainfrom
emma-clawdbot:fix/copilot-token-exchange
Closed

fix: exchange GitHub token for Copilot session token before API calls#7730
emma-clawdbot wants to merge 6 commits into
NousResearch:mainfrom
emma-clawdbot:fix/copilot-token-exchange

Conversation

@emma-clawdbot

Copy link
Copy Markdown

Summary

  • Hermes sends raw GitHub OAuth tokens (gho_*, ghu_*, github_pat_*) directly to api.githubcopilot.com, which returns 403 Forbidden ("not authorized to use this Copilot feature")
  • The Copilot API requires a session token (tid=...) obtained by exchanging the GitHub token with https://api.github.com/copilot_internal/v2/token — the same flow that opencode, openclaw, and VS Code all perform
  • This PR adds the missing token exchange step to copilot_auth.py

What Changed

hermes_cli/copilot_auth.py:

  • Added exchange_github_token_for_copilot_session() — calls GET https://api.github.com/copilot_internal/v2/token with the GitHub token as Authorization: Bearer, receives a short-lived Copilot session token
  • Added CopilotSessionToken dataclass for structured token metadata
  • Added session token caching to ~/.hermes/credentials/github-copilot.token.json with 5-minute expiry margin (compatible with openclaw's cache format)
  • Added _derive_base_url_from_token() — extracts the proxy-ep field from the session token and derives the correct API base URL (e.g., proxy.enterprise.githubcopilot.comapi.enterprise.githubcopilot.com). Supports individual, enterprise, and other Copilot tiers
  • Modified resolve_copilot_token() to perform the full chain: resolve GitHub token → exchange for session token → return session token

Before / After

Before After
Token sent to API Raw gho_* OAuth token tid=... session token
Result 403 Forbidden Successful API call
Enterprise support Broken Works (derives correct base URL from proxy-ep)
Token caching None Cached with expiry, compatible with openclaw format

Testing

Tested with enterprise Copilot license (copilot_enterprise_seat_quota SKU) using ghu_* token via COPILOT_GITHUB_TOKEN env var. Verified:

  • Token exchange returns valid tid=... session token
  • Session token is cached and reused within expiry window
  • Enterprise base URL correctly derived as api.enterprise.githubcopilot.com
  • claude-opus-4.6 model works through the enterprise endpoint
  • Cache format is compatible with openclaw's github-copilot.token.json

@emma-clawdbot emma-clawdbot force-pushed the fix/copilot-token-exchange branch from 96d2dff to a491cdc Compare April 11, 2026 14:05
@qike-ms

qike-ms commented Apr 12, 2026

Copy link
Copy Markdown

Code Review

Overall: The fix is correct and necessary — sending raw GitHub OAuth tokens to the Copilot inference API is the root cause of 403s. The implementation follows the same exchange pattern as opencode/openclaw/VS Code. Good test coverage and well-documented PR.

Several items worth addressing before merge:


Issues

1. TOCTOU race on token cache file permissions (copilot_auth.py)

cache_path.write_text(json.dumps(payload), encoding="utf-8")
try:
    cache_path.chmod(0o600)
except OSError:
    pass

The file is created with default permissions (often 0o644) and then restricted to 0o600. Between write_text and chmod, another process could read the token. Use os.open with O_CREAT | O_WRONLY and mode 0o600, or set os.umask(0o077) before writing (and restore after).

2. base_url is computed but never returned to the caller

resolve_copilot_token() returns (session_token, source) but discards session.base_url. The caller has no way to use the enterprise-specific API base URL without re-parsing the token themselves. This undermines the enterprise support claim in the PR description. Either:

  • Return a 3-tuple (token, base_url, source), or
  • Return the CopilotSessionToken dataclass directly

3. Duplicate expires_at normalization logic

The if expires_at > 1e11: expires_at /= 1000.0 pattern appears in both _load_cached_session_token and exchange_github_token_for_copilot_session. Extract into a helper like _normalize_expiry(value) -> float to avoid drift.

4. Redundant import inside _derive_base_url_from_token

from urllib.parse import urlparse  # inside function body

urllib.parse is already imported at the top of the file. Use urllib.parse.urlparse(proxy_ep) directly.

5. Hardcoded Editor-Version header

"Editor-Version": "vscode/1.104.1",

Fragile. If GitHub validates or rate-limits by editor version, this breaks silently. Consider making it a module-level constant with a comment, or using the actual Hermes version string.

6. resolve_copilot_token swallows exchange errors silently

except RuntimeError as exc:
    logger.error("Copilot token exchange failed: %s", exc)
    return "", ""

Returning ("", "") on exchange failure is indistinguishable from "no token found at all". The caller can't tell if the user has a valid GitHub token but the exchange endpoint is down vs. no credentials exist. Consider raising or returning a more descriptive failure.


Minor

  • Whitespace-only changes (blank lines between methods, dict reformatting) inflate the diff. Consider separating formatting into its own commit.
  • _get_token_cache_path() does try: from hermes_constants import ... on every call. Cache at module level or use a sentinel to avoid repeated import attempts.
  • Cache is checked before input validation in exchange_github_token_for_copilot_session — passing github_token="" still "succeeds" if cache is valid. Probably intentional for perf, worth a comment.

What's Good

  • Correct fix for a real 403 bug
  • Token caching with 5-min refresh margin avoids hammering the exchange endpoint
  • File permissions restricted to 0o600
  • Good test coverage: cache roundtrip, expiry, permissions, exchange success/failure, full resolution chain
  • Compatible with openclaw's cache format
  • _derive_base_url_from_token correctly handles enterprise, individual, and missing proxy-ep

@emma-clawdbot

Copy link
Copy Markdown
Author

All 6 issues and 3 minor items addressed in 8dc572c4:

Issue 1 — TOCTOU race: _save_session_token now uses os.open with O_CREAT | O_WRONLY and mode 0o600. The file is created with restricted permissions atomically.

Issue 2 — base_url not returned: resolve_copilot_token() now returns a 3-tuple (token, base_url, source). Caller in auth.py updated. Enterprise callers get the correct endpoint without re-parsing.

Issue 3 — Duplicate normalization: Extracted _normalize_expiry(value) -> float helper used by both _load_cached_session_token and exchange_github_token_for_copilot_session.

Issue 4 — Redundant import: Removed from urllib.parse import urlparse inside _derive_base_url_from_token. Uses urllib.parse.urlparse() from the module-level import directly.

Issue 5 — Hardcoded Editor-Version: Extracted to _EDITOR_VERSION module constant with a comment noting the fragility.

Issue 6 — Silent error swallowing: resolve_copilot_token now logs at warning level with the github_source context ("GitHub token found (gh auth token) but Copilot session exchange failed: ..."), making it distinguishable from no-token-found.

Minor items:

  • _get_token_cache_path now caches the hermes_home result at module level via _cached_hermes_home sentinel
  • Added comment explaining cache-before-validation ordering in exchange_github_token_for_copilot_session
  • Whitespace changes left as-is (formatter-induced, mixed with the original PR)

Tests updated: 40 copilot_auth tests + 127 api_key_providers tests all pass.

Qi Ke added 2 commits April 11, 2026 22:53
…API calls

Hermes was sending raw GitHub OAuth tokens (gho_*, ghu_*, github_pat_*)
directly to api.githubcopilot.com, which returns 403. The Copilot API
requires a session token (tid=...) obtained by exchanging the GitHub
token with https://api.github.com/copilot_internal/v2/token.

This is the same flow that opencode, openclaw, and VS Code all perform.

Changes:
- Add exchange_github_token_for_copilot_session() to call the internal
  token endpoint and obtain a Copilot session token
- Add session token caching with 5-minute expiry margin
- Add _derive_base_url_from_token() to extract the correct API base URL
  from the proxy-ep field (supports individual, enterprise, etc.)
- Modify resolve_copilot_token() to perform the full chain: resolve
  GitHub token -> exchange -> return session token
- Update tests for the new exchange flow and add tests for caching,
  base URL derivation, and error handling
- Fix TOCTOU race in _save_session_token: use os.open with O_CREAT|O_WRONLY
  and mode 0o600 so the file is never world-readable between creation and
  chmod
- Return base_url from resolve_copilot_token as 3-tuple (token, base_url,
  source) so callers get enterprise-specific API endpoints without
  re-parsing the session token
- Extract _normalize_expiry helper to deduplicate the ms-to-seconds
  conversion used in both cache loading and exchange response parsing
- Remove redundant 'from urllib.parse import urlparse' inside
  _derive_base_url_from_token (already imported at module level)
- Extract _EDITOR_VERSION module constant from hardcoded header values
- Improve error logging in resolve_copilot_token: log warning with
  github_source context when exchange fails, making exchange failure
  distinguishable from no-token-found
- Cache hermes_home in _get_token_cache_path to avoid repeated import
  attempts
- Add comment explaining cache-before-validation ordering in
  exchange_github_token_for_copilot_session
- Update tests: 3-tuple unpacking, new _normalize_expiry tests, mock
  resolve_copilot_token instead of _try_gh_cli_token for tests that
  don't need to test the exchange itself
@emma-clawdbot emma-clawdbot force-pushed the fix/copilot-token-exchange branch from 8dc572c to 9883544 Compare April 12, 2026 02:54
@emma-clawdbot

Copy link
Copy Markdown
Author

Rebased onto latest main (1cec910b), resolved one formatting conflict in auth.py. All 6 issues + 3 minor items from the review remain addressed in the rebased commits.

Tests: 40 copilot_auth + 14 copilot provider tests pass.

Qi Ke added 4 commits April 12, 2026 13:39
The session token's proxy-ep field determines the correct API endpoint
(e.g. api.enterprise.githubcopilot.com for enterprise Copilot). Previously
_resolve_api_key_provider_secret discarded this URL, so the client always
hit the individual-tier endpoint (api.githubcopilot.com), causing 400
'model not supported' for enterprise-only models like claude-opus-4.6.

_resolve_api_key_provider_secret now returns a 3-tuple with the optional
base_url_override. Both resolve_api_key_provider_credentials and
get_api_key_provider_status use it when no explicit env/config override
is set.
Two issues caused copilot to use the wrong API endpoint after the
session token expired:

1. The credential pool stores the raw GitHub token and the default
   base_url (api.githubcopilot.com).  When the pool path selected a
   copilot entry, it used this stale URL instead of the enterprise
   endpoint derived from the session token's proxy-ep field.

   Fix: in _resolve_runtime_from_pool_entry, re-call
   resolve_copilot_token() for copilot to get the session token and
   the correct enterprise base_url.

2. config.yaml persists 'base_url: https://api.githubcopilot.com'
   from initial setup.  resolve_runtime_provider honoured this saved
   value over the session-derived enterprise URL.

   Fix: skip cfg_base_url override for copilot provider since the
   session token is the authoritative source for the endpoint.

E2E verified: hermes chat --provider copilot -m claude-opus-4.6
returns correct response via api.enterprise.githubcopilot.com.
All 10 checks used "api.githubcopilot.com" in url which does NOT match
"api.enterprise.githubcopilot.com" (the enterprise endpoint).  Changed
to "githubcopilot.com" in url which matches both individual and
enterprise endpoints.

Without this, the enterprise endpoint was missing Editor-Version and
other required Copilot headers, causing 400 "missing Editor-Version
header for IDE auth".
…ug logging

F4: Add logger.debug for copilot session exchange failures in pool path
    (was silent except Exception: pass)

F9: Validate proxy-ep hostname contains a dot (must be FQDN) before
    constructing API URL. Prevents javascript:, empty, or single-label
    hostnames from producing invalid base URLs. Added negative tests for
    malformed proxy-ep values.

Code-eval verdict: approve-with-notes (0 CRITICAL, 0 HIGH, 1 MEDIUM
formatting noise, 5 LOW, 3 INFO)
@teknium1

Copy link
Copy Markdown
Contributor

Thanks for the contribution @emma-clawdbot. Closing as superseded — the same token-exchange feature (GitHub OAuth token → Copilot API JWT via /copilot_internal/v2/token) landed in #15114 using a cleaner, focused implementation from @difujia's PR #12876.

Your PR was submitted first and solved the same problem. Both contributors are credited in the salvage PR body. The 6500-line diff made a direct salvage impractical, but the feature you identified is now in main.

Merged: d7ad07d fix(copilot): exchange raw GitHub token for Copilot API JWT
#15114

@teknium1 teknium1 closed this Apr 24, 2026
@alt-glitch alt-glitch added type/bug Something isn't working P1 High — major feature broken, no workaround provider/copilot GitHub Copilot (ACP + Chat) area/auth Authentication, OAuth, credential pools comp/cli CLI entry point, hermes_cli/, setup wizard labels Apr 24, 2026
@emma-clawdbot emma-clawdbot deleted the fix/copilot-token-exchange branch June 11, 2026 01:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/auth Authentication, OAuth, credential pools comp/cli CLI entry point, hermes_cli/, setup wizard P1 High — major feature broken, no workaround provider/copilot GitHub Copilot (ACP + Chat) type/bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants