fix(copilot): use live model catalog for context windows and token exchange#7272
Closed
difujia wants to merge 2 commits into
Closed
fix(copilot): use live model catalog for context windows and token exchange#7272difujia wants to merge 2 commits into
difujia wants to merge 2 commits into
Conversation
32456d7 to
4535c39
Compare
…change The Copilot provider hardcoded context windows from models.dev static data, which did not reflect account-specific models (e.g. claude-opus-4.6-1m with 1M context). This change: - Queries the live Copilot /models API for real context window values and caches results for 1 hour - Implements token exchange (raw GitHub token -> Copilot API JWT) via /copilot_internal/v2/token, with in-process caching and graceful fallback to raw token on failure - Switches the OAuth client ID to VS Code's (Iv1.b507a08c87ecfe98) which grants access to the full model catalog including internal-only models - Derives the correct API base URL (individual vs enterprise) from the proxy-ep field in the exchanged token - Broadens URL matching from 'api.githubcopilot.com' to 'githubcopilot.com' to handle enterprise endpoints - Fetches live model catalog in /model switch so account-specific models appear in the selection list Tested on Ubuntu 24.04 with GitHub Copilot Individual plan.
4535c39 to
de63326
Compare
The Copilot credential pool stores raw GitHub tokens (ghu_/gho_/github_pat_) and the default base URL (api.githubcopilot.com). When _swap_credential() is called during subagent delegation or credential rotation, these raw values would overwrite the already-exchanged JWT and enterprise base URL, causing 400 'model_not_supported' errors for models like claude-opus-4.6-1m that require the enterprise endpoint. Now _swap_credential() detects the copilot provider and calls resolve_copilot_api_token() to exchange the raw token for a JWT and derive the correct base URL before applying the swap. The exchange is cached in-process so repeated calls are near-zero cost. Fixes: delegate_task subagents and cron jobs failing with 400 errors when using Copilot provider with credential pool rotation.
Contributor
Author
|
@teknium1 @alt-glitch Friendly ping — this PR fixes several critical Copilot provider issues that affect all Copilot users:
Includes 8 new test classes (42 tests total in test_copilot_auth.py). Tested end-to-end on Ubuntu 24.04 with Copilot Individual plan running Happy to rebase on latest main if needed! |
|
Thanks for doing this! @difujia |
difujia
added a commit
to difujia/hermes-agent
that referenced
this pull request
Apr 20, 2026
… resolver The Copilot provider resolved context windows via models.dev static data, which does not include account-specific models (e.g. claude-opus-4.6-1m with 1M context). This adds the live Copilot /models API as a higher- priority source for copilot/copilot-acp/github-copilot providers. New helper get_copilot_model_context() in hermes_cli/models.py extracts capabilities.limits.max_prompt_tokens from the cached catalog. Results are cached in-process for 1 hour. In agent/model_metadata.py, step 5a queries the live API before falling through to models.dev (step 5b). This ensures account-specific models get correct context windows while standard models still have a fallback. Part 1 of NousResearch#7731. Refs: NousResearch#7272
Contributor
Author
|
Closing in favor of the split PRs as suggested by @konsisumer in #7731:
Thanks for the review guidance! |
teknium1
pushed a commit
that referenced
this pull request
Apr 24, 2026
… resolver The Copilot provider resolved context windows via models.dev static data, which does not include account-specific models (e.g. claude-opus-4.6-1m with 1M context). This adds the live Copilot /models API as a higher- priority source for copilot/copilot-acp/github-copilot providers. New helper get_copilot_model_context() in hermes_cli/models.py extracts capabilities.limits.max_prompt_tokens from the cached catalog. Results are cached in-process for 1 hour. In agent/model_metadata.py, step 5a queries the live API before falling through to models.dev (step 5b). This ensures account-specific models get correct context windows while standard models still have a fallback. Part 1 of #7731. Refs: #7272
nekorytaylor666
pushed a commit
to nekorytaylor666/hermes-agent
that referenced
this pull request
Apr 24, 2026
… resolver The Copilot provider resolved context windows via models.dev static data, which does not include account-specific models (e.g. claude-opus-4.6-1m with 1M context). This adds the live Copilot /models API as a higher- priority source for copilot/copilot-acp/github-copilot providers. New helper get_copilot_model_context() in hermes_cli/models.py extracts capabilities.limits.max_prompt_tokens from the cached catalog. Results are cached in-process for 1 hour. In agent/model_metadata.py, step 5a queries the live API before falling through to models.dev (step 5b). This ensures account-specific models get correct context windows while standard models still have a fallback. Part 1 of NousResearch#7731. Refs: NousResearch#7272
justrhoto
pushed a commit
to justrhoto/hermes-agent
that referenced
this pull request
Apr 24, 2026
… resolver The Copilot provider resolved context windows via models.dev static data, which does not include account-specific models (e.g. claude-opus-4.6-1m with 1M context). This adds the live Copilot /models API as a higher- priority source for copilot/copilot-acp/github-copilot providers. New helper get_copilot_model_context() in hermes_cli/models.py extracts capabilities.limits.max_prompt_tokens from the cached catalog. Results are cached in-process for 1 hour. In agent/model_metadata.py, step 5a queries the live API before falling through to models.dev (step 5b). This ensures account-specific models get correct context windows while standard models still have a fallback. Part 1 of NousResearch#7731. Refs: NousResearch#7272
ulasbilgen
pushed a commit
to ulasbilgen/hermes-adhd-agent
that referenced
this pull request
May 1, 2026
… resolver The Copilot provider resolved context windows via models.dev static data, which does not include account-specific models (e.g. claude-opus-4.6-1m with 1M context). This adds the live Copilot /models API as a higher- priority source for copilot/copilot-acp/github-copilot providers. New helper get_copilot_model_context() in hermes_cli/models.py extracts capabilities.limits.max_prompt_tokens from the cached catalog. Results are cached in-process for 1 hour. In agent/model_metadata.py, step 5a queries the live API before falling through to models.dev (step 5b). This ensures account-specific models get correct context windows while standard models still have a fallback. Part 1 of NousResearch#7731. Refs: NousResearch#7272
aj-nt
pushed a commit
to aj-nt/hermes-agent
that referenced
this pull request
May 1, 2026
… resolver The Copilot provider resolved context windows via models.dev static data, which does not include account-specific models (e.g. claude-opus-4.6-1m with 1M context). This adds the live Copilot /models API as a higher- priority source for copilot/copilot-acp/github-copilot providers. New helper get_copilot_model_context() in hermes_cli/models.py extracts capabilities.limits.max_prompt_tokens from the cached catalog. Results are cached in-process for 1 hour. In agent/model_metadata.py, step 5a queries the live API before falling through to models.dev (step 5b). This ensures account-specific models get correct context windows while standard models still have a fallback. Part 1 of NousResearch#7731. Refs: NousResearch#7272
donald131
pushed a commit
to donald131/hermes-agent
that referenced
this pull request
May 2, 2026
… resolver The Copilot provider resolved context windows via models.dev static data, which does not include account-specific models (e.g. claude-opus-4.6-1m with 1M context). This adds the live Copilot /models API as a higher- priority source for copilot/copilot-acp/github-copilot providers. New helper get_copilot_model_context() in hermes_cli/models.py extracts capabilities.limits.max_prompt_tokens from the cached catalog. Results are cached in-process for 1 hour. In agent/model_metadata.py, step 5a queries the live API before falling through to models.dev (step 5b). This ensures account-specific models get correct context windows while standard models still have a fallback. Part 1 of NousResearch#7731. Refs: NousResearch#7272
02356abc
pushed a commit
to 02356abc/hermes-agent
that referenced
this pull request
May 14, 2026
… resolver The Copilot provider resolved context windows via models.dev static data, which does not include account-specific models (e.g. claude-opus-4.6-1m with 1M context). This adds the live Copilot /models API as a higher- priority source for copilot/copilot-acp/github-copilot providers. New helper get_copilot_model_context() in hermes_cli/models.py extracts capabilities.limits.max_prompt_tokens from the cached catalog. Results are cached in-process for 1 hour. In agent/model_metadata.py, step 5a queries the live API before falling through to models.dev (step 5b). This ensures account-specific models get correct context windows while standard models still have a fallback. Part 1 of NousResearch#7731. Refs: NousResearch#7272
gweeteve
pushed a commit
to gweeteve/hermes-agent
that referenced
this pull request
Jun 2, 2026
… resolver The Copilot provider resolved context windows via models.dev static data, which does not include account-specific models (e.g. claude-opus-4.6-1m with 1M context). This adds the live Copilot /models API as a higher- priority source for copilot/copilot-acp/github-copilot providers. New helper get_copilot_model_context() in hermes_cli/models.py extracts capabilities.limits.max_prompt_tokens from the cached catalog. Results are cached in-process for 1 hour. In agent/model_metadata.py, step 5a queries the live API before falling through to models.dev (step 5b). This ensures account-specific models get correct context windows while standard models still have a fallback. Part 1 of NousResearch#7731. Refs: NousResearch#7272
Egavasyug
pushed a commit
to Egavasyug/hermes-agent
that referenced
this pull request
Jun 10, 2026
… resolver The Copilot provider resolved context windows via models.dev static data, which does not include account-specific models (e.g. claude-opus-4.6-1m with 1M context). This adds the live Copilot /models API as a higher- priority source for copilot/copilot-acp/github-copilot providers. New helper get_copilot_model_context() in hermes_cli/models.py extracts capabilities.limits.max_prompt_tokens from the cached catalog. Results are cached in-process for 1 hour. In agent/model_metadata.py, step 5a queries the live API before falling through to models.dev (step 5b). This ensures account-specific models get correct context windows while standard models still have a fallback. Part 1 of NousResearch#7731. Refs: NousResearch#7272
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes the Copilot provider to use live model catalog data instead of hardcoded static values from models.dev. This enables proper support for account-specific models (e.g.
claude-opus-4.6-1mwith 1M context) and enterprise Copilot endpoints.The main issues were:
api.githubcopilot.com)Related Issue
Fixes #7731 — Copilot provider uses hardcoded context windows and lacks token exchange
Also fixes #6455 — adds enterprise endpoint support via token exchange (proxy-ep field in the exchanged token automatically derives the correct API base URL for enterprise accounts)
Also related to #1055 (general Copilot provider improvements)
Type of Change
Changes Made
hermes_cli/copilot_auth.py: Implement token exchange (raw GitHub token -> Copilot API JWT) via/copilot_internal/v2/token, with in-process caching and graceful fallback. Switch OAuth client ID to VS Code's for full model catalog access. Addderive_copilot_base_url_from_token()to extract enterprise endpoints from exchanged tokens. Extract header version constants (_EDITOR_VERSION,_EXCHANGE_USER_AGENT).hermes_cli/models.py: Addget_copilot_model_context_window()that queries the live Copilot/modelsAPI (cached 1h) for real context window values. Add_resolve_copilot_catalog_credentials()returning both token and base URL. Passbase_urlthrough catalog fetch functions for enterprise support.hermes_cli/auth.py: Cache the derived Copilot base URL from token exchange and use it in credential resolution.hermes_cli/model_switch.py: Fetch live Copilot model catalog inlist_authenticated_providers()so account-specific models appear in/modelselection.hermes_cli/runtime_provider.py: Exchange raw token for JWT at runtime and use the derived base URL (individual vs enterprise).agent/model_metadata.py: Add Copilot live catalog lookup before falling back to models.dev for context window resolution.agent/models_dev.py: Override models.dev context_window with live Copilot catalog value for copilot/copilot-acp/github-copilot providers.agent/auxiliary_client.py,run_agent.py: Tighten URL matching to.githubcopilot.com(with dot prefix) for enterprise endpoint compatibility while avoiding false positives.How to Test
hermes setup-> select Copilot providerhermesand use/modelto switch models — account-specific models should appear in the listclaude-opus-4.6-1m— context window should reflect the real value (1M) not the defaultChecklist
Code
pytest tests/ -qand all tests pass (2 pre-existing failures unrelated to this PR)Documentation & Housekeeping
cli-config.yaml.exampleif I added/changed config keys — N/AScreenshots / Logs
Tested end-to-end with GitHub Copilot Individual plan running
claude-opus-4.6-1m(1M context window) via Telegram gateway.