github-copilot: live catalog discovery via /models + add gpt-5.5#79566
Merged
steipete merged 4 commits intoMay 9, 2026
Conversation
Contributor
|
Codex review: needs changes before merge. Summary Reproducibility: not applicable. as a feature PR rather than a bug report. Source inspection does confirm the current-main baseline: the catalog hook returns an empty model list, the manifest is static, and Real behavior proof Next step before merge Security Review findings
Review detailsBest possible solution: Keep the provider-owned dynamic catalog design, route the Copilot Do we have a high-confidence way to reproduce the issue? Not applicable as a feature PR rather than a bug report. Source inspection does confirm the current-main baseline: the catalog hook returns an empty model list, the manifest is static, and Is this the best way to solve the issue? No, not as written: the provider plugin is the right boundary and the fallback behavior is maintainable, but the new network discovery must use the existing guarded fetch pattern before merge. Full review comments:
Overall correctness: patch is incorrect Security concerns:
Acceptance criteria:
What I checked:
Likely related people:
Remaining risk / open question:
Codex review notes: model gpt-5.5, reasoning high; reviewed against 612e72ebbd43. |
5475683 to
8eca29d
Compare
a6c50de to
2e4ed4b
Compare
2e4ed4b to
fe5eae0
Compare
The plugin's `catalog.run` hook already exchanged a GitHub OAuth token
for a short-lived Copilot API token and resolved the per-account baseUrl,
but it returned `models: []` and the bundled openclaw runtime relied
entirely on the static manifest catalog. That meant:
- Static `contextWindow` values were a conservative 128k for every
model, far below reality (gpt-5.4/5.5 are 400k, claude-opus-4.6/4.7
internal variants are 1M, claude-sonnet-4 is 200k, etc.).
- Newly published Copilot models (gpt-5.5, gpt-5.1*, gemini-3-pro-preview,
the claude-opus-*-1m internal variants, etc.) didn't appear at all
until the manifest was patched.
- Per-account entitlement was invisible — every user saw the same
hardcoded 22-model list regardless of plan.
Wire it up:
- Add `fetchCopilotModelCatalog` in `extensions/github-copilot/models.ts`.
Calls `${baseUrl}/models` with the resolved Copilot API token and the
same Editor-Version / Copilot-Integration-Id headers used elsewhere in
the plugin. Maps each entry to a `ModelDefinitionConfig`:
- `contextWindow` ← `capabilities.limits.max_context_window_tokens`
- `maxTokens` ← `capabilities.limits.max_output_tokens`
- `input` ← `["text", "image"]` if `supports.vision`, else `["text"]`
- `reasoning` ← `Array.isArray(supports.reasoning_effort) && supports.reasoning_effort.length > 0`
- `api` ← `anthropic-messages` for Anthropic vendor or claude*
ids; otherwise `openai-responses`
Filters out non-chat objects (embeddings) and internal routers
(`accounts/...` ids). Dedupes by id. 10s default timeout.
- Update the `catalog.run` hook in `extensions/github-copilot/index.ts`
to call the new function after token-exchange and return the live
results. On any HTTP/parse failure it falls back to `models: []`,
which preserves the static manifest catalog as the visible fallback —
no behavior regression for users with `discovery.enabled: false` or
in offline scenarios.
- Bump `modelCatalog.discovery."github-copilot"` from `"static"` to
`"refreshable"` in `openclaw.plugin.json` so the catalog hook is
actually invoked at runtime. Without this the discovery infrastructure
treats the provider as static-only and never calls `catalog.run`.
- Add `gpt-5.5` to the static manifest catalog and `DEFAULT_MODEL_IDS`
with the correct values from the API (`contextWindow: 400000`,
`maxTokens: 128000`, `reasoning: true`, multimodal). This means users
on `discovery.enabled: false` still get gpt-5.5 visible without
needing to override `models.providers.github-copilot.models` in their
config.
Tests added (5, all passing alongside the existing 24):
- `fetchCopilotModelCatalog` maps a representative `/models` response
(chat models incl. an internal 1M-context Anthropic variant, a router,
an embedding) to the right `ModelDefinitionConfig` shape with real
context windows.
- baseUrl trailing slash is normalized.
- Duplicate ids in the API response are deduped (first wins).
- Non-2xx HTTP raises so the caller can fall back to the static catalog.
- Empty token / baseUrl reject synchronously without calling fetch.
Targeted run: `pnpm test extensions/github-copilot/models.test.ts` →
29/29 pass. `pnpm exec oxfmt --check extensions/github-copilot/` clean.
`pnpm tsgo:core` clean.
Real-world proof:
Built locally and dropped the resulting tarball into a downstream
container with `gh auth login --hostname github.com` (Copilot
subscription on the linked account). Before this change,
`openclaw models list --provider github-copilot` returned the 22-entry
static catalog with every entry showing 128k context. After this change,
the same command (with `--refresh`) returns 30 entries with API-accurate
context windows including the new gpt-5.1 family, the claude-opus-*-1m
variants, and the corrected `gemini-3*-preview` ids.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Add an accordion under the Built-in provider tab describing the runtime catalog refresh from the Copilot `/models` endpoint and the `plugins.entries.github-copilot.config.discovery.enabled = false` opt-out for offline / air-gapped scenarios. Pairs with the `fetchCopilotModelCatalog` change so users know what the new behavior is and how to disable it if needed. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Galin asked for shorter changelog entries — collapse the 2 long github-copilot bullets (one per code change) into a single one-line entry that points at the runtime behavior. The PR body retains the full mapping/fallback/header detail. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
fe5eae0 to
058e1b4
Compare
Contributor
|
Landed via rebase onto main.
Thanks @efpiva! |
This was referenced May 12, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two interrelated changes to the bundled
github-copilotprovider plugin:${baseUrl}/models. The plugin'scatalog.runhook already exchanged a GitHub OAuth token for a short-lived Copilot API token and resolved the per-accountbaseUrl, but it returnedmodels: []and the runtime relied entirely on the static manifest catalog. This change addsfetchCopilotModelCatalog(inextensions/github-copilot/models.ts) and calls it from the hook so the runtime picks up real per-account model availability + accurate context windows.gpt-5.5to the static manifest catalog andDEFAULT_MODEL_IDSwith correct values from the API (contextWindow: 400000,maxTokens: 128000,reasoning: true, multimodal) so users ondiscovery.enabled: falsestill get it without overridingmodels.providers.github-copilot.modelsin their config.Also bumps
modelCatalog.discovery."github-copilot"from"static"to"refreshable"inopenclaw.plugin.jsonso the catalog hook is actually invoked at runtime — without this the discovery infrastructure treats the provider as static-only and never callscatalog.run.Why
contextWindowvalues were a conservative 128k for every model, far below reality:How
fetchCopilotModelCatalog(new)In
extensions/github-copilot/models.ts. Calls${baseUrl}/modelswith the resolved Copilot API token and the sameEditor-Version/Copilot-Integration-Idheaders used elsewhere in the plugin. Maps each entry to aModelDefinitionConfig:ModelDefinitionConfigfieldcapabilities.limits.max_context_window_tokenscontextWindowcapabilities.limits.max_output_tokensmaxTokenssupports.visioninput: ["text", "image"] | ["text"]Array.isArray(supports.reasoning_effort) && length > 0reasoning: booleanvendor === "Anthropic"or id matchesclaude*api: "anthropic-messages"else"openai-responses"Filters out non-chat objects (
capabilities.type !== "chat", e.g. embeddings) and internal routers (ids starting withaccounts/). Dedupes by id (first wins). 10s default timeout via internalAbortController. Throws on non-2xx HTTP / parse failure so the caller decides recovery shape.catalog.runhook (modified)In
extensions/github-copilot/index.ts. After the existing token-exchange block resolves bothbaseUrlandcopilotApiToken, the hook now callsfetchCopilotModelCatalog. On any failure it returnsmodels: [], which preserves the static manifest catalog as the visible fallback — no behavior regression fordiscovery.enabled: false, offline scenarios, or unauthenticated users.Manifest
"discovery": { - "github-copilot": "static" + "github-copilot": "refreshable" }Real behavior proof
Behavior addressed: the bundled
github-copilotprovider plugin advertised a static model catalog withcontextWindow: 128000baked in for every entry, regardless of the user's Copilot subscription. Newly available Copilot models were invisible until the manifest was hand-patched. The catalog discovery hook was scaffolded (token exchange already worked) but returned an empty model list, so dynamic discovery did nothing.Real environment tested: WSL2 host running Docker Desktop. Container image built locally from this PR branch with
pnpm pack→npm install -g openclaw-2026.5.6.tgz. Auth viaopenclaw models auth login-github-copilot(real device-flow against a personal github.com Copilot Pro subscription on the maintainer's account). No mocks; the container makes live HTTPS calls toapi.github.com/copilot_internal/v2/tokenandapi.githubcopilot.com/models.Exact steps or command run after this patch: from the worktree containing this PR's branch, build the plugin and pack the tarball with
pnpm install --frozen-lockfile && pnpm build && pnpm pack. Dropopenclaw-2026.5.6.tgzinto a downstream container's build-artifacts directory and rebuild the image (docker compose build && docker compose up -d). Inside the running container, authenticate against a personal Copilot account once (persists in the volume):docker compose exec -it codeclaw-openclaw openclaw models auth login-github-copilot. Then force a refresh against the live Copilot API:docker compose exec codeclaw-openclaw openclaw models list --provider github-copilot --refresh. Then run a single agent turn through the resolved model:docker compose exec codeclaw-openclaw openclaw agent --agent main --message 'Reply with exactly: pong'.Evidence after fix: live terminal output captured from the running container after the refresh:
A subsequent live agent turn against the Copilot API returned the expected reply, with the gateway confirming the route through the github-copilot provider:
Gateway log captured at the same moment, confirming the dynamic-resolved model and provider routing:
Observed result after fix:
models list --refreshnow returns 30 entries with API-accurate context windows. The previously-invisible models surface (gpt-5.1 family, gemini-3-pro-preview, claude-opus-*-1m variants). Existing static-catalog ids that overlap (e.g. gpt-5.4, claude-opus-4.6) get their context windows replaced by the live API values (391k for gpt-5.4, 977k = 1M variant for claude-opus-4.6). gpt-5.5 markeddefaultbecause the downstream container's config uses it as the agents.defaults.model.primary; liveagentinvocation reaches the Copilot API and returns the expected response. No regressions to the static fallback path verified by settingdiscovery.enabled: false(catalog reverts to the bundled 22-entry list with gpt-5.5 included).What was not tested: behavior on github-enterprise (GHE) Copilot tokens — the token-exchange endpoint (
api.github.com/copilot_internal/v2/token) is hardcoded by the existing plugin and neither this PR nor stock plugin support GHE. Tested only on personal github.com Copilot Pro. Streaming / inference for every individual model in the new dynamic catalog was not exercised end-to-end — verified inference for gpt-5.5 only; other entries are exposed via the unchangedprepareRuntimeAuth/wrapStreamFnpaths and rely on the same auth token, so behavior is structurally equivalent to the static-catalog entries today. Plugin behavior when the user's Copilot subscription has zero accessible models was not exercised — code returns an empty array, falling back to the static catalog (same behavior as the oldmodels: []stub).Tests added
5 new cases in
extensions/github-copilot/models.test.ts(29/29 total pass):fetchCopilotModelCatalogmaps a representative/modelsresponse (chat models incl. an internal 1M-context Anthropic variant, a router, an embedding) to the rightModelDefinitionConfigshape with real context windows.fetch.pnpm exec oxfmt --check extensions/github-copilot/clean.pnpm tsgo:coreclean.pnpm exec oxlint extensions/github-copilot/clean against this PR's diff (one pre-existing error inindex.test.tsalready onorigin/main, unrelated to this change).Backwards compatibility
discovery.enabled: false: see exactly the same static catalog as before, plus gpt-5.5.nullearly (existing behavior). No fetch attempted.DEFAULT_COPILOT_API_BASE_URLand an empty model list (existing behavior)./modelsHTTP failure: caught, hook returns empty model list. Static manifest catalog continues to be visible.