Skip to content

feat: add Copilot embedding provider#691

Closed
tonyxu-io wants to merge 2 commits into
garrytan:masterfrom
tonyxu-io:feat/copilot-provider-v027
Closed

feat: add Copilot embedding provider#691
tonyxu-io wants to merge 2 commits into
garrytan:masterfrom
tonyxu-io:feat/copilot-provider-v027

Conversation

@tonyxu-io

@tonyxu-io tonyxu-io commented May 6, 2026

Copy link
Copy Markdown

Summary

  • Adds a native copilot embedding provider for GitHub Copilot / Blackbird Metis embeddings.
  • Supports GBRAIN_EMBEDDING_MODEL=copilot:metis-1024-I16-Binary with 1024-dim vectors.
  • Calls GitHub's /embeddings endpoint directly using the Copilot request/response shape instead of pretending it is OpenAI-compatible.
  • Keeps Copilot scoped to embeddings only; chat/expansion still use the existing providers.

Why

This replaces stale PR #450, which targeted the old v0.22 embedding path. Current master uses the v0.27+ AI gateway / recipe provider system, so this PR is intentionally much smaller and built on latest master.

Test plan

  • bun run typecheck
  • bun test test/ai/copilot-provider.test.ts
  • bun test test/ai/copilot-provider.test.ts test/schema-bootstrap-coverage.test.ts test/operations-allow-list.test.ts test/pages-soft-delete.test.ts
  • Local smoke: gbrain providers list shows copilot ✓ ready
  • Local smoke: real Copilot embedding call returns vector length 1024

Replaces #450.


View in Codesmith
Need help on this PR? Tag @codesmith with what you need.

  • Let Codesmith autofix CI failures and bot reviews

Adds native Copilot embedding provider for GitHub Copilot/Blackbird Metis
embeddings. Calls GitHub /embeddings endpoint directly with the Copilot
request/response shape; not OpenAI-compatible.

- New recipe: src/core/ai/recipes/copilot.ts with metis-1024-I16-Binary
- New implementation kind: 'native-copilot' in types.ts
- gateway.ts: native-copilot branches in instantiateEmbedding/Expansion/Chat
  (embedding does real fetch; chat+expansion throw clear errors since this
  provider is embedding-only)
- embedSubBatch short-circuits when model is a Copilot model, bypassing
  Vercel AI SDK and threading abortSignal into native fetch

Replaces garrytan#450. Built on current master (v0.36) instead of v0.22 legacy path.

Auth: GBRAIN_COPILOT_TOKEN, COPILOT_GITHUB_TOKEN, GH_TOKEN, GITHUB_TOKEN,
or ~/.copilot/config.json (Copilot CLI login).

Config:
  export GBRAIN_EMBEDDING_MODEL=copilot:metis-1024-I16-Binary
  export GBRAIN_EMBEDDING_DIMENSIONS=1024
@tonyxu-io tonyxu-io force-pushed the feat/copilot-provider-v027 branch from e0d584e to 5cc285e Compare May 19, 2026 02:32
Silences ai.gateway warning that the copilot embedding touchpoint had
no batch cap declared. 8192 matches zhipu/dashscope/azure-openai which
all use the same OpenAI-style /embeddings shape.
@garrytan

garrytan commented Jun 8, 2026

Copy link
Copy Markdown
Owner

Thanks for this contribution — and apologies for the slow triage. We did a full pass over the entire PR backlog. gbrain has moved fast, and the maintainer's larger "cathedral" rewrites have superseded a big share of community PRs: the AI gateway + recipes + user_provided_models system replaced almost all individual provider PRs; #1805 fixed the whole Postgres module-singleton class; #1542 unified the type taxonomy; #1657 the retrieval path; #1802 the doctor; and so on.

We're closing this one in that cleanup — either the fix already landed on master, it duplicates another PR or merged change, or it's outside the current merge bar. Where a closed PR carried a genuinely valuable idea, we've recorded it in docs/designs/COMMUNITY_IDEAS.md so nothing good is lost (a few may graduate into TODOs).

Please don't read the close as a judgment of the work — thank you for contributing. If you believe the underlying issue is still live on the latest master, reopen with a quick note and we'll take another look. 🙏

@garrytan garrytan closed this Jun 8, 2026
@tonyxu-io

Copy link
Copy Markdown
Author

Verified on master HEAD 959af106 (v0.42.36.0). The recipes system did not supersede this PR — Copilot embeddings are still missing.

What's actually on master

$ ls src/core/ai/recipes/
anthropic.ts azure-openai.ts dashscope.ts deepseek.ts google.ts groq.ts
index.ts litellm-proxy.ts llama-server-reranker.ts llama-server.ts
minimax.ts ollama.ts openai.ts openrouter.ts together.ts voyage.ts
zeroentropyai.ts zhipu.ts
  • No copilot.ts
  • src/core/ai/types.ts Implementation union has no 'native-copilot'
  • recipes/index.ts ALL[] has no copilot
  • git grep -i copilot src/core/ai/ → 0 hits

The "user_provided_models system" mentioned in the close note is litellm-proxy.ts + llama-server.ts — these let users run a local proxy in front of any provider. That's a real feature, but it's a different ergonomic: a Copilot user has to install LiteLLM, configure it as a Copilot bridge, run it as a separate process, then point gbrain at it. This PR is a direct integration that reads the existing ~/.copilot/config.json Copilot CLI login and calls https://api.github.com/embeddings natively.

What this PR adds

  • 1 new recipe (copilot.ts, +40)
  • 1 new Implementation discriminator ('native-copilot')
  • Native fetch path in gateway.ts that bypasses the Vercel AI SDK because GitHub's request/response shape ({ inputs, model }{ embeddings: [{ embedding }] }) isn't OpenAI-compatible
  • Embedding-only by design — chat/expansion switches throw a clear error
  • Auth from GBRAIN_COPILOT_TOKEN / COPILOT_GITHUB_TOKEN / GH_TOKEN / GITHUB_TOKEN / ~/.copilot/config.json
  • Test mocks fetch and verifies endpoint, headers, body shape, dim handling

I forward-ported this to v0.42.36 locally to confirm the patch shape still applies cleanly (recipes/types.ts/gateway.ts integration points are unchanged from v0.36 → v0.42; only recipes/index.ts needs a 1-line merge alongside the new llama-server-reranker import). Happy to rebase the PR on current master if that's what's blocking.

Why this matters

GitHub Copilot's embedding endpoint is the only zero-config embedding option for users who already pay for Copilot — no separate API key, no extra service, just ~/.copilot/config.json which the Copilot CLI maintains. Asking those users to stand up LiteLLM is a real friction step.

This is +192/-0, fully isolated, tests included. Would appreciate a re-look. 🙏

tonyxu-io added a commit to tonyxu-io/gbrain that referenced this pull request Jun 9, 2026
Adds native Copilot embedding provider for GitHub Copilot/Blackbird Metis
embeddings. Calls GitHub's /embeddings endpoint directly with the Copilot
request/response shape (`{ inputs, model }` → `{ embeddings: [{ embedding }] }`),
bypassing the Vercel AI SDK because the shape is not OpenAI-compatible.

Wires up across all four touchpoints required for `isAvailable('embedding')`
to return true on v0.42:

1. types.ts: add 'native-copilot' to the Implementation union
2. recipes/copilot.ts: new recipe with metis-1024-I16-Binary model (1024d)
3. recipes/index.ts: register copilot in ALL[]
4. gateway.ts: marker-based routing in instantiateEmbedding +
   embedSubBatch short-circuit + clear errors in expansion/chat

Auth chain: GBRAIN_COPILOT_TOKEN, COPILOT_GITHUB_TOKEN, GH_TOKEN,
GITHUB_TOKEN, then ~/.copilot/config.json (Copilot CLI login).

Vectors are L2-normalized before return for pgvector cosine distance.
Dim mismatch is caught immediately with a migrate hint. abortSignal is
threaded into native fetch.

Config:
  export GBRAIN_EMBEDDING_MODEL=copilot:metis-1024-I16-Binary
  export GBRAIN_EMBEDDING_DIMENSIONS=1024

Replaces PR garrytan#691 (closed in batch cleanup); rebased on top of v0.42.36
master with the new gateway.ts retry/timeout/abortSignal infrastructure
plus max_batch_tokens declaration on the recipe (v0.32 batch budget
validation hard-fails recipes without it on first import).
@tonyxu-io

Copy link
Copy Markdown
Author

Rebased on top of master 959af106 (v0.42.36.0). Verified end-to-end on the latest tree.

Push: dbab3987 feat: add Copilot embedding provider (single squashed commit; supersedes the original two).

Verified locally

$ bun test test/ai/copilot-provider.test.ts
2 pass, 0 fail

$ bun test test/ai/
317 pass, 0 fail (1070 expect() calls, 31 files, 1.1s)

The full test/ai/ suite passes — no regressions to existing recipes (anthropic, google, openai, voyage, zeroentropyai, openrouter, litellm-proxy, ollama, …).

What changed during the v0.36 → v0.42 forward-port

  • All 4 wire-up touchpoints verified against current master (Implementation union, recipes/index.ts ALL[], instantiateEmbedding, embedSubBatch short-circuit, plus error-only branches in instantiateExpansion / instantiateChat)
  • embedSubBatch signature now takes the v0.42 EmbedOpts (abortSignal, maxRetries) — threaded into the native fetch through the existing { abortSignal } opts contract on embedCopilot
  • recordSubBatchSuccess(recipe) and getEmbeddingModel() references match the v0.42 names
  • Recipe declares max_batch_tokens: 8192 (v0.32 D8 batch-budget validation hard-fails recipes without it on first import — the gateway loads the whole recipe table at startup)
  • User-Agent simplified from a hardcoded gbrain/0.36.0 to plain gbrain (avoids version drift)

Architecture summary

recipe.implementation = 'native-copilot'
                         │
                         ▼
instantiateEmbedding ──► returns marker object { [COPILOT_MARKER]: true, modelId, cfg }
                         │
                         ▼
embedSubBatch ──► isCopilotModel(model) ──► embedCopilot() ──► fetch(api.github.com/embeddings)
                                                               L2 normalize + dim check

Bypasses the Vercel AI SDK because GitHub's { inputs, model }{ embeddings: [{ embedding }] } shape isn't OpenAI-compatible. Chat / expansion paths throw a clear AIConfigError("Copilot recipe is embedding-only.").

Diff is +213/-0 across 5 files. Would appreciate a re-look. 🙏

@tonyxu-io

Copy link
Copy Markdown
Author

Continued in #1998 (rebase couldn't reopen this PR per GitHub's force-push policy).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants