feat: v0.28.9 pluggable embedding providers — Vercel AI SDK#257
Merged
Conversation
Unified AI layer: src/core/ai/gateway.ts routes every AI call through
Vercel AI SDK. Per-touchpoint provider selection via provider:model
config strings. Six typed recipes (OpenAI, Google, Anthropic, Ollama,
Voyage, LiteLLM-proxy template).
Fixes the silent-drop bug at all three sites (operations.ts:237,
hybrid.ts:81, import-file.ts:112): !process.env.OPENAI_API_KEY →
gateway.isAvailable('embedding'). Non-OpenAI brains now actually
embed. Embedding failures propagate as AIConfigError instead of
quietly writing chunks with no vectors.
Schema templating: getPGLiteSchema(dims, model) substitutes
__EMBEDDING_DIMS__ + __EMBEDDING_MODEL__. Postgres initSchema
runtime-replaces vector(1536) + 'text-embedding-3-large' based on
gateway config. Preserves existing 1536-dim brains via explicit
providerOptions.openai.dimensions passthrough (OpenAI API default
is 3072; without this, existing brains break).
Three-class error hierarchy: AIServiceError (base) + AIConfigError
(user fix) + AITransientError (retry). No process.env mutation —
gateway reads from GatewayContext passed in from engine.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
New command: gbrain providers [list|test|env|explain]. Explain emits a schema_version:1 JSON matrix (agent-friendly). Auto-detects env keys + probes localhost:11434 /v1/models (validates JSON shape, not just port-open). Recommends the best provider with one-line reasoning. gbrain init flags: --embedding-model provider:model (verbose) or --model provider (shorthand, picks recipe default). Plus --embedding-dimensions and --expansion-model. AI config flows into saved GBrainConfig; engine.connect() configures gateway before initSchema so vector column gets right dim. config.ts: adds embedding_model, embedding_dimensions, expansion_model, provider_base_urls. loadConfig() reads env vars but NEVER mutates process.env — global-state leakage would break MCP, multi-brain, and long-running workers. cli.ts: routes 'providers' subcommand (CLI_ONLY, no engine needed); connectEngine() calls configureGateway() before engine.connect(). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…(v0.15.0) 28 new unit tests across 4 files: - test/ai/gateway.test.ts — 13 tests covering isAvailable() matrix for the silent-drop regression surface. Critical case: Gemini available when GOOGLE_GENERATIVE_AI_API_KEY set AND OPENAI_API_KEY absent. Pre-v0.15 brains silently dropped vectors in this config. - test/ai/silent-drop-regression.test.ts — 3 source-level grep tests enforcing !process.env.OPENAI_API_KEY cannot re-enter the codebase at any of the three known sites. - test/ai/schema-templating.test.ts — 4 tests for dim/model substitution in getPGLiteSchema() + PGLITE_SCHEMA_SQL back-compat. - test/ai/config-no-env-mutation.test.ts — regression guard ensuring loadConfig() does not mutate process.env (Codex review C3). All 28 pass locally. Existing unit suite (1397) + Tier 1 E2E (129) + Tier 2 skills E2E (3) all green against real Postgres+pgvector and real OpenAI/Anthropic/openclaw. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Adds AI SDK deps (ai, @ai-sdk/openai, @ai-sdk/google, @ai-sdk/anthropic, @ai-sdk/openai-compatible, zod, gray-matter, eventsource-parser). Note: Version jumped from 0.13.0 to 0.15.0 because upstream master shipped 0.14.x (doctor DRY detection, Knowledge Runtime) while this branch was in development. Keeping 0.15.0 as the natural next release number for the AI providers cathedral. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls upstream v0.14.0 + v0.14.1 (doctor DRY detection + --fix auto-repair, Knowledge Runtime, resolvers, integrity, shell jobs) into the embedding providers branch. Conflicts resolved: - VERSION — kept v0.15.0 (this branch); upstream is v0.14.1 - package.json — v0.15.0 wins - CHANGELOG.md — v0.15.0 entry placed above upstream's v0.14.1/v0.14.0/v0.13.1 - src/cli.ts — CLI_ONLY set merges both new commands; routing includes providers (this branch) + resolvers + integrity (upstream) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
CI failure: test hardcoded /Users/garrytan/... absolute paths that obviously don't exist outside my machine. Resolve paths relative to import.meta.dir so the test works on any checkout + in GitHub Actions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls upstream v0.14.2 (#259 — 8 root-cause fixes from /investigate) into the embedding-providers branch. Test count grows 1717 → 1766 on merge. Conflicts resolved: - VERSION — kept 0.15.0; upstream is 0.14.2 - package.json — v0.15.0 wins - CHANGELOG.md — v0.15.0 entry preserved above upstream's v0.14.2 - src/commands/init.ts — upstream wrapped initPGLite + initPostgres in try/disconnect blocks and moved engine.initSchema() into the try. My aiOpts spreads merged into upstream's refactored config blocks; no duplicate saveConfig calls remain. Build clean: 898 modules, ~160ms compile, 0.15.0 binary runs. 1766 unit tests pass, 0 regressions. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Locked to 0.17.0 since other PRs (v0.15.x, v0.16.x) may land first. Also removes the "v0.15" comment in gateway.ts — the v0.15 label belongs to whatever ships next on master, not this branch. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Re-locked to 0.19.0 (from 0.17.0) to leave room for other PRs landing first. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls upstream v0.15.0 + v0.15.1 + v0.15.2 + v0.15.3 + v0.15.4 + v0.16.0 (llms.txt, hot-issue fixes, bulk-action progress streaming, PgBouncer prepare:false, durable agent runtime). Test count grows 1766 → 2000+ on merge. Unit pass 2000/179skip/3 flaky timeouts; the flakes are pre-existing shared-state setup-hook issues that pass in isolation. Conflicts resolved: - VERSION — kept 0.19.0; upstream is 0.16.0 - package.json — v0.19.0 wins - CHANGELOG.md — v0.19.0 preserved above upstream's v0.16.0/v0.15.x entries - src/cli.ts — CLI_ONLY merges both: upstream's `agent` + this branch's `providers` - src/core/embedding.ts — kept gateway delegation but added upstream's onBatchComplete progress callback via sub-batching (BATCH_SIZE=100) - src/core/operations.ts — kept upstream's subagent namespace enforcement + kept this branch's gateway.isAvailable() silent-drop fix Build clean: 939 modules, ~130ms compile, 0.19.0 binary runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls upstream v0.16.1–v0.18.1: minions worker deploy guide (#287/#317), subagent Anthropic SDK fix + tsc CI gate (#318), check-resolvable CLI (#325), dream + runCycle primitive (#321), multi-source brains with federation + dotfile resolution (#337), RLS hardening + schema backfill (#343). Test count grows 2000 → 2354. Conflicts resolved: - VERSION — kept 0.19.0; upstream is 0.18.1 - package.json — v0.19.0 wins - CHANGELOG.md — v0.19.0 preserved above upstream's v0.18.1/v0.18.0/v0.17.0/v0.16.x - src/cli.ts — CLI_ONLY merges `agent`, `providers`, and upstream's new `sources`, `dream`, `check-resolvable` - src/core/config.ts — merged: kept embedding_model / embedding_dimensions / expansion_model / provider_base_urls (mine) + storage (upstream) Build clean: 948 modules, ~165ms compile, 0.19.0 binary runs. Typecheck green. 18 flaky failures in `bun test` are all PGLite shared-state timeouts in setup hooks — every failing file passes cleanly in isolation (dream 11/0, orphans 35/0, check-update 20/0). Pre-existing infra, not introduced by this merge. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls upstream v0.18.2 (#356): migration hardening + integrity fix + reserved-connection primitive. New withReservedConnection() method on BrainEngine interface auto-merged cleanly into pglite-engine.ts and postgres-engine.ts. Conflicts resolved: - VERSION — kept 0.19.0; upstream is 0.18.2 - package.json — v0.19.0 wins - CHANGELOG.md — v0.19.0 preserved above upstream's v0.18.2 Build clean: 948 modules, ~165ms compile, 0.19.0 binary runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Re-locked to 0.21.0 (from 0.19.0) to leave room for other PRs landing first. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls upstream v0.19.0 (#326): check-resolvable OpenClaw skills-dir fallback + docs/tests. Upstream skipped v0.20 to leave room for this branch at v0.21. Conflicts resolved: - VERSION — kept 0.21.0; upstream is 0.19.0 - package.json — v0.21.0 wins - CHANGELOG.md — v0.21.0 preserved above upstream's v0.19.0 - src/cli.ts — CLI_ONLY merges both: my `providers` + upstream's new `skillpack`, `routing-eval`, `skillify` Build clean: 948+ modules, 0.21.0 binary runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls upstream v0.19.1 (#369): smoke-test skillpack (post-restart health + auto-fix). New `smoke-test` command added to CLI_ONLY. Conflicts resolved: - VERSION — kept 0.21.0; upstream is 0.19.1 - package.json — v0.21.0 wins - CHANGELOG.md — v0.21.0 preserved above upstream's v0.19.1 - src/cli.ts — CLI_ONLY merged: upstream's `smoke-test` + this branch's `providers` Build clean: 0.21.0 binary runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls upstream v0.20.0 (#195): extract BrainBench to sibling gbrain-evals repo. Evals move out of gbrain proper. Conflicts resolved: - VERSION — kept 0.21.0; upstream is 0.20.0 - package.json — v0.21.0 wins - CHANGELOG.md — v0.21.0 preserved above upstream's v0.20.0 Build clean: 0.21.0 binary runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls upstream v0.20.2 (#364): gbrain jobs supervisor — self-healing worker process manager. Conflicts resolved: - VERSION — kept 0.21.0; upstream is 0.20.2 - package.json — v0.21.0 wins - CHANGELOG.md — v0.21.0 preserved above upstream's v0.20.2 Build clean: 0.21.0 binary runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls upstream v0.20.3 (#379): queue resilience — wall-clock timeouts, backpressure, --no-worker, env concurrency. Conflicts resolved: - VERSION — kept 0.21.0; upstream is 0.20.3 - package.json — v0.21.0 wins - CHANGELOG.md — v0.21.0 preserved above upstream's v0.20.3 Build clean: 0.21.0 binary runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Pulls upstream v0.20.4 (#381): merge gbrain-jobs into minion-orchestrator — single unified minions skill. Conflicts resolved: - VERSION — kept 0.21.0; upstream is 0.20.4 - package.json — v0.21.0 wins - CHANGELOG.md — v0.21.0 preserved above upstream's v0.20.4 Build clean: 0.21.0 binary runs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…providers # Conflicts: # CHANGELOG.md # bun.lock # package.json # src/cli.ts # src/core/embedding.ts # src/core/import-file.ts
…providers # Conflicts: # CHANGELOG.md # VERSION
kengwei
added a commit
to kengwei/gbrain
that referenced
this pull request
Apr 26, 2026
Two GitHub Actions workflows on kengwei/gbrain to track divergence from garrytan/gbrain (the upstream the local-patches branch is 'kept local until that ships' per ~/.gbrain/CLAUDE.md). - upstream-pr-257-watcher: weekly check of garrytan/gbrain PR garrytan#257 (v0.17 pluggable embedding providers). When merged, opens a sunset issue with the checklist for retiring kengwei/local-patches. - upstream-drift-report: daily diff of master vs upstream/master. Opens (or comments on) a tracking issue summarizing new upstream commits not yet merged into the fork. Both workflows use the default GITHUB_TOKEN with issues:write + contents:read scope. No external secrets needed (garrytan/gbrain is public).
…providers # Conflicts: # CHANGELOG.md # VERSION # package.json # src/cli.ts # src/core/pglite-engine.ts # src/core/postgres-engine.ts
…providers # Conflicts: # CHANGELOG.md # VERSION # bun.lock # package.json # src/cli.ts # src/commands/init.ts
…providers # Conflicts: # CHANGELOG.md # VERSION # bun.lock # package.json # src/cli.ts
…providers # Conflicts: # CHANGELOG.md # VERSION # package.json
Foundation for multi-provider Minions. Purely additive — no behavior change
to existing embedding/expansion paths or to subagent.ts.
- types.ts: 'chat' added to TouchpointKind. New ChatTouchpoint shape with
supports_subagent_loop separate from supports_tools (Codex F-OV-2: some
chat-capable models are bad at durable tool loops). supports_prompt_cache
gates Anthropic-specific cacheControl. AIGatewayConfig gains chat_model
+ chat_fallback_chain.
- Recipe.aliases?: Record<string,string> (Codex F-OV-5). Friendly undated
forms like 'anthropic:claude-sonnet-4-6' resolve to the dated canonical
at parse time.
- recipes/anthropic.ts, openai.ts, google.ts: each gains a chat touchpoint.
Only Anthropic claims supports_prompt_cache=true.
- recipes/deepseek.ts, groq.ts, together.ts: NEW openai-compat recipes.
DeepSeek powers refusal-fallback + cheap-research. Groq is the speed
tier. Together is the open-weights house (Qwen, Llama-3.3-70B-Turbo).
- gateway.ts: chat() function wraps Vercel AI SDK's generateText. Returns
a provider-neutral ChatResult with normalized usage (input/output +
cache_read/cache_creation pulled from providerMetadata.anthropic per
D7 review decision). cacheSystem: ephemeral marker only when
recipe.supports_prompt_cache===true. Stop-reason mapping is
structural-signal-first per D8 (Anthropic stop_reason='refusal',
OpenAI finish_reason='content_filter') — refusal regex layer ships
in commit 3.
- config.ts: GBrainConfig adds chat_model + chat_fallback_chain. Env
overrides GBRAIN_CHAT_MODEL + GBRAIN_CHAT_FALLBACK_CHAIN.
- cli.ts: connectEngine plumbs chat config into configureGateway.
- providers.ts: --touchpoint chat smoke harness. List shows EMBED/EXPAND/
CHAT columns. Explain matrix surfaces chat options with input/output
cost. Recipe alias forms accepted in --model.
- init.ts: --chat-model PROVIDER:MODEL flag.
- test/ai/gateway-chat.test.ts: 21 cases covering recipe registry,
resolver alias resolution, config plumbing, isAvailable('chat')
semantics for chat-only/embedding-only providers.
49/49 ai/* tests pass. Typecheck clean.
D11 cross-model resolution. Codex F-OV-1 noted that subagent_messages and subagent_tool_executions store Anthropic-shaped tool_use / tool_result blocks as JSONB. When a worker resumes mid-loop and the live model is OpenAI/DeepSeek, the persisted shape becomes the runtime contract — read-side translation is lossy. Mechanical schema-only migration. No code uses these columns yet; commit 2 (subagent refactor onto gateway.chat()) starts writing schema_version=2 with provider-neutral ChatBlock[] in content_blocks. - migrate.ts: v34 ALTERs subagent_messages + subagent_tool_executions to add schema_version (DEFAULT 1) and provider_id (TEXT). All ALTERs use ADD COLUMN IF NOT EXISTS so re-runs are idempotent. - src/schema.sql + pglite-schema.ts: fresh-install DDL gains the same columns. New idx_subagent_messages_provider for cost rollups + per- provider replay diagnostics. - schema-embedded.ts: regenerated via bun run build:schema. - test/migrate.test.ts: 7 new cases pin the migration shape — column names + types, idempotency, fresh-install schema parity, embedded schema parity. 75/75 migrate tests pass. Existing rows backfill to schema_version=1 via DEFAULT, tagging them as legacy Anthropic shape. Subagent.ts read path (commit 2) checks the version and dispatches the right block mapper.
…providers # Conflicts: # CHANGELOG.md # VERSION # package.json # src/core/migrate.ts
CI's check:privacy gate caught a banned name in src/core/ai/recipes/deepseek.ts:5.
CLAUDE.md (per the privacy rule) bans the private OpenClaw fork name in any
checked-in code. Replaces it with neutral language describing the same
capability ("second hop in a refusal-fallback chain and cheap-research
delegation").
bun run verify now passes locally.
…providers # Conflicts: # CHANGELOG.md # VERSION # package.json
…providers # Conflicts: # CHANGELOG.md # VERSION # package.json # src/core/migrate.ts
1 task
thomaswang-shift
added a commit
to wzh-labs/gbrain
that referenced
this pull request
May 7, 2026
Reconciles four conflicts with the v0.27 AI gateway refactor (garrytan#257) by taking upstream's gateway-aware checks (`isAvailable('embedding')`) over the original PR's hardcoded `getSecret('OPENAI_API_KEY')` checks — the gateway version covers Gemini / Ollama / Voyage brains, not just OpenAI. The secrets-keychain feature stays intact: - `connectEngine()` in src/cli.ts now overlays `getSecret(name)` onto the gateway env snapshot for OPENAI_API_KEY / ANTHROPIC_API_KEY / GROQ_API_KEY before calling configureGateway. Resolved values stay in module-private memory; never written to process.env; never inherited by spawned children. - Direct call sites that didn't migrate to the gateway in v0.27 (transcription.ts, cycle/patterns.ts, cycle/synthesize.ts) still use getSecret() directly. Also regenerates llms.txt + llms-full.txt — fork/master had committed stale output relative to its own v0.28.1 CLAUDE.md content. Verified: typecheck clean; test/secrets.test.ts + test/doctor.test.ts + test/ai/ + test/oauth.test.ts all pass.
vincedk-alt
added a commit
to vincedk-alt/gbrain
that referenced
this pull request
May 16, 2026
…garrytan#203 fully) PR garrytan#1060 closed the flat-shape case (config has `embedding_model` / `embedding_dimensions` but init ignored them). This commit closes the ORIGINAL reproducer in jamebobob's issue body: the v0.10.x NESTED shape `{embedding: {provider, model, dimensions, base_url}}`. v0.27 (PR garrytan#257) flattened the embedding config from nested to top-level fields. No migration was written. Users on v0.10.x configs silently fall through to gateway defaults (OpenAI 1536) on every command because no code path reads the nested shape. PR garrytan#1060's seed step only matched flat fields, so it didn't help users still carrying the original shape. This commit adds a back-compat read in `loadConfig()` that maps the nested shape to flat fields IN MEMORY before the env-merge step: - `embedding.{provider, model}` → `embedding_model: "provider:model"` - `embedding.dimensions` → `embedding_dimensions` - `embedding.{base_url, provider}` → `provider_base_urls[provider] = base_url` Flat fields win when both shapes are present (case 7). The next `saveConfig` serializes only declared flat fields, so the nested object drops out on save. Tests: 2 new cases in test/init-config-first.test.ts (case 6: jamebobob reproducer, case 7: flat-wins-over-nested). Suite now 9 pass / 0 fail. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
v0.27 adds pluggable embedding providers to gbrain via Vercel's AI SDK.
Query expansion (the Haiku call during search) moves to the same seam. Chunking / transcription / enrichment / fail-improve stay on their direct SDK calls for now — those migrate in a follow-up.
The only real takeaway: we took Vercel's AI SDK. That's it. No custom provider code, no hand-rolled adapters. Users pick OpenAI / Google / Ollama / Voyage / any OpenAI-compatible endpoint via
provider:modelconfig strings.Also fixes a silent-drop bug at three sites where non-OpenAI brains were invisibly dropping embeddings on every
put_page.What's in the PR
src/core/ai/gateway module. Patches silent-drop atoperations.ts:237,hybrid.ts:81,import-file.ts:112. Schema templating viagetPGLiteSchema(dims, model)preserves existing 1536-dim brains (OpenAI's API default is actually 3072 — gateway passesproviderOptions.openai.dimensions: 1536explicitly).gbrain providersCLI + init flags + config —list | test | env | explainsubcommands.--explain=jsonemits a schema-versioned choice matrix for agent-driven installs.gbrain init --embedding-model openai:text-embedding-3-largeetc.Dependencies
One vendor: Vercel AI SDK (
ai,@ai-sdk/openai,@ai-sdk/google,@ai-sdk/anthropic,@ai-sdk/openai-compatible). Pluszodfor schema validation. That's it. No source vendoring; upgrades viabun update.Test plan
bun test— 1766 pass / 0 fail (up from 1717 pre-merge)bun run test:e2eper-file with clean DB — 133 pass / 0 fail across 13 filestest/e2e/skills.test.ts) with real OpenAI + Anthropic + openclaw — 3/0bun run build— 65MB binary, 898 modules, ~170ms compile./bin/gbrain --version→ 0.27.0./bin/gbrain providers list/explain/testall workgbrain providers test --model openai:text-embedding-3-large→ 1281ms, 1536 dims!process.env.OPENAI_API_KEYpattern can't re-enter at any of the 3 known sitesFollow-ups (not in this PR)
chunkers/llm.ts,enrichment-service.ts,fail-improve.tstogateway.chunk()/gateway.enrich()/gateway.improve()— low-hanging, gateway stubs already in placetranscription.tsonce AI SDK'sexperimental_transcribestabilizes (or route through@ai-sdk/openai-compatiblefor Groq whisper)gbrain migrate --embedding-model <id>with HNSW-aware dim migrationCloses
All 8 community PRs close with config recipes for the author's chosen provider:
operations.ts:237bug find)encoding_format: float)🤖 Generated with Claude Code