fix: warn when env vars override ze-switch embedding config#1421
Closed
garrytan-agents wants to merge 1 commit into
Closed
fix: warn when env vars override ze-switch embedding config#1421garrytan-agents wants to merge 1 commit into
garrytan-agents wants to merge 1 commit into
Conversation
Two fixes for the silent env-override bug where ze-switch writes to DB + file config but GBRAIN_EMBEDDING_MODEL env var takes highest precedence in loadConfig(), causing the switch to have no runtime effect. 1. ze-switch (retrieval-upgrade-planner.ts): After applying the switch, check if GBRAIN_EMBEDDING_MODEL or GBRAIN_EMBEDDING_DIMENSIONS are set in the environment and differ from the target. Print a prominent warning box with the exact env vars to update. Without this, users think the switch worked but nothing changed at runtime. 2. doctor (doctor.ts): New 'embedding_env_override' check detects when the env var disagrees with the DB config and warns with the fix command. This surfaces the mismatch on every hourly doctor run instead of just reporting a confusing dimension mismatch. Real-world impact: 716K chunks had to be re-embedded because the env override meant the switch to ZeroEntropy silently didn't take effect. Doctor reported 'openai:text-embedding-3-large' dimension mismatch for hours before the root cause was found.
Owner
|
Closing in favor of fix-wave incorporation on garrytan/pr-1414-1416-1421 (shipping as v0.41.2.1). Rebuilt with /plan-eng-review + codex outside-voice structural improvements:
Credit on the merged commit. |
5 tasks
garrytan
added a commit
that referenced
this pull request
May 25, 2026
…mpotency + ze-switch env-gate (#1445) * fix-wave: dream.* DB merge + batch retry + extract_atoms idempotency + ze-switch env-gate + doctor check Closes PRs #1414, #1416, #1421 (rebuilt from designs by @garrytan-agents with structural improvements from /plan-eng-review + codex outside-voice). Three production reliability fixes in one wave: 1. dream.* DB-config merge (closes PR #1416 silent-config gap) - loadConfigWithEngine() sparse-merge extends with 7 dream.* keys - File > DB > defaults precedence (no GBRAIN_DREAM_* env vars) - extract-atoms switches to loadConfigWithEngine() so DB-plane keys reach it 2. Batch retry on transient connection drops (closes PR #1416 ~30%-loss bug) - withRetry() pure primitive exported from src/commands/extract.ts - 6 flush() sites snapshot-before-clear with onRetry callback - Reuses isRetryableConnError from src/core/retry-matcher.ts - retry-matcher extended with GBrainError{problem:'No database connection'} 3. extract_atoms source-hash idempotency + page-based discovery (closes #1414) - One raw SQL with NOT EXISTS subquery replaces 6 listPages + N atom checks - sourceId threaded through every putPage call (codex caught real bug) - NULL content_hash filter + dream_generated exclusion + transcript-side idempotency - cycle.ts passes union of syncPagesAffected + synthesizeWrittenSlugs 4. ze-switch pre-apply + pre-resume env-override gate (closes PR #1421) - Gate fires FIRST in apply AND resume; zero setConfig calls on refusal - ASCII warning box (no Unicode per repo D10) - --ignore-env-override escape hatch for power users - ApplyResult extended with refused variant 5. doctor embedding_env_override check (defense-in-depth for #1421) - Cross-surface parity: buildChecks() + doctorReportRemote() - Uses Check.details (not Check.issues per codex schema review) Co-Authored-By: garrytan-agents <garrytan-agents@users.noreply.github.com> Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.41.10.0) Adds 61 new tests across 5 new files pinning the fix-wave contracts: - test/extract-batch-retry.test.ts (16 cases) — withRetry primitive + snapshot contract - test/extract-atoms-page-discovery.test.ts (17 cases) — discovery SQL + dual-source idempotency - test/ze-switch-env-override.test.ts (17 cases) — env-gate apply + resume + ZERO-setConfig assertion - test/doctor-embedding-env-override.test.ts (7 cases) — cross-surface parity - test/e2e/extract-atoms-discovery-sql.test.ts (4 cases) — real-Postgres parity for raw SQL Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(test): pin gateway to 1536-dim in 2 PGLite tests that hardcode 1536-vector inserts CI shards 1 + 4 failed persistently (not flake — confirmed via retry) after the v0.41.6.0 merge with this error: error: expected 1280 dimensions, not 1536 file: "vector.c", routine: "CheckExpectedDim" Two test files insert 1536-dim Float32Array vectors into `content_chunks.embedding` / `facts.embedding`, but v0.41.5.0 flipped `DEFAULT_EMBEDDING_DIMENSIONS` from 1536 to 1280 (ZE Matryoshka default). On a fresh CI bun process where no prior test pre-configured the gateway, `initSchema()` sizes the vector column at vector(1280) and the inserts throw. Locally this is hidden when an earlier test file in the shard happens to have called `configureGateway({embedding_dimensions: 1536})` — that state leaks forward through bun's shared process. The v0.41.6.0 LPT shard re-balancing reordered files so these two ran cold, surfacing the latent bug. Fix follows the canonical hermetic pattern from test/consolidate-valid-until.test.ts:23-34: pin the gateway to 1536d in beforeAll, reset in afterAll. Test is now isolated from shard ordering. test/search-types-filter.test.ts — shard 1 fail test/operations-find-trajectory.test.ts — shard 4 (6 fails) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: empty commit to trigger CI * chore: trigger CI again * chore: renumber v0.41.10.0 -> v0.41.10.1 Per request — version slot moved to .1 micro tier to leave .0 available for unrelated wave landing on master. --------- Co-authored-by: garrytan-agents <garrytan-agents@users.noreply.github.com> Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
ze-switchwrites the new embedding model to DB config and~/.gbrain/config.json, butloadConfig()gives highest precedence toprocess.env.GBRAIN_EMBEDDING_MODEL. If the user has this env var set (common in.envfiles loaded at gateway/shell startup), the switch silently does nothing at runtime.The user sees
Switch status: appliedand thinks it worked. But every gbrain command still uses the old OpenAI model because the env var overrides DB + file config.Real-world impact: 716K chunks had to be re-embedded because the env override meant the switch to ZeroEntropy silently didn't take effect. Doctor reported
openai:text-embedding-3-largedimension mismatch for hours before the root cause was traced to stale env vars.Fix
1.
ze-switchenv override warningAfter applying the switch, checks if
GBRAIN_EMBEDDING_MODELorGBRAIN_EMBEDDING_DIMENSIONSare set in the process environment and differ from the target. Prints a prominent warning box:2.
doctorenv/DB mismatch detectorNew
embedding_env_overridecheck: if the env var disagrees with DB config, warns with the exact fix. Surfaces the mismatch on every doctor run instead of reporting a confusing dimension error.Testing
GBRAIN_EMBEDDING_MODEL=openai:text-embedding-3-largein envgbrain ze-switch --non-interactive --forcegbrain doctor --fastembedding_env_overridewarning appears