v0.42.35.0 fix(sync): recover from unreachable last_commit instead of full-walking forever (#1970)#1975
Merged
Merged
Conversation
… full-walking forever (#1970) When a source's history is rewritten (force-push, master→main consolidation, squash), the recorded last_commit can fall outside HEAD's history. The old guard sent both "object missing" and "not an ancestor" to performFullSync — a full repo re-walk that never advances the bookmark under a cron timeout on a large cross-region brain, so the source goes silently stale. Fix: only a truly-absent object forces a full reconcile. A present-but-non- ancestor bookmark is diffed tree-to-tree directly (git diff A..B needs no ancestry), importing only the real delta. Adds: oversized-diff fallback to full reconcile (F-B); performFullSync now purges deleted files, gated to file-backed pages by source_path so manual/put_page and metafile pages are spared (F-A); rename-to-unsyncable deletes the stale old page (F-C). 7 new PGLite e2e tests. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…elete reconcile in KEY_FILES Update the sync.ts entry to current truth: entry-time bookmark-reachability guard (gc'd anchor → full reconcile; non-ancestor-but-present → direct tree-to-tree diff), oversized-diff fallback, performFullSync now authoritative for deletes (file-backed pages by source_path), and rename-to-unsyncable delete. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
mgunnin
added a commit
to mgunnin/gbrain
that referenced
this pull request
Jun 9, 2026
* upstream/master: v0.42.37.0 fix(security,ingest): source-isolation grant enforcement + non-string frontmatter guard + papercuts (garrytan#1999) v0.42.36.0 fix(sync): resumable, durable, single-flight sync — converges under pool exhaustion + repeated kills (garrytan#1794) (garrytan#1980) v0.42.35.0 fix(sync): recover from unreachable last_commit instead of full-walking forever (garrytan#1970) (garrytan#1975) v0.42.34.0 feat(search): typed-edge relational retrieval — relationship questions get relationship answers (garrytan#1959) docs(designs): add COMMUNITY_IDEAS ledger from open-PR backlog triage (garrytan#1969) v0.42.33.0 fix(sources): confine sync re-clone to gbrain-owned clones; never delete a user working tree (garrytan#1881) (garrytan#1960) # Conflicts: # src/core/operations.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
A source's
last_commitbookmark can point at a commit that no longer sits in HEAD's history — orphaned by a history rewrite (force-push,master→mainconsolidation, squash, branch delete). The entry-guard inperformSyncInnertreated both "object missing" and "not an ancestor" as a reason to fall back toperformFullSync, a full repo re-walk. On a large brain (~150K files) with a cross-region DB (#1958) that walk never finishes inside the cron timeout, so the bookmark never advances and the source goes silently stale with no error surfaced. Sibling of the #1939 poison-file staleness mode.Closes #1970.
The fix
git diff A..Bis an endpoint-tree comparison — it does not require A to be an ancestor of B. So only a truly-absent object should force a full reconcile:merge-base --is-ancestorcheck is demoted to an observability-only log line:[sync] last_commit … not an ancestor of HEAD (history rewritten).git()'s 30s/100 MiB limits) falls back to a full reconcile instead of throwing.performFullSyncis now authoritative for deletes: it purges pages whose backing file is gone, gated to file-backed pages (source_path != null+isSyncable(source_path, strategy)) so manualput_pagepages (nullsource_path) and metafiles are never swept — preserving thegbrain sync --skip-faileddrops bare-markdown log files from the index #1433 invariant.R, notD) now deletes the stale old page.Tests
7 new PGLite tests in
test/sync.test.ts(run in the default unit suite, not the DB-gated e2e shard): orphan-present recovery, orphan-absent fallback, divergence delete, rename-to-unsyncable, full-reconcile delete pass (with explicit assertions that manual + metafile pages survive), oversized-diff fallback (blob-SHA trigger), and post-recovery convergence. The reconcile test caught a real path-format bug (absolute vs repo-relative) during development.Verification
bun run typecheck— cleanbun run verify— 30/30 greenReview trail
/plan-eng-review(pivoted from a date-reseed heuristic to the direct tree-diff) +/codexconsult (independently confirmed thegit diffancestry semantics; surfaced F-A/F-B/F-C, all folded in).🤖 Generated with Claude Code