What & why (#1794)

A large gbrain sync killed partway through used to throw away all its progress: last_commit advanced only on full completion, so the next run re-walked a growing diff and never caught up. The reporter's case: a background enrich process added ~40K pages overnight (one page per commit), the next sync had a 44K-file backlog, it got killed at ~16% by a session timeout, banked nothing, and the next hour the backlog was bigger. Mathematically couldn't converge.

This makes incremental sync resumable and fixes the two co-conspirators the review surfaced.

How

Pinned-target path-set checkpoint (commit-watermark was rejected in eng + Codex review for non-linear-history bugs):

• performSyncInner drains a fixed lastCommit..pin range. pin is a pinned target commit held in two op_checkpoints rows (sync = completed file paths, sync-target = [pin]), keyed by syncFingerprint({sourceId, lastCommit}) — keyed on the anchor, never HEAD, so the checkpoint survives a backlog growing underneath it.
• The import loop resume-filters deletes/renames/adds against the banked completedPaths and flushes every GBRAIN_SYNC_CHECKPOINT_EVERY (default 1000).
• last_commit + last_sync_at advance to pin (via commitTimeMs) only at full import completion, then both checkpoint rows clear. A killed/aborted/blocked run banks completed and leaves the anchor untouched — so the source correctly stays "stale" and reschedules.

Pinned target eliminates the staleness window (Codex #3): completion advances to pin, not live HEAD, so commits landing after the pin are a clean next-sync pin..HEAD diff and never get skipped. A file added in-range but deleted from disk after the pin is a SKIP, not a failure. A history rewrite (pin no longer an ancestor of HEAD) discards the checkpoint and re-pins.

Forward-progress head gate: the pre-existing strict HEAD == captured head-drift gate — which blocked the run on any concurrent commit (exactly the "enrich commits every ~2 min" case) — is replaced by git merge-base --is-ancestor pin HEAD. Forward progress is safe; only a real rewrite blocks.

Downstream decoupled (convergence == import convergence): large syncs defer link/timeline extraction to the resumable gbrain extract --stale watermark + the autopilot cycle (and embedding to embed --stale/backfill) instead of running a 44K-page pass inline. Small syncs are unchanged — still inline extract/facts/embed.

Connection-budget clamp (GBRAIN_MAX_CONNECTIONS, opt-in): caps a single sync's worker fan-out so it can't exhaust a low-cap pooler (Supabase's 20-client default) and starve its own retries. gbrain doctor's new pool_budget check nudges toward GBRAIN_POOL_SIZE=2 when the math doesn't fit.

Single-flight: performSync throws a typed SyncLockBusyError; the Minion sync handler marks the job skipped (not failed) on a held lock, so a cron/autopilot tick defers to the holder without polluting failed-job/crash metrics.

Commits (bisectable)

1. feat(sync) — checkpoint primitives (syncFingerprint, commitTimeMs, connection-budget clamp)
2. feat(sync) — resumable performSyncInner (pinned target) + SyncLockBusyError skip in the Minion handler
3. feat(doctor) — pool_budget check + ops-category registration
4. test(sync) — 13-case resumable-sync regression suite + vanished-file contract update
5. chore — version bump + CHANGELOG + CLAUDE.md + llms

Tests

• New test/sync-resumable-import.serial.test.ts (13): convergence regression, resume-skips-checkpointed, pinned-target/forward-drift, history-rewrite re-pin, last_sync_at-not-bumped-on-block + good-file banking, vanished-file skip, dry-run/empty-diff, + pure fingerprint/clamp/pool-budget helpers.
• test/sync-parallel.test.ts vanished-mid-sync test updated to the new skip contract (supersedes the v0.22.13 CODEX-3 failedFiles behavior).
• Full unit suite: 12,923 pass; the only 3 failures were from this change (vanished contract, doctor-category registry, a torn package.json read during a mid-run bump) — all fixed and re-verified.
• bun run verify (typecheck + 29 guards) green.

Reviewed

Eng review (plan-stage) + Codex outside-voice both ran in plan mode — Codex reversed the mechanism from commit-watermark to the pinned-target path-set checkpoint (caught the non-linear-history bug, the scheduler-staleness lie, and the facts-stranding gap). Plan: ~/.claude/plans/system-instruction-you-are-working-modular-kahan.md.

🤖 Generated with Claude Code