Skip to content

v0.41.30.0 fix(brainstorm/lsd): --save writes the advertised .md file via canonical ingestion path#1655

Merged
garrytan merged 4 commits into
masterfrom
garrytan/lsd-save-dual-sink
May 30, 2026
Merged

v0.41.30.0 fix(brainstorm/lsd): --save writes the advertised .md file via canonical ingestion path#1655
garrytan merged 4 commits into
masterfrom
garrytan/lsd-save-dual-sink

Conversation

@garrytan

Copy link
Copy Markdown
Owner

Summary

Supersedes #1634 (closed). Fixes a silent-data-loss bug in gbrain brainstorm/lsd --save and hardens the shared disk write-through.

--save printed Saved to <slug> unconditionally but only did a raw DB engine.putPage:

  1. The advertised committable wiki/ideas/<slug>.md file (promised by the help text) was never written.
  2. The page had no chunks/tags, so gbrain search couldn't find it and the next gbrain sync churned the row.
  3. When the DB write failed under PgBouncer transaction-mode, it still claimed success — a real LSD run reported "saved" while gbrain get <slug> returned page_not_found. Nothing landed anywhere.

The fix routes --save through the same canonical ingestion path the rest of gbrain uses:

  • refactor: extract shared atomic writePageThrough helper — lifts the v0.38 put_page disk write-through out of operations.ts into src/core/write-through.ts, shared by put_page and brainstorm save. Upgraded to atomic write (unique temp sibling + rename, cleanup on failure) so a crash or a concurrent gbrain sync can never read a half-written .md. This hardens put_page's own write-through as a side effect.
  • fix(brainstorm/lsd): --save writes the advertised file via canonical ingestionpersistSavedIdea runs importFromContent({noEmbed:true}) (canonical chunked/tagged row, no embedding cost) then writePageThrough renders the file from the saved row (sinks can't diverge). Pure formatSaveOutcome reports exactly which sinks landed and exits nonzero only when nothing persisted, so a scripted --save can't be mistaken for success. buildIdeaSlug gets a random nonce to stop same-day slug clobber. New buildBrainstormFrontmatterObject feeds serializeMarkdown (the old string builder is left untouched).

The synthesize.ts surrogate-slice change from #1634 was dropped — master already has the canonical safeSplitIndex (v0.41.13) that supersedes it.

Test Coverage

Two new test files: test/write-through.test.ts (helper: written/skip/error branches + ENOTDIR atomicity, no .tmp leftover) and test/brainstorm/save.test.ts (persistSavedIdea + formatSaveOutcome all branches + buildIdeaSlug collision). Includes 3 mandatory regression cases: DB import throws → exit 1 "NOT persisted"; repo set but file write fails → DB-saved + exit 0; the dbSaved && !fileSaved honesty branch can no longer claim "no repo set" when the repo was set.

Gate evidence on this diff's blast radius:

  • bun run typecheck — clean
  • bun run verify — 29/29 CI checks green
  • Curated at-risk suite (17 files: write-through, put_page write-through + provenance + namespace, ingestion/capture, markdown-serializer, operations contract/trust-boundary, brainstorm save, mcp-tool-defs, public-exports) — 254 pass / 0 fail
  • test/brainstorm/ full dir — 112 pass

Note: the full ~780-file bun run test suite was killed by SIGTERM at ~23% on this machine (PGLite WASM cold-starts × 4 shards under load; empty failure log = infrastructure, not test failures). The behavioral change is tightly localized to the put_page write-through block + the new helper + brainstorm save, and the curated suite plus verify/typecheck cover that surface. Recommend a full CI run on the PR to confirm.

Pre-Landing Review

Reviewed via /plan-eng-review (architecture/code/tests/perf) + a Codex outside-voice pass. Codex caught that raw putPage produces a non-canonical row (sync churn + unsearchable) and pointed at the existing put_page write-through; the plan pivoted to reuse it via importFromContent({noEmbed}). All findings resolved before implementation. No open issues.

Plan Completion

All 7 planned tasks DONE: shared helper extraction, put_page rewire, buildBrainstormFrontmatterObject, persistSavedIdea/formatSaveOutcome + CLI wiring, regression tests, buildIdeaSlug nonce, TODOS source-awareness follow-up.

Documentation

CLAUDE.md "Key files" updated with src/core/write-through.ts (v0.41.30.0, NEW) plus v0.41.30.0 notes on the brainstorm and operations entries; llms-full.txt regenerated (bun run build:llms, CI gate verified). CHANGELOG v0.41.30.0 entry with how-to + "To take advantage of" block.

Test plan

  • bun run typecheck clean
  • bun run verify (29 checks) green
  • Curated blast-radius suite 254 pass / 0 fail
  • Full CI suite (recommend confirming on PR — local full run hit machine-load SIGTERM, not failures)

🤖 Generated with Claude Code

garrytan and others added 4 commits May 30, 2026 08:49
Lift the v0.38 put_page disk write-through (operations.ts) into a shared
src/core/write-through.ts helper and upgrade it to write atomically (unique
temp file + rename) so a crash or a concurrent gbrain sync can never read a
half-written .md. put_page now calls the helper; behavior is preserved (repo
guards, source-awareness, provenance overrides) and its write is now atomic.
brainstorm/lsd --save will call the same helper.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…ingestion

--save printed 'Saved to <slug>' unconditionally but only did a raw DB
putPage: the promised wiki/ideas/<slug>.md file was never written, the page
had no chunks (unsearchable, and churned by the next sync), and a failed DB
write under PgBouncer still claimed success.

Route save through importFromContent({noEmbed:true}) for a canonical row, then
the shared writePageThrough helper renders the file from that row. persistSavedIdea
+ formatSaveOutcome report honestly which sinks landed and exit nonzero when
nothing persisted. buildIdeaSlug gets a random nonce so same-day ideas don't
clobber. New buildBrainstormFrontmatterObject feeds serializeMarkdown.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…0.41.30.0)

Add Key Files entry for the new shared atomic src/core/write-through.ts
helper, note the brainstorm/lsd --save canonical-ingestion rewrite on the
brainstorm entry, and note put_page now calls the shared atomic helper on
the operations.ts entry. Regenerate llms-full.txt to match.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@garrytan garrytan merged commit 6397705 into master May 30, 2026
20 checks passed
mgunnin added a commit to mgunnin/gbrain that referenced this pull request Jun 3, 2026
* upstream/master:
  v0.41.36.0 feat(mcp): publish agent skills (list_skills / get_skill) for thin clients (garrytan#1661)
  v0.41.35.0 feat(guardrails): vendor-neutral content guardrail seams (supersedes garrytan#1652) (garrytan#1660)
  v0.41.34.0 feat(search): retrieval cathedral — max-pool + title + alias + evidence (garrytan#1657)
  v0.41.33.0 feat(search): intent-aware adaptive return-sizing + agent-facing query param (garrytan#1640)
  v0.41.32.0 fix(staleness): commit-relative sync staleness (supersedes garrytan#1623) (garrytan#1656)
  v0.41.31.0 feat(embed): delta-aware sync --all cost gate + real stale-embedding semantics (garrytan#1632)
  v0.41.30.0 fix(brainstorm/lsd): --save writes the advertised .md file via canonical ingestion path (garrytan#1655)

# Conflicts:
#	src/core/operations.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant