What does this PR do?

Fixes #1537. mempalace mine currently prints Done. and exits 0 even when the resulting chroma.sqlite3 left FTS5 in a malformed state. Automation, hooks, and CI then proceed against a palace whose wing-filtered searches will fail with Error finding id. The sibling repair --yes path already runs PRAGMA quick_check and refuses on the same failure (closed via #1216 / #1523); this PR closes the same gap on the mine path.

Change shape

• mempalace/palace.py: new MineValidationError(RuntimeError) plus _validate_palace_fts5_after_mine(palace_path) helper. Helper composes two existing repair-side primitives: _close_chroma_handles(palace_path, backend=_DEFAULT_BACKEND) to release the live writer's handles (Windows mmap guard) and sqlite_integrity_errors(palace_path) to run PRAGMA quick_check on a fresh read-only connection. Raises MineValidationError(palace_path, errors) on any non-ok row.
• mempalace/miner.py: validator call after the topic-tunnels block, before the Done. banner, inside if not dry_run:. A new explicit except MineValidationError: raise clause sits before the existing except Exception as exc: partial-progress handler. Without it the operator would see the misleading "Mine aborted by exception. files_processed: N. last_file: ." banner before the recovery banner, since the loop completed cleanly and validation ran post-loop.
• mempalace/convo_miner.py: symmetric call site. mine_convos has no outer try/except (pre-existing), so the exception bubbles directly to the CLI handler.
• mempalace/format_miner.py: symmetric call site in mine_formats (the --mode extract entry point), placed in the post-loop else: branch after _compute_topic_tunnels_for_wing. Closes a gap that opened on develop after this PR was rebased (see next section).
• mempalace/cli.py: cmd_mine adds an except MineValidationError arm that prints print_sqlite_integrity_abort (the exact recovery banner cmd_repair already prints), appends a mine-specific note to stderr, and exits 1. The arm wraps all three mode branches (project / convos / extract).

Post-#1555 extract-mode coverage

When this PR was opened on 2026-05-18, _mine_impl was the single mine entry point and the validator there covered every mine path. Merge #1555 (3.3.6 release, 2026-05-21) introduced mempalace/format_miner.py::mine_formats as a third mine entry point, wired into cli.py as elif args.mode == "extract":. Without an additional validator call, mempalace mine <dir> --mode extract would have exited 0 on a corrupted palace exactly as the original reporter scenario describes, just from a different operator-facing command. The rebase in this branch resolves the develop conflict in _mine_impl (preserving the upstream hallways + entity-tunnels additions while keeping the validator call at the end) and extends the same _validate_palace_fts5_after_mine helper into mine_formats's else: branch so all three modes share the same end-of-mine integrity contract.

Why reuse repair-side primitives

The same PRAGMA quick_check failure can be reached from two paths: a corrupted-by-repair sqlite (#1517, fixed in #1216 / #1523) and a corrupted-by-mine sqlite (this issue). Operators should see the same authoritative recovery banner regardless of which command surfaced it. Reusing print_sqlite_integrity_abort keeps the wording singular and the recovery steps coherent.

Threaded backend=_DEFAULT_BACKEND

_close_chroma_handles without a backend argument falls back to a transient ChromaBackend() instance whose _clients dict is empty, so it never closes the live writer's PersistentClient. The helper passes palace._DEFAULT_BACKEND so the cached client for the just-mined palace actually gets closed and WAL flushes before the read-only sqlite3 re-open. Matches the backend=backend pattern cmd_repair already uses (mempalace/repair.py:838, :851; mempalace/cli.py:972).

Wording note

The mine-specific stderr note explicitly hedges attribution: "the corruption may pre-date the mine itself." quick_check runs once at end-of-mine, so it cannot distinguish corruption produced by this mine from pre-existing corruption surfaced by it. mempalace repair --yes rebuilds the FTS5 virtual table either way.

How to test

uv sync --extra dev
uv run pytest tests/test_miner_fts5_validation.py -v
uv run pytest tests/ --ignore=tests/benchmarks -q
uvx --from 'ruff==0.15.9' ruff check .
uvx --from 'ruff==0.15.9' ruff format --check .

End-to-end against the actual mempalace binary:

# Happy path: clean mine still exits 0 with Done banner.
rm -rf /tmp/mp1537 && mkdir -p /tmp/mp1537/src
echo "lorem ipsum " > /tmp/mp1537/src/a.md
PALACE=/tmp/mp1537/palace
cd /tmp/mp1537
mempalace --palace "$PALACE" init . --yes
# (mine ran via init or directly)
mempalace --palace "$PALACE" mine .

# Reporter-shape failure: soft FTS5-only corruption.
python3 -c "
import sqlite3
conn = sqlite3.connect('$PALACE/chroma.sqlite3')
rows = conn.execute('SELECT id, block FROM embedding_fulltext_search_data').fetchall()
target = next((r for r in rows if r[0] > 10), rows[0])
conn.execute('UPDATE embedding_fulltext_search_data SET block=? WHERE id=?',
             (b'\\xde\\xad\\xbe\\xef' * (len(target[1]) // 4), target[0]))
conn.commit()
"
mempalace --palace "$PALACE" mine .   # should exit 1 with recovery banner
echo "exit=$?"                          # → 1

The corrupted second run prints ABORT: SQLite-layer corruption detected before repair rebuild plus quick_check output: - malformed inverted index for FTS5 table main.embedding_fulltext_search, verbatim the operator-facing message reporter cited. The --mode extract and --mode convos paths surface the same banner via the shared cmd_mine handler.

Test coverage

17 new cases in tests/test_miner_fts5_validation.py:

1. Helper returns silently on a clean palace.
2. Helper raises MineValidationError on a page-mangled sqlite (corruption recipe mirrors tests/test_repair.py).
3. Helper raises on FTS5-segment-only corruption: the reporter's exact failure mode (chromadb opens cleanly, only the inverted index is malformed). Skips cleanly with pytest.skip if chromadb renames the shadow table in a future version.
4. cmd_mine project mode surfaces MineValidationError as exit 1 + recovery banner.
5. cmd_mine convos mode symmetric.
6. Dry-run skips the validator in project miner (no writes, nothing to check).
7. _close_chroma_handles runs before sqlite_integrity_errors (Windows guard order).
8. Full chain through real _mine_impl (no monkeypatch): mine, corrupt FTS5, re-mine, assert validator was the explicit raise-source via spy.
9. _mine_impl does NOT print the "Mine aborted by exception" partial-progress banner on MineValidationError (the new explicit except MineValidationError: raise clause bypasses the generic handler).
10. MineValidationError constructor rejects empty errors list and blank palace_path (defense-in-depth against future mis-construction).
11. MineValidationError.errors is a tuple (immutable forensic snapshot; handlers cannot mutate).
12. Helper is silent on missing chroma.sqlite3 (palace dir without sqlite file).
13. mine_convos(dry_run=True) skips the validator (mirrors project-miner guarantee).
14. cmd_mine --mode extract surfaces MineValidationError as exit 1 + recovery banner (post-feat(3.3.6): virtual line numbering + format coverage via --mode extract #1555 third entry point).
15. mine_formats(dry_run=True) skips the validator (matches the other two miners).
16. mine_formats on KeyboardInterrupt skips the validator: the per-file except Exception does not catch BaseException, so the interrupt propagates past the else: branch where validation sits.
17. Full chain through real mine_formats (no monkeypatch on the validator itself): build palace, corrupt FTS5, invoke mine_formats, assert the helper raised MineValidationError with the corrupt path in errors.

Out of scope

A codebase-wide audit found additional write paths that print success and exit 0 without validating FTS5. They are deliberately not touched here: each is a separate user-visible surface, and the reporter scope is mempalace mine specifically. A follow-up issue will track adopting the same _validate_palace_fts5_after_mine helper across these sites.

CLI surface:

• cmd_sweep (sweeper.sweep / sweep_directory)
• cmd_sync --apply (sync_palace; deletes can leave FTS5 shadow tables stale)
• cmd_compress (comp_col.upsert of compressed closets)
• dedup_palace (deletes drawers)
• closet_llm.regenerate_closets
• diary_ingest (drawers_col.upsert + upsert_closet_lines)
• migrate.migrate (col.add batch into temp palace, then swap; no post-swap quick_check)
• repair.rebuild_from_sqlite (multi-collection upsert; preflight covers the legacy rebuild_index path but not this sub-mode)

Library surface:

• sources/context.py:PalaceContext.upsert_drawer (shared helper used by source adapters)

MCP surface:

• tool_add_drawer, tool_diary_write, tool_delete_drawer (mcp_server.py:1171). Different surface: JSON-RPC responses rather than CLI exit codes.

The recovery path this PR's stderr note directs operators to (mempalace repair --yes → rebuild_index) is itself safe: it preflights via sqlite_integrity_errors and post-#1523 rebuilds the FTS5 virtual table.

Three pre-existing structural asymmetries that this PR observes but does not address:

• mine_convos does not wrap its body in mine_palace_lock the way mine() does. The validator therefore runs outside the palace lock in convos mode, which means a concurrent writer (e.g. MCP server) could in principle slip in between the last upsert and the validator. The new stderr note hedges attribution rather than falsely blaming the mine.
• mine_formats (introduced by feat(3.3.6): virtual line numbering + format coverage via --mode extract #1555) also lacks mine_palace_lock, with the same concurrent-writer caveat as mine_convos. The validator wired in by this PR runs in the same lock-free zone. Locking parity across the three miners is a separate refactor.
• mine_convos and mine_formats have no outer try/except wrapping their Done. banners, so project mode prints a partial-progress summary on MineValidationError while convos and extract modes print only the recovery banner. Recovering symmetry is a separate refactor.

Checklist

• Tests pass (uv run pytest tests/ --ignore=tests/benchmarks): 2076 passed, 3 skipped
• No hardcoded paths
• Linter passes (uvx --from 'ruff==0.15.9' ruff check . + ruff format --check)

Closes #1537.

Co-authored-by: Caleb Wells 15988028+calebcwells@users.noreply.github.com