Skip to content

Commit bf0a4d0

Browse files
jpheinjphein
committed
docs(fork): document .sh shim delegation strategy + sample shim (#69)
Counter-position to upstream MemPalace#1069. Upstream wants to consolidate the per-event hook .sh wrappers into shims that delegate to `mempalace hook run`. This fork went the opposite direction post-2026-05-11: shims delegate to `palace-daemon/clients/hook.py`, which talks HTTP to a single FastAPI gateway — `mempalace` is no longer in the hook call path. docs/fork-decisions/sh-shim-strategy.md captures: - why the .sh shims stay (back-compat, ops, graceful absence) - why delegation goes through palace-daemon (single-writer gateway) - the delegation pattern diagram - a copy-paste template - re-converge conditions scripts/mempalace-search.sh is a non-hook sample of the same pattern (HTTP → /search), for contributors adding new delegating shims. Refs #69
1 parent 441cb9c commit bf0a4d0

2 files changed

Lines changed: 228 additions & 0 deletions

File tree

‎docs/fork-decisions/sh-shim-strategy.md‎

Lines changed: 166 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,166 @@
1+
# `.sh` shim delegation strategy
2+
3+
**Status:** active fork-ahead decision
4+
**First filed:** 2026-05-22 ([issue #69](https://github.com/techempower-org/mempalace/issues/69))
5+
**Counter-position to:** upstream [MemPalace/mempalace#1069](https://github.com/MemPalace/mempalace/issues/1069)
6+
**Related discussion:** [MemPalace/mempalace#1497](https://github.com/MemPalace/mempalace/issues/1497) — gateway pattern recommendation
7+
8+
## TL;DR
9+
10+
Upstream wants to consolidate the per-event `.sh` hook wrappers
11+
(`mempal_save_hook.sh`, `mempal_precompact_hook.sh`, …) into thin
12+
shims that delegate to **`mempalace hook run --hook <name>`** — i.e.
13+
move all hook logic into the `mempalace` Python CLI.
14+
15+
This fork went the **opposite direction**: the `.sh` shims delegate
16+
to **`palace-daemon/clients/hook.py`** (a stdlib-only Python script
17+
in a separate repo) which talks HTTP to a single FastAPI gateway
18+
(`disks.jphe.in:8085`). The `mempalace` Python package is no longer
19+
in the hook call path at all.
20+
21+
We are not asking upstream to do this. We are documenting the
22+
divergence so future contributors don't "fix" the shims back to
23+
the upstream shape and so the upstream discussion in #1497 has a
24+
working reference.
25+
26+
## Why we keep `.sh` shims
27+
28+
The temptation is to delete them: `hooks.json` can call `python3
29+
/path/to/hook.py` directly, and recent Claude Code releases honor
30+
that. We keep the shims for three concrete reasons:
31+
32+
1. **Backward compatibility with stale Claude Code sessions.** A
33+
Claude Code session loads its hook config once, at startup.
34+
Sessions that started before the 2026-05-11 split-brain fix
35+
still have the *old* hook config in memory — the one that
36+
points at `.claude-plugin/hooks/mempal-stop-hook.sh`. Keeping
37+
the shim as a thin pass-through means those sessions still
38+
route through the daemon instead of erroring out or, worse,
39+
silently writing to a stale local palace.
40+
2. **Operational simplicity.** The shim is the one knob a host
41+
admin can flip without editing JSON. Override `PALACE_DAEMON_HOOK_PY`
42+
to point at a different `hook.py` location (CI fixture,
43+
non-default install path) and every harness that calls the
44+
shim picks it up. No need to chase `hooks.json` per repo.
45+
3. **Graceful absence.** If `palace-daemon/clients/hook.py` isn't
46+
present on this host (fresh checkout, CI runner without the
47+
sibling repo cloned), the shim exits 0. The harness keeps
48+
working; hooks become no-ops. The Python-CLI direction has
49+
no equivalent escape — a missing `mempalace` binary surfaces
50+
as a hook timeout.
51+
52+
## Why delegate to palace-daemon
53+
54+
Three constraints made the daemon the right write authority:
55+
56+
| Upstream #1069 direction | This fork's direction |
57+
|---|---|
58+
| `mempalace hook run` (Python CLI) | `palace-daemon/clients/hook.py` (stdlib Python) |
59+
| Requires `mempalace` import at hook time | Hook has no `mempalace` dep |
60+
| Optional daemon routing via `PALACE_DAEMON_URL` | Always-on, via HTTP |
61+
| Library-level lock in `mempalace` | Single-writer gateway (daemon `_mine_sem`) |
62+
| Vulnerable to CLI version-mismatch hangs (upstream #1465) | Sidestepped — no CLI in the call path |
63+
| Failure mode if daemon down: undefined | Hook returns `{}` (silent), no error |
64+
65+
The single-writer-gateway property is the load-bearing one.
66+
With 300K+ drawers and multiple harnesses (Claude Code, codex,
67+
gemini-cli, opencode, the MCP server) all writing concurrently,
68+
serialization through a single FastAPI process running on
69+
`disks.jphe.in:8085` is the only thing that keeps ChromaDB's
70+
HNSW from corrupting under concurrent mine jobs (upstream #1161).
71+
72+
## The delegation pattern
73+
74+
```
75+
Claude Code Stop event
76+
│
77+
▼
78+
hooks.json: "command": "/path/to/mempal-stop-hook.sh"
79+
│
80+
▼
81+
mempal-stop-hook.sh ◄── thin .sh shim, ~5 lines of logic
82+
│
83+
│ exec python3 $HOOK_PY --hook stop --harness claude-code
84+
▼
85+
palace-daemon/clients/hook.py ◄── stdlib Python, no mempalace import
86+
│
87+
│ urllib.request → POST http://disks.jphe.in:8085/mine
88+
▼
89+
palace-daemon (FastAPI) ◄── single writer, _mine_sem=1
90+
│
91+
▼
92+
mempalace.mcp_server (in-process)
93+
│
94+
▼
95+
postgres + pgvector + AGE (on disks.jphe.in)
96+
```
97+
98+
Two properties matter:
99+
100+
- **The shim doesn't know about MemPalace internals.** It just
101+
forwards `argv` to `hook.py` and respects the
102+
`PALACE_DAEMON_HOOK_PY` override.
103+
- **`hook.py` doesn't know about the database.** It speaks HTTP
104+
to the daemon. Schema changes, backend swaps (chroma→pgvector,
105+
pgvector→whatever's next) don't ripple through hook code.
106+
107+
## Shim template
108+
109+
Three live shims in this repo follow the same shape — copy
110+
them when adding a new harness:
111+
112+
```bash
113+
#!/bin/bash
114+
# MemPalace <event> Hook — thin wrapper delegating to palace-daemon's hook.py.
115+
#
116+
# Override the hook.py location with PALACE_DAEMON_HOOK_PY=/path/to/hook.py
117+
# — required on hosts where palace-daemon lives somewhere other than the
118+
# default below (e.g. CI fixtures, alternate install paths).
119+
#
120+
# If palace-daemon's hook.py is missing on this machine, we exit 0 (not
121+
# a hard error) so a <event> event from a host without palace-daemon
122+
# doesn't gum up the harness.
123+
124+
HOOK_PY="${PALACE_DAEMON_HOOK_PY:-/home/jp/Projects/palace-daemon/clients/hook.py}"
125+
if [ -x "$(command -v python3)" ] && [ -f "$HOOK_PY" ]; then
126+
exec python3 "$HOOK_PY" --hook <event> --harness <harness> "$@"
127+
fi
128+
exit 0
129+
```
130+
131+
Live examples:
132+
133+
- `.claude-plugin/hooks/mempal-stop-hook.sh` — `--hook stop --harness claude-code`
134+
- `.claude-plugin/hooks/mempal-precompact-hook.sh` — `--hook precompact --harness claude-code`
135+
- `.codex-plugin/hooks/mempal-hook.sh` — generic codex shim (hook name from `$1`)
136+
137+
`SessionStart` is registered directly in `hooks.json` (no shim) —
138+
it predates the split-brain fix and didn't have a stale-session
139+
problem to solve. New harnesses can follow either pattern.
140+
141+
## Sample: search-via-daemon
142+
143+
For CLI use beyond hooks, the same delegation pattern works for
144+
read operations. `scripts/mempalace-search.sh` (see file) is a
145+
1:1 demo: HTTP GET against `/search`, fall back silently if the
146+
daemon is unreachable. The daemon's API surface (from
147+
`palace-daemon/main.py`) covers everything a shim might want to
148+
expose: `/search`, `/list`, `/stats`, `/mine`, `/silent-save`,
149+
`/memory`, `/graph`, `/cypher`, `/health`.
150+
151+
## When to re-converge
152+
153+
This decision is reversible. Re-converge with upstream's #1069
154+
if any of the following becomes true:
155+
156+
- Upstream's gateway recommendation in #1497 lands with the
157+
same single-writer property and a Python entry point that's
158+
cheap to call (no cold-start cost).
159+
- The `mempalace` CLI grows a daemon-aware mode that doesn't
160+
require a separate sibling repo to install.
161+
- All harnesses we care about migrate off `.sh` hook configs
162+
(so the back-compat reason evaporates) AND the daemon's
163+
HTTP entry point becomes the *upstream* recommended pattern.
164+
165+
Until then, the shims stay thin, the daemon stays the writer,
166+
and the `mempalace` package stays out of the hook call path.

‎scripts/mempalace-search.sh‎

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
#!/usr/bin/env bash
2+
# mempalace-search.sh — thin .sh shim demonstrating palace-daemon delegation
3+
# for a non-hook use case (CLI search).
4+
#
5+
# See docs/fork-decisions/sh-shim-strategy.md for the broader rationale.
6+
# This script exists as a copy-paste template for adding new delegating
7+
# shims, not as a daily-driver search command — most users will hit the
8+
# daemon through the mempalace MCP server or `mempalace search` CLI.
9+
#
10+
# Usage:
11+
# mempalace-search.sh <query> [limit]
12+
#
13+
# Examples:
14+
# mempalace-search.sh "pgvector cutover"
15+
# mempalace-search.sh "Claude Code hook timeout" 20
16+
#
17+
# Env:
18+
# PALACE_DAEMON_URL default http://disks.jphe.in:8085
19+
# PALACE_API_KEY optional Bearer token; omitted from header if empty
20+
#
21+
# Exit codes:
22+
# 0 daemon answered (results printed as JSON on stdout)
23+
# 1 bad usage (missing query)
24+
# 2 daemon unreachable or returned non-2xx
25+
#
26+
# Requires: curl, jq (optional; raw JSON if jq missing).
27+
28+
set -euo pipefail
29+
30+
if [ $# -lt 1 ]; then
31+
echo "Usage: $0 <query> [limit]" >&2
32+
exit 1
33+
fi
34+
35+
query="$1"
36+
limit="${2:-5}"
37+
daemon_url="${PALACE_DAEMON_URL:-http://disks.jphe.in:8085}"
38+
api_key="${PALACE_API_KEY:-}"
39+
40+
# Encode the query — daemon /search reads it from a query parameter.
41+
# python3 is more portable than relying on jq for url-encoding.
42+
encoded_query=$(python3 -c 'import urllib.parse, sys; print(urllib.parse.quote(sys.argv[1]))' "$query")
43+
44+
auth_header=()
45+
if [ -n "$api_key" ]; then
46+
auth_header=(-H "Authorization: Bearer $api_key")
47+
fi
48+
49+
# --fail makes curl exit non-zero on 4xx/5xx; -s silences progress;
50+
# --max-time keeps a stuck daemon from hanging the caller.
51+
response=$(curl -sS --fail --max-time 10 \
52+
"${auth_header[@]}" \
53+
"${daemon_url}/search?q=${encoded_query}&limit=${limit}" 2>&1) || {
54+
echo "palace-daemon unreachable at ${daemon_url}: ${response}" >&2
55+
exit 2
56+
}
57+
58+
if command -v jq >/dev/null 2>&1; then
59+
echo "$response" | jq .
60+
else
61+
echo "$response"
62+
fi

0 commit comments

Comments
 (0)