fix(hooks): replace mapfile with sed for bash 3.2 compat (#1440)#1441
Merged
igorls merged 4 commits intoMay 17, 2026
Conversation
Contributor
There was a problem hiding this comment.
Code Review
This pull request replaces Bash 4-specific mapfile with sed in the hook scripts to ensure compatibility with macOS Bash 3.2. It also introduces a sentinel-based error detection mechanism, captures Python stderr, and implements a fail-loud guard that logs malformed input (capped at 4KB) with restricted file permissions. A comprehensive test suite is added to verify these changes. The review feedback suggests using printf '%s' instead of echo when passing input to the Python interpreter to handle special characters more robustly.
cfd8248 to
0f6cd65
Compare
0f6cd65 to
09d91a9
Compare
) The legacy hook scripts used `mapfile -t` (introduced in MemPalace#1231) to read sanitized values from the inline Python JSON parser. On stock macOS /bin/bash is GNU 3.2.57 (Apple GPLv3 freeze) and `mapfile` only landed in bash 4.0, so the hook silently failed: every parsed value fell back to its default, the log wrote "Session unknown: 0 exchanges", and zero drawers were saved. This commit replaces the array read with `sed -n 'Np'` line extraction, which is POSIX and runs on /bin/bash 3.2 unmodified. It also adds a defense-in-depth fail-loud guard: Python prints a `__MEMPAL_PARSE_OK__` sentinel and the hook surfaces the raw input plus Python traceback to 0600-mode sibling files when the sentinel is missing, so the next silent failure has a debuggable trail instead of an unsolved zero-drawer day.
Follow-up on c271e7b after the bot review and an adversarial pass. - Replace `echo "$INPUT"` with `printf '%s' "$INPUT"` in both hooks so payloads with leading `-n`/`-e`/`-E` or backslashes are not eaten by echo flag parsing. - Wrap the `last_python_err.log` and `last_input.log` writes in a `( umask 077; ... )` subshell so the files are created at mode 0600 atomically. The existing `chmod 600` calls stay as belt-and-suspenders. - Refactor the `_run_hook` test helper: internal assert on returncode with stderr surfaced in the failure message; new `extra_env=` kwarg subsumes the ad-hoc `subprocess.run` in the UTF-8 locale test. - Promote the world-readable, bounded-overwrite, UTF-8, unicode and literal-unknown tests to a shared `_BOTH_HOOKS` parametrize with readable ids so the symmetry across both hooks is enforced. - `test_sed_extraction_present` strips line comments before counting `sed -n '` occurrences so a regression that swapped sed back to mapfile while keeping the prose cannot false-pass. - Rewrite the precompact hook header docblock: the previous "ALWAYS blocks" wording was stale (the hook returns `{}` and does not emit a `decision: block` payload to the Stop-hook protocol; the "always run" guarantee comes from the synchronous mine call). - Apply ruff 0.4.x format to the new test file (matches CI pin `pip install "ruff>=0.4.0,<0.5"`). Tests: 1749 passed (was 21 cases in c271e7b's test file; now 26 after parametrize promotions, +5 cases).
Cycle 2 follow-up: harden the permission tests so they actively falsify the "ambient umask masking a regression" hypothesis raised during the adversarial review pass. - `_run_hook` now sets `preexec_fn=lambda: os.umask(0o022)` on the bash subprocess. The child starts with a permissive 022 umask so a bare file would land at mode 0644. The hook's own `umask 077` in the parse subshell remains the only thing protecting `last_python_err.log` and `last_input.log`, so `test_dump_is_not_world_readable` and `test_python_stderr_log_is_not_world_readable_on_failure` now empirically prove that protection rather than passively benefitting from a CI runner's restrictive ambient umask. - Docstring precision: the helper's primary diagnostic surface is `hook.log` (a file under `STATE_DIR`), not the subprocess's stderr. Subprocess stderr is captured here only to surface unexpected interpreter failures (e.g., a bash syntax error introduced by a future edit), and the docstring now says so. No production-code changes. 26/26 tests still pass.
9e7228a to
1988f3d
Compare
arnoldwender
pushed a commit
to arnoldwender/mempalace
that referenced
this pull request
May 24, 2026
Bumps version 3.3.5 → 3.3.6 across pyproject.toml, version.py, plugin manifests (.claude-plugin/plugin.json, .claude-plugin/marketplace.json, .codex-plugin/plugin.json), README badge, and uv.lock. Flips CHANGELOG.md from ``[Unreleased]`` to ``[3.3.6] — 2026-05-24`` and backfills the major user-facing entries that landed without changelog entries during the cycle: Features: - MemPalace#1555 office-document mining via --mode extract + virtual line numbers - MemPalace#1584 surgical closet pointers with date+line locators (Tier 6a) - MemPalace#1558 + MemPalace#1560 within-wing hallways (entity co-occurrence graph) - MemPalace#1565 cross-wing tunnels auto-promoted from hallways - MemPalace#1578 Hebbian potentiation + Ebbinghaus decay on hallways/tunnels - MemPalace#1236 API-tool transcripts auto-route to wing_api - MemPalace#711 hooks.auto_save toggle for silent-mode sessions - MemPalace#1605 COCA content-word filter for entity detection - MemPalace#1557 case-insensitive entity matching at mine time - MemPalace#1483 multilingual embeddings (embeddinggemma-300m) by default Bug Fixes (selected, user-visible): - MemPalace#1540 silent data loss in three unchunked upsert sites - MemPalace#1538 paragraph chunker oversized chunks - MemPalace#1554 per-file chunk cap too low for transcripts - MemPalace#1562 Windows hook subprocess/ChromaDB deadlock - MemPalace#1529 create_tunnel corrupted hyphenated wing names - MemPalace#1424 save-hook truncated hyphenated project folders - MemPalace#1383 KG cache duplicated graphs for symlinked/cased paths - MemPalace#1466 silent symlink skip now logged - MemPalace#1441 macOS stock-bash 3.2 hook compatibility - MemPalace#1500 / MemPalace#1513 structured JSON-RPC errors on bad MCP input - MemPalace#1523 VACUUM + FTS5 rebuild after repair - MemPalace#1548 FTS5 validation at end of mine - plus MemPalace#1216, MemPalace#1408, MemPalace#1438, MemPalace#1439, MemPalace#1445, MemPalace#1452, MemPalace#1459, MemPalace#1461, MemPalace#1466, MemPalace#1470, MemPalace#1477, MemPalace#1485, MemPalace#1500, MemPalace#1513, MemPalace#1528, MemPalace#1532, MemPalace#1543, MemPalace#1546, MemPalace#1585 Performance: - MemPalace#1474 convo miner pre-fetches mined-set - MemPalace#1487 rebuild_index progress callback - MemPalace#1530 MCP cold-start diagnostics + opt-in warmup Lint passes (ruff 0.15.14); mempalace-mcp entry point alignment verified per RELEASING.md.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does
The legacy
hooks/mempal_save_hook.shandhooks/mempal_precompact_hook.shusemapfile -t(introduced in #1231) to read sanitized values from the inline Python JSON parser. On stock macOS/bin/bashis GNU 3.2.57 (Apple GPLv3 freeze) andmapfileonly landed in bash 4.0 (2009), so the hook silently fails: every parsed value falls back to its default, the log writesSession unknown: 0 exchanges, and the hook returns{}+ exit 0. Users see hooks "firing" with zero drawers added (#1440).This PR replaces the array read with
sed -n 'Np'line extraction, keeping the same one-value-per-line contract from the #1231 security hardening but staying within POSIX so/bin/bash3.2 runs the scripts unmodified.It also adds a defense-in-depth fail-loud guard. Python now prints a leading
__MEMPAL_PARSE_OK__sentinel; if the sentinel is missing after parsing, the hook writes aWARNline tohook.logand dumps the first 4096 bytes of the raw input tolast_input.logplus the Python traceback tolast_python_err.log, both at mode 0600. This means the next time parsing silently fails for any reason (sanitizer regex too tight, broken interpreter, future inline-script regression), a maintainer has the raw payload and the Python error instead of a day of "Session unknown" log lines.How to test
tests/test_hooks_bash_compat.py(26 cases, mirrored across both hooks where applicable) cover (a)mapfile/readarrayabsent in both scripts, (b)sed -nextraction shape present, (c) hook produces the expected log line under controlled stdin, (d) fail-loud guard fires on malformed JSON but not on empty stdin / unicode-stripped / literal"unknown"session_id, (e) the dump caps at exactly 4096 bytes including under a UTF-8 locale with a multibyte payload, (f)last_input.logandlast_python_err.logare mode 0600 on failure and absent on success, (g) hook stdout stays valid JSON.uv run pytest tests/ -v --ignore=tests/benchmarksstays green.echo '{"session_id":"abc","stop_hook_active":false,"transcript_path":""}' | bash hooks/mempal_save_hook.shand inspect~/.mempalace/hook_state/hook.logforSession abc:.Checklist