PR Review: fix: reconfigure MCP stdin/stdout to UTF-8 on Windows (fixes #363)

Executive Summary

Affected Areas: mempalace/mcp_server.py — main() entry point

Business Impact: Unblocks all Windows users encountering non-ASCII content via MCP (#363)

Flow Changes: All sys.stdin.readline() and sys.stdout.write() calls in the main loop now decode/encode as UTF-8 on Windows instead of the system ANSI codepage

Ratings

PR Health

• Has clear description with root cause analysis
• References ticket/issue (Windows: mempalace_search fails with "TextInputSequence must be str in query" on any non-ASCII query due to stdin encoding #363)
• Appropriate size (12 added lines, single concern)
• Has relevant tests — no test added (hard to unit-test platform-specific stdin reconfigure)

High Priority Issues

(Must fix before merge)

None. The fix is correct and well-scoped.

• sys is already imported (line ~13)
• sys.platform == "win32" is the canonical Python Windows check
• reconfigure(encoding="utf-8", errors="strict") is the right call — strict mode prevents silently corrupting data; any UnicodeDecodeError is caught by the main loop's except Exception as e handler at line ~943 which logs and continues
• The try/except guard handles Python < 3.7 (no reconfigure()) and non-standard streams
• Placement before the main loop ensures all downstream I/O benefits

Medium Priority Issues

(Should fix, not blocking)

[Error Handling] #1: Silent failure swallows reconfigure errors with no diagnostic

Location: mempalace/mcp_server.py:930 (PR branch) | Confidence: HIGH

If reconfigure() fails on a Windows machine (e.g., frozen executable with custom I/O wrappers), the original bug silently persists with zero diagnostic output. The user gets the same crash as #363 and no clue that the fix was bypassed.

         except Exception:
-            pass  # older Python or non-reconfigurable stream
+            logger.warning("Could not reconfigure stdio to UTF-8: %s", e)

Note: requires renaming the except clause to except Exception as e:.

[Bug] #2: sys.stderr not reconfigured — logger output with non-ASCII may still corrupt on Windows

Location: mempalace/mcp_server.py:928-929 (PR branch) | Confidence: MED

The logger.error(f"Server error: {e}") at line ~943 and logger.exception() calls throughout the file write to stderr. If the exception message contains non-ASCII content (e.g., user-provided memory text in a traceback), stderr still uses the system codepage and could produce mojibake in logs.

             sys.stdin.reconfigure(encoding="utf-8", errors="strict")
             sys.stdout.reconfigure(encoding="utf-8", errors="strict")
+            sys.stderr.reconfigure(encoding="utf-8", errors="strict")

Lower priority since it only affects log readability, not protocol correctness.

Low Priority Issues

(Nice to have)

None.

Flow Impact Analysis

main()
  ├── [NEW] sys.stdin.reconfigure(utf-8)   ← only on win32
  ├── [NEW] sys.stdout.reconfigure(utf-8)  ← only on win32
  │
  ├── sys.stdin.readline()     ← now decodes as UTF-8 (was system codepage)
  │     └── json.loads(line)   ← receives clean UTF-8 strings
  │           └── handle_request(request)
  │                 └── TOOLS[...]["handler"](**tool_args)
  │                       └── ChromaDB → HuggingFace tokenizer  ← no more surrogate escapes
  │
  └── sys.stdout.write(json.dumps(response))  ← now encodes as UTF-8

Blast radius: Zero on non-Windows. On Windows, every stdin read and stdout write in the main loop is affected — this is the intended behavior.

Created by Octocode MCP https://octocode.ai

Applied both: stderr is now reconfigured alongside stdin/stdout, and the except clause logs the reason instead of silently passing.

Edit 2026-05-03: two follow-ups landed since the above.

ea816f9 refined the policy from blanket errors="strict" to per-stream: stdin=surrogateescape (so a malformed byte from a misbehaving MCP client survives as a lone surrogate the downstream json.loads exception path can surface as a clean -32700 instead of UnicodeDecodeError killing the readline loop on the first bad byte), stdout/stderr stay strict (server-controlled writes, failures stay loud). Adds two mocked Windows tests pinning the per-stream policy.

dbdf5e6 extracts the per-stream loop to mempalace/_stdio.py so the CLI / fact_checker entry points (#1282) share the same helper -- internal-only refactor, no public API change. The MCP-side reconfigure now reads as a one-line reconfigure_stdio_utf8_on_windows(on_failure=...) call with the strict defaults, while the CLI side overrides stdout/stderr to replace.

Rebased on develop now that #1060 has landed. Repositioned this PR's scope: with mempalace/_stdio.py shipped on develop, this PR wires mcp_server.py:main() through that shared helper instead of keeping a parallel inline reconfigure block. The helper's docstring specifies strict for stdout/stderr (server-controlled JSON-RPC; encode failures should be loud) and surrogateescape for stdin (a malformed byte survives as a lone surrogate so the json.loads error path can surface a clean -32700 instead of UnicodeDecodeError killing readline). The current inline block uses errors="replace" for stdin + stdout only with no stderr coverage; this PR brings main() in line with that helper's policy.

If the lighter replace form is preferred, happy to close -- the bug itself is functionally fixed by #1060.