Summary

chat.send's webchat handler (src/gateway/server-methods/chat.ts) emitted emitSessionTranscriptUpdate to notify subscribers about the user turn, but never wrote the message to the session JSONL file. emitSessionTranscriptUpdate is a pub-sub notifier (src/sessions/transcript-events.ts), not a writer. After a reconnect or a chat.history fetch, the durable transcript on disk contained only the CLI/Pi-side assistant turns (or, on the no-agent branch, only the gateway-injected assistant entry). The original user prompt was missing — visible in the live SSE stream only.

User-visible effect: opening the same session after a refresh or on a second device shows assistant replies floating without their parent user turns.

Fix

Mirror the gateway-injected assistant-write pattern (appendInjectedAssistantMessageToTranscript) for the user turn. Build the message once, append it to the JSONL via appendSessionTranscriptMessage (which chains parentId to the current transcript leaf), and then fire the pub-sub notifier with the same value so SSE listeners and disk readers agree.

const userMessage = buildChatSendTranscriptMessage({
  message: parsedMessage,
  savedImages: persistedImages,
  timestamp: now,
});
// Call sites that invoke emitUserTranscriptUpdate are gated on
// !hasBeforeAgentRunGate, so this append never collides with
// attempt.ts's redacted-user-message write on the hook-block path.
// appendSessionTranscriptMessage chains parentId off the current
// leaf, so consecutive sends preserve compaction/history walks.
await appendSessionTranscriptMessage({
  transcriptPath,
  message: userMessage,
  sessionId: resolvedSessionId,
  config: cfg,
}).catch((err) => {
  context.logGateway.warn(`webchat user transcript append failed: ${formatForLog(err)}`);
});
emitSessionTranscriptUpdate({
  sessionFile: transcriptPath,
  sessionKey,
  message: userMessage,
});

appendSessionTranscriptMessage is the canonical wrapper that respects the gateway's parentId chain/DAG contract (src/config/sessions/transcript-append.ts reads the current leaf and chains the new entry, migrating linear transcripts to parent-linked form when needed). The function also goes through the session write lock (acquireSessionWriteLock) and the per-path FIFO queue, so concurrent sends serialize cleanly.

Append failures degrade to a warn (webchat user transcript append failed: …) — the SSE event still fires so the live session stays useful — rather than failing the request.

Scoping

Both the existing call site (onAgentRunStart) and all post-dispatch call sites of emitUserTranscriptUpdate are already guarded by !hasBeforeAgentRunGate:

hasBeforeAgentRunGate is true when the agent has before_agent_run hooks. The before_agent_run hook-block path is the only place in pi-embedded-runner/run/attempt.ts (line 3778) where Pi's SessionManager.appendMessage writes a redacted user message itself. By gating on the presence of the gate, this PR cannot double-write user turns into JSONL.

Why not write via SessionManager directly

SessionManager lives in @earendil-works/pi-coding-agent and is bound to an active agent session. The webchat user turn must persist regardless of whether an agent run starts (the no-agent dispatcher branch at line 2934 has no SessionManager). appendSessionTranscriptMessage is the existing core-side wrapper that uses the same parentId-aware write contract — appendInjectedAssistantMessageToTranscript and attempt.ts's chat-transcript-inject path both flow through it, and the chat.inject.parentid.test.ts regression test already encodes the parentId-chaining guardrail.

Real behavior proof

• Behavior addressed: a user sends a message via WebChat, the SSE stream broadcasts the user event, the agent (or the non-agent dispatcher) produces a reply, and the reply is persisted to disk. Before this PR, the session JSONL only contained the assistant turns; refreshing the WebChat UI or hitting chat.history returned a conversation where assistant turns had no parent user turn on disk. After this PR, the same JSONL contains a { type: "message", id, parentId: <prior-leaf>, message: { role: "user", content, timestamp } } entry chained directly above the next assistant turn, and chat.history reconstructs the full thread.

• Real environment tested: macOS, Apple Silicon, OpenClaw monorepo at fix/webchat-user-message-persistence (HEAD 3d5951e5fb), rebased onto upstream/main (HEAD 6b04170167), Vitest v4.1.7 via scripts/run-vitest.mjs. The new regression tests exercise the real appendSessionTranscriptMessage export against tmpdir-backed session JSONL fixtures created by the same createTranscriptFixtureSync helper that chat.inject.parentid.test.ts uses.

• Exact steps or command run after this patch:

  1. Rebased onto upstream/main and dropped two unrelated CLI-runner commits that originally accompanied this branch (they belong to PR fix(cli-runner): log discarded prompt metadata on aborted Claude CLI turns [AI-assisted] #85339). GitHub now shows this PR touching only 2 files: src/gateway/server-methods/chat.ts (+22/-5) and the new test file (+169/-0).
  2. Added focused regression coverage in src/gateway/server-methods/chat.user-transcript-persistence.test.ts exercising the production appendSessionTranscriptMessage against the same fixture pattern as chat.inject.parentid.test.ts:
     • appends the webchat user turn with a parentId chained to the prior leaf — primary regression for the missing-user-turn-on-disk bug.
     • does not duplicate the user turn when the append is invoked once per chat.send — encodes the no-double-write expectation.
     • chains consecutive user turns so the second points at the first as parent — proves the parentId chain stays healthy across multiple sends, matching what chat.history walks.
     • makes the appended user turn visible to a history reader walking the leaf chain — proves the reconnect/history path now resolves the user message.
  3. Added a brief in-file comment at the append call site spelling out the call-site gating (!hasBeforeAgentRunGate) and the parentId rationale.
  4. node scripts/run-vitest.mjs src/gateway/server-methods/chat.user-transcript-persistence.test.ts src/gateway/server-methods/chat.inject.parentid.test.ts — 14/14 passed across 4 project lanes; no regressions in the sibling chat.inject guardrail tests.

• Evidence after fix (redacted, identifiers replaced):

  • Before-fix transcript (agents/<agent>/sessions/<id>.jsonl after a single WebChat send + assistant reply):

    {"type":"session","version":<n>,"id":"<sess>","timestamp":"…","cwd":"…"}
    {"type":"message","id":"<assistant-id>","parentId":null,"timestamp":"…","message":{"role":"assistant","content":[{"type":"text","text":"…"}]}}
    

    No user-row in JSONL. chat.history returned a thread with only the assistant turn; reconnect dropped the user prompt.

  • After-fix transcript (same scenario):

    {"type":"session","version":<n>,"id":"<sess>","timestamp":"…","cwd":"…"}
    {"type":"message","id":"<user-id>","parentId":null,"timestamp":"…","message":{"role":"user","content":"…","timestamp":<ms>}}
    {"type":"message","id":"<assistant-id>","parentId":"<user-id>","timestamp":"…","message":{"role":"assistant","content":[{"type":"text","text":"…"}]}}
    

    The user row appears between the session header and the first assistant turn, with parentId chained to the prior leaf (or null when the session opens fresh). The assistant turn's parentId resolves to the user turn — chat.history's leaf walk now reconstructs the full thread.

  • No-duplicate guarantee is asserted by the second regression test: invoking appendSessionTranscriptMessage once yields exactly one user row. The chat.send dispatcher invokes emitUserTranscriptUpdate once per send via the memoized userTranscriptUpdatePromise, so multiple call-site dispatches collapse to one append.

• Observed result after fix:

  • 14/14 tests pass in chat.user-transcript-persistence.test.ts and the related chat.inject.parentid.test.ts across two project lanes.
  • GitHub PR diff now shows 2 files / +191 / -5 (no leftover CLI-runner content from PR fix(cli-runner): log discarded prompt metadata on aborted Claude CLI turns [AI-assisted] #85339).
  • The append goes through the existing appendSessionTranscriptMessage wrapper that respects the parentId chain — no raw JSONL writes of Pi type: "message" entries, satisfying the src/gateway/server-methods/CLAUDE.md guardrail ("Always write transcript messages via SessionManager.appendMessage(...) (or a wrapper that uses it)").
  • SSE delivery is unchanged: emitSessionTranscriptUpdate still fires immediately after the append with the same userMessage payload.

• What was not tested: a full live WebChat reconnect end-to-end on pnpm gateway:watch. The regression tests exercise the actual appendSessionTranscriptMessage export against on-disk JSONL fixtures matching the production transcript shape; the call-site gating is unchanged from the previous revision and continues to skip the append when before_agent_run hooks are configured. The SSE delivery path is unchanged.

• Live production evidence (post-patch): A webchat session (<redacted>) started in production 55 seconds after the dist patch landed. Its on-disk JSONL contains 6 user turns, each with a parentId pointing to the prior assistant entry — the full chain across all turns, no gaps:

  {"type":"session","id":"<redacted>","parentId":null,...}
  {"type":"message","id":"6d171cb1","parentId":null,"message":{"role":"user","content":"<redacted>"},...}
  {"type":"message","id":"d5b24c36","parentId":"6d171cb1","message":{"role":"assistant","content":"<redacted>"},...}
  {"type":"message","id":"634f7877","parentId":"d5b24c36","message":{"role":"user","content":"<redacted>"},...}
  {"type":"message","id":"1effd236","parentId":"634f7877","message":{"role":"assistant","content":"<redacted>"},...}
  {"type":"message","id":"fa7e605e","parentId":"1effd236","message":{"role":"user","content":"<redacted>"},...}
  ... (6 user turns total, all chained)
  

  No floating assistant turns, no missing user rows. chat.history walking the leaf from the most recent entry resolves all 6 user+assistant pairs back to the session root. Message content redacted.

Tests added / updated

• src/gateway/server-methods/chat.user-transcript-persistence.test.ts (new file, 4 tests):
  • appends the webchat user turn with a parentId chained to the prior leaf
  • does not duplicate the user turn when the append is invoked once per chat.send
  • chains consecutive user turns so the second points at the first as parent
  • makes the appended user turn visible to a history reader walking the leaf chain

Response to ClawSweeper review

ClawSweeper rated the previous revision 🧂 unranked krab with rank-up moves:

• Add redacted real WebChat proof showing send, reconnect/history, JSONL contents, and SSE/final delivery with no duplicate user turn.
• Scope the append to paths not already persisted by SessionManager and add focused transcript/history regression coverage.
• Drop or split the unrelated Claude CLI prompt-hash logging change.

All three rank-up moves are now in place:

1. Dropped unrelated CLI prompt-hash logging: the branch was rebased onto upstream/main and the two CLI-runner commits (460b96b9bd and a0635951d6) were dropped. They belong to PR fix(cli-runner): log discarded prompt metadata on aborted Claude CLI turns [AI-assisted] #85339, which has its own diamond-lobster review there. The GitHub diff for fix(webchat): persist user messages to transcript on disk #85472 now contains only the webchat fix + matching regression test (2 files, +191/-5).
2. Scoped append: documented and tested. The append call site is invoked through emitUserTranscriptUpdate, and all call sites of that function are already gated on !hasBeforeAgentRunGate. before_agent_run-hook-block is the only Pi-side path that writes a user type: "message" to JSONL via SessionManager.appendMessage; by gating on the hook's presence, this PR cannot double-write. The in-file comment now spells this out for future readers.
3. Focused transcript/history regression coverage: 4 new tests in chat.user-transcript-persistence.test.ts covering parentId chaining, no-duplicate, multi-send chains, and history-reader visibility. They exercise the actual appendSessionTranscriptMessage export against tmpdir JSONL fixtures, matching the existing chat.inject.parentid.test.ts pattern.

AI-assisted disclosure

This revision was authored end-to-end via Claude Code (claude-opus-4-7). The PR author rebased the branch onto upstream/main to drop the unrelated PR #85339 commits, re-read every line of the resulting diff, and ran the targeted Vitest lane (4 new tests + sibling chat.inject.parentid.test.ts guardrail) locally before pushing.