Skip to content

Gateway/TUI: hide internal memory-flush prompts in chat history#32068

Closed
liuxiaopai-ai wants to merge 2 commits into
openclaw:mainfrom
liuxiaopai-ai:codex/tui-hide-internal-memory-flush-32057
Closed

Gateway/TUI: hide internal memory-flush prompts in chat history#32068
liuxiaopai-ai wants to merge 2 commits into
openclaw:mainfrom
liuxiaopai-ai:codex/tui-hide-internal-memory-flush-32057

Conversation

@liuxiaopai-ai

@liuxiaopai-ai liuxiaopai-ai commented Mar 2, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Problem: internal pre-compaction memory-flush prompts can appear in interactive chat history (notably in openclaw tui).
  • Why it matters: internal orchestration text leaks into user-facing conversation surfaces and creates confusion/noise.
  • What changed:
    • mark memory-flush runs as inputProvenance.kind="internal_system" (sourceTool: "memory_flush")
    • filter chat.history to omit user messages with internal_system provenance
    • add regression tests for both provenance tagging and history filtering
  • What did NOT change (scope boundary): no change to model selection, compaction thresholds, or user-authored transcript entries.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

User-visible / Behavior Changes

  • chat.history no longer returns internal-system user prompts (including memory-flush internal prompts), so TUI/webchat history views do not surface that internal text.

Security Impact (required)

  • New permissions/capabilities? (No)
  • Secrets/tokens handling changed? (No)
  • New/changed network calls? (No)
  • Command/tool execution surface changed? (No)
  • Data access scope changed? (No)
  • If any Yes, explain risk + mitigation:

Repro + Verification

Environment

  • OS: macOS
  • Runtime/container: local Node/pnpm
  • Model/provider: N/A (history filtering + provenance tagging)
  • Integration/channel (if any): gateway chat history/TUI path
  • Relevant config (redacted): default

Steps

  1. Seed transcript with a normal user message, an internal-system provenance user message (memory-flush style), and a normal assistant reply.
  2. Call chat.history for the session.
  3. Inspect returned messages.

Expected

  • Internal-system message is hidden from returned history.
  • User + assistant visible messages remain.

Actual

  • Matches expected after fix.

Evidence

  • Failing test/log before + passing after
  • Trace/log snippets
  • Screenshot/recording
  • Perf numbers (if relevant)

Human Verification (required)

What you personally verified (not just CI), and how:

  • Verified scenarios:
    • pnpm exec vitest run src/auto-reply/reply/agent-runner.misc.runreplyagent.test.ts src/gateway/server.chat.gateway-server-chat.test.ts
    • Added assertion that memory-flush run passes inputProvenance: { kind: "internal_system", sourceTool: "memory_flush" }
    • Added gateway history regression test ensuring internal-system user prompt is omitted
  • Edge cases checked:
    • regular user message still visible
    • assistant final reply still visible
  • What you did not verify:
    • manual end-to-end TUI interaction against a live long-running compaction session

Compatibility / Migration

  • Backward compatible? (Yes)
  • Config/env changes? (No)
  • Migration needed? (No)
  • If yes, exact upgrade steps:

Failure Recovery (if this breaks)

  • How to disable/revert this change quickly:
    • revert this PR commit
  • Files/config to restore:
    • src/gateway/server-methods/chat.ts
    • src/auto-reply/reply/agent-runner-memory.ts
  • Known bad symptoms reviewers should watch for:
    • expected user messages missing from chat.history

Risks and Mitigations

  • Risk: over-filtering non-internal user content
    • Mitigation: filter is strictly provenance-based (role === "user" + provenance.kind === "internal_system"), not text-pattern-based.

@greptile-apps

greptile-apps Bot commented Mar 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR fixes internal memory-flush prompts leaking into user-facing chat history surfaces (e.g., openclaw tui) by tagging memory-flush agent runs with inputProvenance.kind = "internal_system" and filtering those tagged messages from chat.history responses.

Key changes:

  • agent-runner-memory.ts: Passes inputProvenance: { kind: "internal_system", sourceTool: "memory_flush" } to runEmbeddedPiAgent for memory-flush runs.
  • chat.ts: Introduces shouldHideChatHistoryMessage which filters out user-role messages carrying provenance.kind === "internal_system" before the limit/slice step. Filtering correctly operates on the already-unwrapped messages returned by readSessionMessages (which strips the { message: ... } JSONL envelope on ingestion), so the provenance field lookup is at the right level.
  • Tests: Covers both provenance tagging (unit) and history filtering (gateway integration), including assertions that normal user messages and assistant replies remain visible.

The implementation is strictly provenance-based and avoids fragile text-pattern matching. No behavioral changes outside the chat.history response path.

Confidence Score: 5/5

  • This PR is safe to merge — changes are narrowly scoped, well-tested, and logically correct.
  • The filtering logic is applied at the correct pipeline stage (after JSONL envelope unwrapping by readSessionMessages), the provenance field name (provenance) is consistent between applyInputProvenanceToUserMessage and shouldHideChatHistoryMessage, the limit is applied to already-filtered messages which is the correct behavior, and both unit and integration tests validate the key scenarios. No edge-case risks identified.
  • No files require special attention.

Last reviewed commit: 30c3611

@liuxiaopai-ai

liuxiaopai-ai commented Mar 2, 2026

Copy link
Copy Markdown
Contributor Author

Addressed in 66ccc06.

What changed:

  • Added gateway-side provenance sanitization in agent handler.
  • inputProvenance.kind="internal_system" is now reserved for admin-scoped callers (operator.admin).
  • Non-admin callers now have internal_system provenance stripped before agentCommand execution.
  • Non-internal provenance (inter_session, external_user) remains unchanged.

Validation:

  • pnpm exec vitest run src/gateway/server-methods/agent.test.ts
  • pnpm exec vitest run src/gateway/server.chat.gateway-server-chat.test.ts
  • pnpm lint src/gateway/server-methods/agent.ts src/gateway/server-methods/agent.test.ts
  • pnpm exec oxfmt --check src/gateway/server-methods/agent.ts src/gateway/server-methods/agent.test.ts

Also added regressions in agent.test.ts to cover:

  • write-scoped caller cannot forward internal_system
  • admin-scoped caller can forward internal_system
  • write-scoped caller still forwards inter_session

@prtags

prtags Bot commented Apr 23, 2026

Copy link
Copy Markdown

Related work from PRtags group powerful-tadpole-muy2

Title: Open PR candidate: internal compaction artifacts in chat.history

Number Title
#32068* Gateway/TUI: hide internal memory-flush prompts in chat history
#70348 Filter internal compaction artifacts from chat history

* This PR

@prtags prtags Bot mentioned this pull request Apr 23, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

app: web-ui App: web-ui gateway Gateway runtime size: S

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

TUI: Pre-compaction memory flush system messages visible to user

2 participants

@liuxiaopai-ai @thewilloftheshadow