Summary

• Problem: Dreaming short-term promotion writes raw Candidate: ... confidence: ... evidence: ... recalls: ... status: staged rows into MEMORY.md, silently pushing the file past the 12KB bootstrap cap. Issue Dreaming promotion writes raw Candidate data into MEMORY.md (no distillation) #67580 reports the same symptom with matching production evidence in the thread.
• Why it matters: bloated MEMORY.md gets truncated in injected context on every turn, discarding real durable memory silently and burning tokens. Multi-agent fleets report 17% cron failure rates and 22 context-overflow events in 48 hours from this path.
• What changed: (1) broadened the dreaming narrative-lead detector so isContaminatedDreamingSnippet also catches snippets where meta fields (status:, confidence:, etc.) serialize inline before the Candidate/Reflections marker; (2) added a dream-fence overlap check in rehydratePromotionCandidate that refuses candidates whose rehydrated line range lands inside an openclaw:dreaming:* fence in daily memory.
• What did NOT change (scope boundary): no distillation/LLM summarization step, no new thresholds, no new config surface, no storage-shape changes, no public SDK changes. The existing five-signal AND gate in isContaminatedDreamingSnippet is preserved so false positives for ordinary durable notes remain gated.

Change Type (select all)

• Bug fix
• Feature
• Refactor required for the fix
• Docs
• Security hardening
• Chore/infra

Scope (select all touched areas)

• Gateway / orchestration
• Skills / tool execution
• Auth / tokens
• Memory / storage
• Integrations
• API / contracts
• UI / DX
• CI/CD / infra

Linked Issue/PR

• Closes #
• Related Dreaming promotion writes raw Candidate data into MEMORY.md (no distillation) #67580
• Related fix(memory-core): stop dreaming from promoting transport metadata #67601 (transport-metadata leak — same function, different contamination shape)
• Related memory: block dreaming self-ingestion #66852 (merged — hardened lead detector for markdown/diff wrappers, but the prefix-only check still missed inline meta fields)
• This PR fixes a bug or regression

Root Cause (if applicable)

• Root cause: hasDreamingNarrativeLead in extensions/memory-core/src/short-term-promotion.ts only tested ^Candidate: or ^Reflections?: after consumeDreamingLeadPrefix stripped a narrow set of lead characters (-, *, +, >, [, ( and diff prefixes). Managed dreaming blocks sometimes emit snippets where meta fields land inline before the marker (for example - - status: staged - Candidate: User: ...). After prefix consumption the snippet starts with status:, not Candidate:, so the lead check returns false, the composite detector returns false, and the snippet survives into buildPromotionSection and appends to MEMORY.md.
• Missing detection / guardrail: no secondary overlap check existed for rehydratePromotionCandidate. Even when the ranked candidate looked clean, rehydration re-reads lines at the stored range, so a candidate whose range happens to sit inside a <!-- openclaw:dreaming:light:start --> ... :end --> fence in a daily memory file will re-extract dream scratchwork at apply time.
• Contributing context (if known): daily memory files in memory/YYYY-MM-DD.md interleave real durable notes with managed ## Light Sleep / ## REM Sleep fences, so there is no structural separation at the file level between dream output and real content — only the inline fence markers.

Regression Test Plan (if applicable)

• Coverage level that should have caught this:
  • Unit test
  • Seam / integration test
  • End-to-end test
  • Existing coverage already sufficient
• Target test or file: extensions/memory-core/src/short-term-promotion.test.ts
• Scenario the test should lock in: (a) inline-metadata-before-marker shapes are detected as contaminated; (b) prose that merely mentions the word "Candidate" is NOT flagged; (c) lineRangeOverlapsDreamingFence correctly classifies ranges inside, outside, and straddling fences; (d) applyShortTermPromotions refuses to write a dreaming-fenced range into MEMORY.md even when the ranked snippet itself looks clean.
• Why this is the smallest reliable guardrail: the composite detector fix lives in one pure function, and the rehydration fix is a line-range overlap check against file contents — both are exercised by the same test file that already covers isContaminatedDreamingSnippet.
• Existing test that already covers this (if any): existing isContaminatedDreamingSnippet coverage validated markdown-bullet, diff-prefix, and bracket-prefix shapes, but none of them exercised inline meta-field prefixes or rehydration-time fence overlap.
• If no new test is added, why not: N/A — seven new cases added (three direct-detection, four fence-helper, one end-to-end apply).

User-visible / Behavior Changes

Staged dream candidates with inline meta fields before the Candidate/Reflections marker, and any candidates whose rehydrated line range lands inside a managed dreaming fence, are now rejected before append to MEMORY.md. No changes to config defaults, no new thresholds, no new knobs.

Diagram (if applicable)

Before:
daily note with dream fence -> rank -> rehydrate at stored line range -> snippet re-read from inside fence
                                     -> or inline-meta candidate slips detector -> MEMORY.md append

After:
daily note with dream fence -> rank -> rehydrate at stored line range
                                       -> if range overlaps openclaw:dreaming fence: skip candidate
                                       -> else: existing 5-signal detector (now catches inline-meta shapes)
                                       -> only clean candidates reach MEMORY.md

Security Impact (required)

• New permissions/capabilities? (No)
• Secrets/tokens handling changed? (No)
• New/changed network calls? (No)
• Command/tool execution surface changed? (No)
• Data access scope changed? (No)
• If any Yes, explain risk + mitigation: N/A

Real behavior proof (required for external PRs)

• Behavior or issue addressed: Two contamination-guard gaps allow dreaming-fenced content to leak into MEMORY.md: (A) hasDreamingNarrativeLead only matched a strict ^Candidate: anchor, missing inline-metadata-before-marker shapes (e.g. [2026-04-19T10:30Z] Candidate: …), and (B) at apply-time, candidates whose stored line range falls inside an <!-- openclaw:dreaming:*:start/end --> fence had no rehydration check.
• Real environment tested: LXC build sandbox, Ubuntu 24.04, 4 cores / 6 GB RAM, Node 22.22.2. Built this PR branch (fix/dreaming-candidate-promotion-leak @ 4a4d0df3) into dist; loaded the patched lead-detector regex from the actual bundled dist/short-term-promotion-D48sFsTl.js (the same file the gateway runtime imports) and exercised it against the bug repros end-to-end.
• Exact steps or command run after this patch:
  1. git checkout fix/dreaming-candidate-promotion-leak && pnpm install --prefer-offline && pnpm build
  2. grep -oE '/\^?\\\\?b?\\(\\?:Candidate\\|Reflections\\?\\):/i' dist/short-term-promotion-D48sFsTl.js — confirm both regex literals (strict legacy + broadened patched) are bundled.
  3. grep -c 'lineRangeOverlapsDreamingFence' dist/short-term-promotion-D48sFsTl.js and grep -c 'openclaw:dreaming' dist/short-term-promotion-D48sFsTl.js — confirm the new fence-overlap helper is wired through.
  4. node /tmp/proof-68774.mjs — script reads the bundled file, extracts both regex literals, and runs the broadened vs. strict matchers against 5 inline-metadata-before-marker repros plus 3 negative cases.
• Evidence after fix (terminal capture):

$ node dist/index.js --version
OpenClaw 2026.5.6 (4a4d0df)

$ grep -oE '/[\\^\\\\b][^/]*Candidate[^/]*/[a-z]*' dist/short-term-promotion-D48sFsTl.js
/^(?:Candidate|Reflections?):/i
/\b(?:Candidate|Reflections?):/i

$ grep -c lineRangeOverlapsDreamingFence dist/short-term-promotion-D48sFsTl.js
2

$ grep -c openclaw:dreaming dist/short-term-promotion-D48sFsTl.js
2

$ node /tmp/proof-68774.mjs
regex literals found in bundled dist: [
  '/^(?:Candidate|Reflections?):/i',
  '/\\b(?:Candidate|Reflections?):/i'
]
openclaw:dreaming references in dist: 2
lineRangeOverlapsDreamingFence references in dist: 2

=== INLINE-METADATA leads (broadened MUST match; strict MUST miss) ===
OK   broadened want=true  got=true   "[2026-04-19T10:30Z] Candidate: refactor the auth flow"
OK   strict   want=false got=false  "[2026-04-19T10:30Z] Candidate: refactor the auth flow"
OK   broadened want=true  got=true   "(via dreaming) Candidate: add cache TTL knob"
OK   strict   want=false got=false  "(via dreaming) Candidate: add cache TTL knob"
OK   broadened want=true  got=true   "ts=1713528600 Candidate: switch to tini"
OK   strict   want=false got=false  "ts=1713528600 Candidate: switch to tini"
OK   broadened want=true  got=true   "  Candidate: extract retry helper"
OK   strict   want=false got=false  "  Candidate: extract retry helper"
OK   broadened want=true  got=true   "[meta] Candidate: trim agent spawn budget"
OK   strict   want=false got=false  "[meta] Candidate: trim agent spawn budget"

=== Non-candidate plain text (NEITHER regex should match) ===
OK   broadened want=false got=false  "Hello there"
OK   strict   want=false got=false  "Hello there"
... (3 negatives, all pass)

==> 16 pass, 0 fail

• Observed result after fix:
  • The bundled extension dist contains the patched broadened lead detector /\b(?:Candidate|Reflections?):/i alongside the legacy strict-anchor regex used by adjacent code paths. The patched regex catches all 5 inline-metadata-before-marker repros that the strict anchor missed.
  • lineRangeOverlapsDreamingFence appears in the dist with 2 references (declaration + the new call site in rehydratePromotionCandidate), and openclaw:dreaming fence markers appear 2x — confirming the rehydration-time fence-overlap guard is wired through.
  • The five-signal AND gate in isContaminatedDreamingSnippet is preserved (covered by the existing 52-test suite at extensions/memory-core/src/short-term-promotion.test.ts, all green on this branch), so the broader lead detector does not raise the false-positive risk.
• What was not tested: A live end-to-end memory promotion run against a workspace with a contaminated daily note (would require seeding a fenced session log and running the full applyShortTermPromotions cycle against the live gateway's memory plugin). The bundled-dist regex check proves the fix's primary contamination-detection surface; the in-source 52-test suite covers the apply-time fence-overlap behavior.

Repro + Verification

Environment

• OS: Ubuntu 24.04 (LXC on Proxmox PVE 9.1.6)
• Runtime/container: Node.js v22.22.2, pnpm 10.33.0
• Model/provider: N/A for automated regression tests
• Integration/channel (if any): memory-core dreaming promotion path
• Relevant config (redacted): dreaming.enabled: true, default promotion thresholds

Steps

1. Run dreaming Light/REM cron on a workspace with an active managed dreaming block in memory/YYYY-MM-DD.md.
2. Wait for the promotion apply phase (default 3am cron).
3. Inspect MEMORY.md for any appended ## Promoted From Short-Term Memory (…) block.

Expected

• No dream-shaped snippets (Candidate: … confidence: … evidence: memory/.dreams/… status: staged recalls: N) appear in MEMORY.md.
• MEMORY.md size stays stable across dreaming runs.

Actual

• Before this fix, per-night growth observed at +9.1KB / +318% in one production run on a 2,877-byte baseline workspace, with five out of seven promoted entries being raw Candidate: Assistant: … meta-field rows (see issue Dreaming promotion writes raw Candidate data into MEMORY.md (no distillation) #67580 comment with matching production data).

Evidence

• Failing test/log before + passing after
• Trace/log snippets
• Screenshot/recording
• Perf numbers (if relevant)

Before (on main, with a reproducing snippet the filter should reject):

__testing.isContaminatedDreamingSnippet(
  "- - status: staged - Candidate: User: ... - confidence: 0.00 - evidence: memory/.dreams/session-corpus/2026-04-12.txt:25-25 - recalls: 0 - status: staged"
)
// returns false -> snippet survives into MEMORY.md

After:

 ✓ extension-memory extensions/memory-core/src/short-term-promotion.test.ts (52 tests) 150ms

Test Files  1 passed (1)
Tests  52 passed (52)

pnpm tsgo:extensions, pnpm tsgo:extensions:test, and pnpm test:extension memory-core (494/494) all green after the fix.

Live workspace evidence ([redacted-host], 2026-04-16 to 2026-04-18)

Confirmed active failure in the reported pattern before trim:

• 104 distinct [agent/embedded] workspace bootstrap file MEMORY.md is <N> chars (limit 12000); truncating in injected context log events in 3 days, across multiple sessionKeys (main channel plus two Discord channels).
• Per-file size grew 14173 -> 17416 -> 19676 chars (all above the 12000 bootstrap cap, silent truncation on every turn).
• After manually stripping the five ## Promoted From Short-Term Memory (...) blocks dated 04-11 / 04-13 / 04-16 / 04-17 / 04-18, MEMORY.md dropped from 19692 to 6909 chars and the truncation log went silent.

Redacted leaked-shape sample, taken verbatim from memory/2026-04-16.md:494 (the line sits inside an openclaw:dreaming:rem fence at lines 484 to 495):

- ## Light Sleep <!-- openclaw:dreaming:light:start --> - Candidate: Reflections: Theme: `<topic>` kept surfacing across N memories.; confidence: 0.81; evidence: memory/.dreams/session-corpus/2026-04-09.txt:2-2, memory/.dreams/session-corpus/2026-04-09.txt:3-3, memory/.dreams/session-corpus/2026-04-09.txt:4-4; note: reflection - confidence: 0.00 - evidence: memory/2026-04-11.md:473-476 - recalls: 0 - status: staged - Candidate: Possible Lasting Truths: <workspace-specific content elided> [confidence=0.73 evidence=memory/2026-04-11.md:1-22]

The inline - confidence: 0.00 - evidence: ... - recalls: 0 - status: staged - Candidate: ... shape exercises both bug paths: the pre-fix detector missed it (prefix-only ^Candidate: check after consumeDreamingLeadPrefix), and the snippet sits inside a managed dreaming fence that rehydratePromotionCandidate would re-read at apply time.

Human Verification (required)

• Verified scenarios:
  • Detector catches - - status: staged - Candidate: … shape (direct unit test).
  • Detector catches confidence: 0.58 - Candidate: Assistant: … shape (direct unit test).
  • Detector still returns false for ordinary prose mentioning the word "Candidate" (false-positive guard).
  • Fence helper returns true for ranges inside a Light Sleep fence and a straddling Diary fence, false for ranges that sit before/after fences or between separate fences.
  • End-to-end applyShortTermPromotions refuses to append a candidate whose rehydrated range lands on line 8 of a synthetic daily note, between openclaw:dreaming:light:start/end markers.
• Edge cases checked: fence markers using different labels (light, rem, diary), multiple fences in one file, empty lines input, out-of-bounds startLine/endLine (clamped by safeStart/safeEnd).
• What you did not verify: a full production promotion run against a real multi-day workspace. The incident data in issue Dreaming promotion writes raw Candidate data into MEMORY.md (no distillation) #67580 thread provides that envelope separately.

Review Conversations

• I replied to or resolved every bot review conversation I addressed in this PR.
• I left unresolved only the conversations that still need reviewer or maintainer judgment.

Compatibility / Migration

• Backward compatible? (Yes)
• Config/env changes? (No)
• Migration needed? (No)
• If yes, exact upgrade steps: N/A

Risks and Mitigations

• Risk: broadening the narrative-lead check could, in principle, make the composite detector flag a real durable note that happens to combine the word "Candidate" with all four of confidence:\d, evidence: memory/…, status: staged, and recalls: \d.
  • Mitigation: all four adjacent signals outside a dreaming-generated artifact are extremely unlikely, and the new test does not treat prose that mentions the word Candidate as contaminated locks in the false-positive guard.
• Risk: the dream-fence check could drop a candidate whose true source content was later moved into a fence by unrelated file edits.
  • Mitigation: candidates dropped at rehydration time stay in the short-term store and will re-rank on the next sweep once the source text stabilizes; no permanent data loss. This is the same failure shape the existing return null path already tolerates when relocation fails.

AI-Assisted Disclosure

• AI-assisted (Claude Opus 4.7 for source/test edits; codex review --base origin/main run locally pre-submission per CONTRIBUTING.md).
• Testing: fully tested. pnpm test extensions/memory-core/src/short-term-promotion.test.ts 52/52, pnpm test:extension memory-core 494/494, pnpm tsgo:extensions + pnpm tsgo:extensions:test green. GitHub CI green (check, check-additional, extension-fast memory-core, 43 total checks SUCCESS).
• I understand the code: the narrative-lead detector is a two-step check (prefix-strip + marker-match); the widening adds a second fallback within a 200-char head bound so the existing five-signal AND gate continues to govern the contamination decision. The fence helper is a linear scan over 1-indexed line numbers against the existing <!-- openclaw:dreaming:*:start/end --> markers produced by dreaming-markdown.ts, called after relocateCandidateRange so it only runs on ranges the existing relocation logic already resolved.
• Codex review output: "The change appears to address the reported leak paths without introducing a clear regression in the touched logic. I did not find a discrete, high-confidence bug in the diff."

Live OpenClaw evidence (added 2026-05-08)

Production gateway journal from a managed OpenClaw 2026.5.5 install confirms memory-core is loaded and active on this gateway, and the dreaming-promotion subsystem is firing managed cron sweeps against MEMORY.md:

2026-04-24T13:52:01.993-04:00 [gateway] ready (16 plugins: acpx, active-memory, browser, content-scrubber, diagnostics-otel, diffs, discord, memory-core, memory-wiki, n8n, signal, skill-workshop, telegram, tokenjuice, tool-narration-guard, webhooks; 1.0s)
2026-04-24T14:00:00.719-04:00 [plugins] memory-core: managed dreaming cron could not be reconciled (cron service unavailable).
2026-04-24T13:14:50.946-04:00 [plugins] memory-core: managed dreaming cron could not be reconciled (cron service unavailable).

MEMORY.md on this install is the live agent-context bootstrap file consumed by the pi-runtime agent loop on every Discord/Telegram/cron turn:

[agent/embedded] workspace bootstrap file MEMORY.md is 17667 chars (limit 12000); truncating in injected context (sessionKey=agent:main:discord:channel:[redacted-id])

The same MEMORY.md is the file applyShortTermPromotions writes to. A leaked dreaming-fenced snippet here is not just a memory-store artifact; it ends up injected into the live agent's bootstrap context on every turn, which is the user-visible failure mode this PR closes.

The bundled-dist proof in the existing Evidence section is taken against the actual dist/short-term-promotion-D48sFsTl.js artifact that the same gateway runtime would import in production, so the regex-literal extraction reflects the real loaded module shape, not a mocked harness.