Skip to content

fix(memory-core): treat dreaming fence marker lines as inside-fence in promotion guard (#80613)#83718

Open
grifjef wants to merge 1 commit into
openclaw:mainfrom
grifjef:fix/dreaming-fence-marker-lines
Open

fix(memory-core): treat dreaming fence marker lines as inside-fence in promotion guard (#80613)#83718
grifjef wants to merge 1 commit into
openclaw:mainfrom
grifjef:fix/dreaming-fence-marker-lines

Conversation

@grifjef

@grifjef grifjef commented May 18, 2026

Copy link
Copy Markdown

Summary

  • Problem. lineRangeOverlapsDreamingFence in extensions/memory-core/src/short-term-promotion.ts tracked insideFence state from the fence marker lines but never flagged ranges that included the marker lines themselves. A relocated promotion range that ended on a <!-- openclaw:dreaming:*:start --> line, began on a <!-- openclaw:dreaming:*:end --> line, or covered only a marker line passed the guard, and the snippet built from those raw lines carried the marker text into MEMORY.md.
  • Why it matters. The guard's contract is to refuse rehydrated ranges that overlap managed dreaming fences so dream artifacts cannot be promoted. Treating the marker lines themselves as outside-the-fence leaves a narrow but real leak: any window the relocator picks that clips a marker produces a snippet that includes the literal <!-- openclaw:dreaming:*:start/end --> text, and (when the window straddles content on either side of the marker) the adjacent line. This is the residual edge case noted by ClawSweeper on fix(memory-core): redact managed dreaming blocks during promotion rehydration (#80613) #80702 ("the managed-block leak half is related but should be handled by a focused memory-core promotion fix"), narrowed to the guard itself rather than a parallel sanitizer.
  • What changed. extensions/memory-core/src/short-term-promotion.ts: marker lines are now checked against the range bounds; if a marker is inside the relocated range, the range is rejected. The pre-existing inside-fence tracking remains so interior lines between a start/end pair are still flagged. The contract elsewhere — the contamination predicate, the relocator, the apply path, the MEMORY.md writer — is untouched.
  • What did not change (scope boundary). No change to relocateCandidateRange, isContaminatedDreamingSnippet, applyShortTermPromotions, buildDailySnippetChunks, stripManagedDailyDreamingLines, the CJK dedupe path (the second sub-bug of [Bug]: dreaming pipeline leaks raw candidate content into MEMORY.md and CJK dedup is ineffective in tokenizeSnippet #80613 is being addressed separately by fix(memory-core): use CJK-aware tokenizer for dreaming dedupe (#80613) #80620), or any MEMORY.md rendering. No new sanitizer is introduced — this is a narrowing of the existing guard.

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor required for the fix
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

Real behavior proof (required for external PRs)

  • Behavior addressed: lineRangeOverlapsDreamingFence now flags relocated promotion ranges that include a dreaming fence marker line, preventing the marker text (and any adjacent fenced content captured by the same raw-line window) from being promoted into MEMORY.md.
  • Real environment tested: Local OpenClaw source checkout on macOS (Apple Silicon, Node v22.14.0, pnpm 11.1.0). The fix is mechanical and exercised by Vitest unit tests against the helper directly.
  • Exact steps or command run after this patch:
    • pnpm test extensions/memory-core/src/short-term-promotion.test.ts -t "lineRangeOverlapsDreamingFence" — targeted run of the affected describe block.
    • pnpm test extensions/memory-core/src/short-term-promotion.test.ts — full file regression.
    • pnpm test extensions/memory-core/src/dreaming-phases.test.ts — adjacent file regression.
    • pnpm format:check extensions/memory-core/src/short-term-promotion.ts extensions/memory-core/src/short-term-promotion.test.ts — format on changed files.
    • pnpm lint extensions/memory-core/src/short-term-promotion.ts extensions/memory-core/src/short-term-promotion.test.ts — lint on changed files.
    • pnpm tsgo:extensions and pnpm tsgo:extensions:test — typecheck.
  • Evidence after fix:
    • Before the source change, the four added tests reproduce the gap (output captured locally):
      Tests  4 failed | 4 passed | 50 skipped (58)
      × returns true when the range ends on a Light Sleep start marker
      × returns true when the range begins on a Light Sleep end marker
      × returns true when the range covers only a marker line
      × returns true for REM marker single-line ranges even with no body between markers
      
      Each failure form: expected false to be true, confirming that the pre-patch guard returns false for a range that includes only a marker line.
    • After the source change:
      Tests  8 passed | 50 skipped (58)            # lineRangeOverlapsDreamingFence -t filter
      Tests  58 passed (58)                         # full short-term-promotion.test.ts
      Tests  41 passed (41)                         # full dreaming-phases.test.ts
      
    • Format/lint/typecheck on changed files: all clean. (Repo-wide format reports 34 pre-existing issues on main unrelated to this PR.)
  • Observed result after fix: the helper rejects any rehydrated range that touches a <!-- openclaw:dreaming:*:start --> or <!-- openclaw:dreaming:*:end --> marker line. No regressions in the existing lineRangeOverlapsDreamingFence describe block or the broader test files.
  • What was not tested: An end-to-end repro via applyShortTermPromotions that drives relocateCandidateRange to organically pick a marker-line-bordering window. Probe attempts on current main could not construct an organic trigger — the relocator's compare/quality scoring rarely prefers such windows when cleaner alternatives exist in the file — so the demonstration is at the unit level. The repo-wide format/lint/build/CI suite was not run locally; the changed files pass all relevant lanes individually.

@openclaw-barnacle openclaw-barnacle Bot added extensions: memory-core Extension: memory-core size: S proof: supplied External PR includes structured after-fix real behavior proof. labels May 18, 2026
@clawsweeper

clawsweeper Bot commented May 18, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs maintainer review before merge.

Workflow note: Future ClawSweeper reviews update this same comment in place.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Summary
The PR changes lineRangeOverlapsDreamingFence so openclaw:dreaming start/end marker lines count as fence overlap, and adds unit plus applyShortTermPromotions regression coverage.

Reproducibility: yes. The current main source path skips marker lines before range coverage checks, and the PR adds a direct helper fixture plus an applyShortTermPromotions fixture that would exercise the leak path.

PR rating
Overall: 🐚 platinum hermit
Proof: 🐚 platinum hermit
Patch quality: 🐚 platinum hermit
Summary: Good normal PR readiness: the fix is narrow, the prior test alias blocker is resolved, and the proof now exercises the real promotion apply path, with only CI completion remaining.

Rank-up moves:

  • none
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

PR egg
✨ Hatched: 🥚 common Brave Clawlet

       _..------.._          
    .-'  .-.  .-.  '-.       
   /    ( * )( * )    \      
  |        .--.        |     
  |   <\   ====   />   |     
   \    '.______.'    /      
    '-._   ____   _.-'       
        `-.____.-'           
       __/|_||_|\__          
      /__.'    '.__\         
       .-----------.         
      '-------------'        

Rarity: 🥚 common.
Trait: keeps receipts.
Share on X: post this hatch
Copy: My PR egg hatched a 🥚 common Brave Clawlet in ClawSweeper.

What is this egg doing here?
  • Eggs appear after the PR passes real-behavior proof. It is here for vibes, not verdicts: it does not change labels, ratings, merge decisions, or automation.
  • The shell reacts to review momentum: open follow-up work warms it up, re-review makes it wobble, and a clean final review lets it hatch.
  • How to hatch it: reach status: 👀 ready for maintainer look or status: 🚀 automerge armed; that usually means sufficient real-behavior proof, no blocking P0/P1/P2 findings, no security attention needed, and clean correctness.
  • The hatch is seeded from this repository and PR number, so the same PR keeps the same creature; the reviewed head SHA can only change safe visual details.
  • Rarity is just collectible sparkle: 🥚 common, 🌱 uncommon, 💎 rare, ✨ glimmer, and 🌈 legendary.

Real behavior proof
Sufficient (terminal): The author supplied after-fix local terminal proof and added an on-disk applyShortTermPromotions regression showing the marker is no longer written to MEMORY.md; private data should remain redacted in any future logs.

Risk before merge
Why this matters: - Some broad CI checks were still in progress when inspected, so merge should wait for the required check set to complete.

Maintainer options:

  1. Decide the mitigation before merge
    Land the narrow guard change after required CI finishes, keeping the remaining CJK dedupe work on the separate linked PR path.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge
No repair lane is needed; the actionable blocker from the first review is resolved and the remaining action is ordinary maintainer review plus CI completion.

Security
Cleared: The diff only changes memory-core TypeScript logic and colocated tests; it does not touch secrets, dependencies, CI, package resolution, or code-execution surfaces.

Review details

Best possible solution:

Land the narrow guard change after required CI finishes, keeping the remaining CJK dedupe work on the separate linked PR path.

Do we have a high-confidence way to reproduce the issue?

Yes. The current main source path skips marker lines before range coverage checks, and the PR adds a direct helper fixture plus an applyShortTermPromotions fixture that would exercise the leak path.

Is this the best way to solve the issue?

Yes. Treating marker lines as managed fence content in the existing overlap guard is the narrow maintainable fix, and it avoids adding a parallel sanitizer or changing promotion policy.

Label justifications:

  • P2: This is a focused memory-core bug fix for preventing managed dreaming marker text from being promoted into durable memory, with limited blast radius.

What I checked:

Likely related people:

  • solomonneas: Merged predecessor work added the dreaming-fence overlap guard and tests that this PR narrows for marker-line ranges. (role: introduced predecessor behavior; confidence: high; commits: 314903417fcc; files: extensions/memory-core/src/short-term-promotion.ts, extensions/memory-core/src/short-term-promotion.test.ts)
  • steipete: Recent lint cleanup changed the test helper usage to the current testing alias and touched the same memory-core test file. (role: recent adjacent contributor; confidence: medium; commits: 4f4d10863916; files: extensions/memory-core/src/short-term-promotion.test.ts)

Codex review notes: model gpt-5.5, reasoning high; reviewed against f07c87405c30.

@clawsweeper clawsweeper Bot added rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels May 18, 2026
…n promotion guard

The lineRangeOverlapsDreamingFence guard tracked insideFence state from
the marker lines but did not flag ranges that included the marker lines
themselves. A relocated promotion range that ended on a start marker,
began on an end marker, or covered only a marker line passed the guard,
and the snippet built from those raw lines carried the marker text into
MEMORY.md.

Treat marker lines as inside-fence content for range overlap purposes
and add unit + integration coverage. The integration test exercises the
real applyShortTermPromotions path: pre-patch a recall whose stored
snippet is the start-marker text produces a 'Promoted From Short-Term
Memory' entry containing the literal marker; post-patch the same input
yields applied=0 and no MEMORY.md write. Refs openclaw#80613.
@grifjef grifjef force-pushed the fix/dreaming-fence-marker-lines branch from f9fcc43 to cf8981c Compare May 19, 2026 13:09
@grifjef

grifjef commented May 19, 2026

Copy link
Copy Markdown
Author

Thanks @clawsweeper. Addressed both findings:

  1. Type blocker. Rebased onto upstream/main and updated the new test cases to use the testing alias (replacing the old __testing references from the original branch, before the rename in 4f4d108639 merged). node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.extensions.test.json is now clean.
  2. Real promotion-path proof. Added an integration test does not promote rehydrated candidates whose relocated range covers a managed dreaming fence marker line (#80613) that drives the real applyShortTermPromotions path. Pre-patch the same input writes <!-- openclaw:dreaming:light:start --> into MEMORY.md as a Promoted From Short-Term Memory entry (applied=1); post-patch the range is rejected (applied=0, no MEMORY.md written). Diagnostic before/after MEMORY.md contents are inlined in the updated PR body.

Full local test run on rebased cf8981c8: Tests 59 passed (59) for short-term-promotion.test.ts and 41 passed (41) for dreaming-phases.test.ts. Format and tsgo:extensions:test clean on changed files.

@clawsweeper re-review please.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. and removed rating: 🦪 silver shellfish Thin PR readiness signal; proof, validation, or implementation needs work. status: 📣 needs proof The PR needs real behavior proof before ClawSweeper can clear the contributor ask. labels May 19, 2026
@RomneyDa

Copy link
Copy Markdown
Member

Heads up: this PR needs to be updated against current main before the new required Dependency Guard check can pass.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: memory-core Extension: memory-core P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: S status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants