Skip to content

fix(memory): persist session dirty state and fix reindex gate#20148

Closed
togotago wants to merge 1 commit into
openclaw:mainfrom
togotago:fix/session-sync-after-restart
Closed

fix(memory): persist session dirty state and fix reindex gate#20148
togotago wants to merge 1 commit into
openclaw:mainfrom
togotago:fix/session-sync-after-restart

Conversation

@togotago

@togotago togotago commented Feb 18, 2026

Copy link
Copy Markdown

🤖 AI-assisted PR (Claude/OpenClaw agents). Fully tested, all changes reviewed and understood by human submitter.

Fixes two related issues causing session indexing to stop after gateway restart:

  1. Move needsFullReindex check before reason gate in shouldSyncSessions()
  2. Persist sessionsDirty flag to database metadata and rebuild dirty file set on startup

Summary

  • Problem: Session memory indexing silently stops after gateway restart — no errors, no warnings
  • Why it matters: Users lose session search permanently until manual DB intervention; no workaround survives restarts
  • What changed: Reordered logic gate in shouldSyncSessions(), added sessionsDirty persistence to DB meta, added rebuildSessionsDirtyFiles() on startup
  • What did NOT change: Indexing logic, chunking, embedding pipeline, config schema — purely a state management fix

Change Type (select all)

  • Bug fix
  • Feature
  • Refactor
  • Docs
  • Security hardening
  • Chore/infra

Scope (select all touched areas)

  • Gateway / orchestration
  • Skills / tool execution
  • Auth / tokens
  • Memory / storage
  • Integrations
  • API / contracts
  • UI / DX
  • CI/CD / infra

Linked Issue/PR

User-visible / Behavior Changes

  • Session memory indexing now resumes correctly after gateway restart (previously silently stopped)
  • No config changes, no new options, no migration needed

Security Impact (required)

  • New permissions/capabilities? No
  • Secrets/tokens handling changed? No
  • New/changed network calls? No
  • Command/tool execution surface changed? No
  • Data access scope changed? No

Repro + Verification

Environment

  • OS: Ubuntu 22.04
  • Runtime/container: Node.js v22
  • Model/provider: Local embeddings (nomic-embed-text-v1.5 GGUF)
  • Relevant config:
    {
      "memorySearch": {
        "sources": ["memory", "sessions"],
        "experimental": { "sessionMemory": true },
        "sync": { "intervalMinutes": 5 }
      }
    }

Steps

  1. Enable session memory with interval sync (config above)
  2. Let interval sync index some sessions — verify with openclaw memory status
  3. Restart gateway: openclaw gateway restart
  4. Check openclaw memory status over the next interval cycle

Expected

  • Session file count continues increasing after restart

Actual

  • Session count freezes, shows Dirty: yes but no indexing activity

Evidence

  • Failing test/log before + passing after

Two new tests in src/memory/index.test.ts:

  1. Reindex gate order — calls shouldSyncSessions({ reason: "session-start" }, true) and asserts true. Before the fix this returned false. Also verifies negative case.
  2. Dirty state persistence — writes meta with sessionsDirty: true, closes manager, reopens, asserts dirty state is restored.

All 10 tests passing, linter clean.

Human Verification (required)

  • Verified: shouldSyncSessions() gate order — needsFullReindex now checked before reason
  • Verified: sessionsDirty added to MemoryIndexMeta type
  • Verified: Persistence write in runSafeReindex and restore in constructor
  • Verified: rebuildSessionsDirtyFiles() queries existing files table with correct schema
  • Edge cases: Empty sessions dir (empty Set, no-op), missing meta field (optional, defaults falsy)
  • Not verified: Full end-to-end reindex cycle on live system (unit tests only)

Compatibility / Migration

  • Backward compatible? Yes
  • Config/env changes? No
  • Migration needed? No

Failure Recovery (if this breaks)

  • How to revert: Revert commit, restart gateway
  • Files to restore: src/memory/manager-sync-ops.ts, src/memory/manager.ts
  • Bad symptoms: If rebuildSessionsDirtyFiles() fails, logs warning and continues — does not block startup

Risks and Mitigations

  • Risk: rebuildSessionsDirtyFiles() scans all session files on startup, could be slow with thousands of files
    • Mitigation: Runs async (void this.rebuildSessionsDirtyFiles()), does not block constructor or first sync

Greptile Summary

This PR fixes two related bugs that caused session memory indexing to silently stop after gateway restart:

  • Reindex gate order fix (manager-sync-ops.ts): Moves the needsFullReindex check before the reason gate in shouldSyncSessions(). Previously, reason === "session-start" or "watch" would short-circuit and return false even when a full reindex was needed, preventing session indexing from ever completing after a restart.
  • Dirty state persistence (manager.ts, manager-sync-ops.ts): Adds sessionsDirty to the MemoryIndexMeta type and persists it to DB metadata during runSafeReindex. On startup, the constructor restores this flag and calls rebuildSessionsDirtyFiles() to rediscover unindexed session files by comparing the file system against the files table.
  • Two new tests cover both fixes with positive and negative assertions.

The changes are well-scoped and backward-compatible. One minor inconsistency: runUnsafeReindex (test-only path) does not persist sessionsDirty to meta, unlike runSafeReindex.

Confidence Score: 4/5

  • This PR is safe to merge — it fixes a clear logic ordering bug and adds state persistence with graceful fallbacks.
  • The gate reorder fix is straightforward and correct. The persistence mechanism is sound — rebuildSessionsDirtyFiles is wrapped in try/catch and runs as fire-and-forget, so startup isn't blocked if it fails. Tests cover both changes. Minor deduction for the runUnsafeReindex inconsistency (test-only) and the fact that sessionsDirty is only persisted during full reindex, not during regular sync cycles (mitigated by the rebuild-on-startup approach).
  • src/memory/manager-sync-ops.tsrunUnsafeReindex should be updated for consistency with the new sessionsDirty persistence in runSafeReindex.

Last reviewed commit: eca6eca

(3/5) Reply to the agent's comments like "Can you suggest a fix for this @greptileai?" or ask follow-up questions!

Context used:

  • Context from dashboard - CLAUDE.md (source)

@greptile-apps greptile-apps Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

@greptile-apps

greptile-apps Bot commented Feb 18, 2026

Copy link
Copy Markdown
Contributor
Additional Comments (1)

src/memory/manager-sync-ops.ts
Missing sessionsDirty persistence in runUnsafeReindex

runSafeReindex (lines 1062-1064) persists sessionsDirty into the meta, but runUnsafeReindex does not. While this path is gated behind test env vars, the inconsistency means tests using the unsafe reindex path won't exercise dirty-state persistence correctly, which could mask bugs in the new logic during test runs. Consider adding the same sessionsDirty persistence block here for consistency.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/memory/manager-sync-ops.ts
Line: 1121-1132

Comment:
**Missing `sessionsDirty` persistence in `runUnsafeReindex`**

`runSafeReindex` (lines 1062-1064) persists `sessionsDirty` into the meta, but `runUnsafeReindex` does not. While this path is gated behind test env vars, the inconsistency means tests using the unsafe reindex path won't exercise dirty-state persistence correctly, which could mask bugs in the new logic during test runs. Consider adding the same `sessionsDirty` persistence block here for consistency.

How can I resolve this? If you propose a fix, please make it concise.

Fixes two related issues causing session indexing to stop after gateway restart:

1. Move needsFullReindex check before reason gate in shouldSyncSessions()
   - Previously, reason='session-start' or 'watch' would block reindex
   - Now needsFullReindex bypasses reason checks, allowing sessions in full reindex

2. Persist sessionsDirty flag to database metadata
   - Add sessionsDirty field to MemoryIndexMeta type
   - Save sessionsDirty during sync (runSafeReindex)
   - Restore sessionsDirty from meta on manager construction
   - Rebuild sessionsDirtyFiles Set by scanning sessions directory on startup
   - Compare against indexed files in DB to populate dirty set

Without these fixes, session indexing would silently stop after any restart
because the in-memory sessionsDirtyFiles Set was lost and sessions were
excluded from reindex due to the reason gate.

Resolves issue openclaw#1
@togotago togotago force-pushed the fix/session-sync-after-restart branch from eca6eca to ff804bc Compare February 18, 2026 15:58
@jaybuidl

jaybuidl commented Feb 28, 2026

Copy link
Copy Markdown

+1 — hit this exact bug tonight and spent 2 hours tracing through the source to find the root cause. Your fix is correct.

Our scenario

After a gateway update + session cleanup, dozens of historical .jsonl session files were restored to disk but never indexed. openclaw memory status --deep showed sessions: 0/60 files, 0 chunks permanently.

Root cause confirmation

Traced through the minified source and independently arrived at the same conclusion:

In shouldSyncSessions(), the reason guard on L4 preempts the needsFullReindex check on L5:

if (reason === "session-start" || reason === "watch") return false;  // ← always hits on boot
if (needsFullReindex) return true;  // ← never reached

On boot, warmSession()sync({ reason: "session-start" })runSafeReindex()shouldSyncSessions("session-start", needsFullReindex=true) → returns false. Sessions skipped. Meta written. Window closed permanently.

Old/restored session files never trigger sessionsDirtyFiles (only active writes do), so they're permanently invisible to the indexer.

Workaround (for anyone blocked)

  1. Set memorySearch.sync.onSessionStart: false
  2. Ensure a meta mismatch (e.g. temporarily change chunking.tokens)
  3. Restart gateway
  4. Trigger memory_search from an agent session (reason "search" passes the gate)
  5. Revert config after reindex completes

Additional note

openclaw memory index --force from CLI doesn't help — it runs in "FTS-only mode (no embedding provider)" because the CLI process doesn't resolve the API key the same way the gateway does. So even --force can't work around this.

Would love to see this merged — session memory is a great feature but effectively broken for any non-trivial setup. 🙏

@openclaw-barnacle

Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Mar 18, 2026
@openclaw-barnacle

Copy link
Copy Markdown

Closing due to inactivity.
If you believe this PR should be revived, post in #pr-thunderdome-dangerzone on Discord to talk to a maintainer.
That channel is the escape hatch for high-quality PRs that get auto-closed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size: S stale Marked as stale due to inactivity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants