fix(memory): persist session dirty state and fix reindex gate#20148
fix(memory): persist session dirty state and fix reindex gate#20148togotago wants to merge 1 commit into
Conversation
Additional Comments (1)
Prompt To Fix With AIThis is a comment left during a code review.
Path: src/memory/manager-sync-ops.ts
Line: 1121-1132
Comment:
**Missing `sessionsDirty` persistence in `runUnsafeReindex`**
`runSafeReindex` (lines 1062-1064) persists `sessionsDirty` into the meta, but `runUnsafeReindex` does not. While this path is gated behind test env vars, the inconsistency means tests using the unsafe reindex path won't exercise dirty-state persistence correctly, which could mask bugs in the new logic during test runs. Consider adding the same `sessionsDirty` persistence block here for consistency.
How can I resolve this? If you propose a fix, please make it concise. |
Fixes two related issues causing session indexing to stop after gateway restart: 1. Move needsFullReindex check before reason gate in shouldSyncSessions() - Previously, reason='session-start' or 'watch' would block reindex - Now needsFullReindex bypasses reason checks, allowing sessions in full reindex 2. Persist sessionsDirty flag to database metadata - Add sessionsDirty field to MemoryIndexMeta type - Save sessionsDirty during sync (runSafeReindex) - Restore sessionsDirty from meta on manager construction - Rebuild sessionsDirtyFiles Set by scanning sessions directory on startup - Compare against indexed files in DB to populate dirty set Without these fixes, session indexing would silently stop after any restart because the in-memory sessionsDirtyFiles Set was lost and sessions were excluded from reindex due to the reason gate. Resolves issue openclaw#1
eca6eca to
ff804bc
Compare
|
+1 — hit this exact bug tonight and spent 2 hours tracing through the source to find the root cause. Your fix is correct. Our scenarioAfter a gateway update + session cleanup, dozens of historical Root cause confirmationTraced through the minified source and independently arrived at the same conclusion: In if (reason === "session-start" || reason === "watch") return false; // ← always hits on boot
if (needsFullReindex) return true; // ← never reachedOn boot, Old/restored session files never trigger Workaround (for anyone blocked)
Additional note
Would love to see this merged — session memory is a great feature but effectively broken for any non-trivial setup. 🙏 |
|
This pull request has been automatically marked as stale due to inactivity. |
|
Closing due to inactivity. |
🤖 AI-assisted PR (Claude/OpenClaw agents). Fully tested, all changes reviewed and understood by human submitter.
Fixes two related issues causing session indexing to stop after gateway restart:
Summary
shouldSyncSessions(), addedsessionsDirtypersistence to DB meta, addedrebuildSessionsDirtyFiles()on startupChange Type (select all)
Scope (select all touched areas)
Linked Issue/PR
User-visible / Behavior Changes
Security Impact (required)
NoNoNoNoNoRepro + Verification
Environment
{ "memorySearch": { "sources": ["memory", "sessions"], "experimental": { "sessionMemory": true }, "sync": { "intervalMinutes": 5 } } }Steps
openclaw memory statusopenclaw gateway restartopenclaw memory statusover the next interval cycleExpected
Actual
Dirty: yesbut no indexing activityEvidence
Two new tests in
src/memory/index.test.ts:shouldSyncSessions({ reason: "session-start" }, true)and assertstrue. Before the fix this returnedfalse. Also verifies negative case.sessionsDirty: true, closes manager, reopens, asserts dirty state is restored.All 10 tests passing, linter clean.
Human Verification (required)
shouldSyncSessions()gate order —needsFullReindexnow checked beforereasonsessionsDirtyadded toMemoryIndexMetatyperunSafeReindexand restore in constructorrebuildSessionsDirtyFiles()queries existingfilestable with correct schemaCompatibility / Migration
YesNoNoFailure Recovery (if this breaks)
src/memory/manager-sync-ops.ts,src/memory/manager.tsrebuildSessionsDirtyFiles()fails, logs warning and continues — does not block startupRisks and Mitigations
rebuildSessionsDirtyFiles()scans all session files on startup, could be slow with thousands of filesvoid this.rebuildSessionsDirtyFiles()), does not block constructor or first syncGreptile Summary
This PR fixes two related bugs that caused session memory indexing to silently stop after gateway restart:
manager-sync-ops.ts): Moves theneedsFullReindexcheck before thereasongate inshouldSyncSessions(). Previously,reason === "session-start"or"watch"would short-circuit and returnfalseeven when a full reindex was needed, preventing session indexing from ever completing after a restart.manager.ts,manager-sync-ops.ts): AddssessionsDirtyto theMemoryIndexMetatype and persists it to DB metadata duringrunSafeReindex. On startup, the constructor restores this flag and callsrebuildSessionsDirtyFiles()to rediscover unindexed session files by comparing the file system against thefilestable.The changes are well-scoped and backward-compatible. One minor inconsistency:
runUnsafeReindex(test-only path) does not persistsessionsDirtyto meta, unlikerunSafeReindex.Confidence Score: 4/5
rebuildSessionsDirtyFilesis wrapped in try/catch and runs as fire-and-forget, so startup isn't blocked if it fails. Tests cover both changes. Minor deduction for therunUnsafeReindexinconsistency (test-only) and the fact thatsessionsDirtyis only persisted during full reindex, not during regular sync cycles (mitigated by the rebuild-on-startup approach).src/memory/manager-sync-ops.ts—runUnsafeReindexshould be updated for consistency with the newsessionsDirtypersistence inrunSafeReindex.Last reviewed commit: eca6eca
(3/5) Reply to the agent's comments like "Can you suggest a fix for this @greptileai?" or ask follow-up questions!
Context used:
dashboard- CLAUDE.md (source)