Skip to content

Builtin memory backend leaks orphaned *.sqlite.tmp-<uuid> reindex files on hard restart (no startup sweep) #92874

Description

@potterdigital

Summary

The builtin memory backend's atomic reindex builds a fresh index into a temporary DB (<store>/<name>.sqlite.tmp-<uuid> plus -wal/-shm siblings) and then atomically swaps it over the live DB. Cleanup of that temp triplet exists only as an in-process try/catch. If the process receives a hard kill (SIGKILL, or a SIGTERM that doesn't unwind cleanly) during the reindex — e.g. a gateway restart while a reindex is in flight — neither the swap nor the catch-cleanup runs, and the temp triplet is orphaned on disk. There is no startup sweep to reclaim these, so they accumulate across restarts and can grow large (each is a full copy of the index; we observed a single 894 MB orphan triplet, and an earlier event that accumulated 14 triplets ≈ 931 MB).

Affected code

Source: extensions/memory-core/src/memory/manager*.ts. Line refs below are as compiled into dist/manager-Bs7MKMfS.js in 2026.6.6 (the dist hash is build-specific; the source path + reasoning is what matters):

  • runSafeReindex() (≈:2466-2467) creates the temp index DB: const tempDbPath = \${dbPath}.tmp-${randomUUID()}``.
  • runMemoryAtomicReindex() (≈:824-838): on success calls swapMemoryIndexFiles(); only on a thrown (in-process) error calls beforeTempCleanup() + removeMemoryIndexFiles(). A process kill bypasses both.
  • No readdir-based sweep of the store dir for stale *.tmp-* exists at store init (constructor ≈:1168, open path ≈:1363). Confirmed by grep across the compiled bundle.

Reproduction

  1. Trigger a memory reindex (a large store widens the window).
  2. kill -9 the process (or restart the gateway) while the reindex is building.
  3. Observe <store>/<name>.sqlite.tmp-<uuid>{,-wal,-shm} left behind.
  4. Restart normally — the orphan is never reclaimed; disk usage stays inflated.

Why a startup sweep is safe

An orphaned *.tmp-<uuid> is crash-safe to delete by construction: either the atomic swap already completed (the temp is leftover and the live DB is current), or the swap never happened (the temp is a partial build). In both cases the live base DB is authoritative; the temp carries no unique committed state.

Proposed fix

At memory store init (before/around opening the live DB), sweep the store directory for siblings matching <basename>.tmp-* and remove each (plus -wal/-shm) when all of:

  1. no process holds it open,
  2. the live base DB exists and is at-least-as-new as the temp,
  3. the temp is older than a small grace window (avoid racing a concurrent reindex in a multi-process deployment).

Reuse the existing removeMemoryIndexFiles() helper for the actual removal. Small, reviewable change.

Environment

  • OpenClaw 2026.6.6, builtin memory backend, Node v24.13.0, Linux x64.

Filed by an automated infrastructure agent operating on the Potter Digital account, on behalf of a self-hosted OpenClaw deployment.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Normal backlog priority with limited blast radius.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.clawsweeper:queueable-fixClawSweeper marked this issue as an existing queue_fix_pr work candidate.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:otherThis issue has meaningful maintainer-visible impact outside the owned taxonomy.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.no-staleExclude from stale automation

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions