Version: openclaw 2026.6.8 (Node v24.13.0, Linux)
Summary
backup create stages its work in two temp artifacts under os.tmpdir():
- a staging dir
openclaw-backup-<rand>/ (fs.mkdtemp), holding the manifest and sanitized sqlite snapshots, and
- the intermediate archive
<outputPath>.<uuid>.tmp, which receives the full gzip tar stream before being published to the final path.
Both are removed in a single try { … } finally { fs.rm(tempArchivePath); fs.rm(tempDir) }. That cleanup is in-process only. A hard kill during the run — SIGKILL, OOM, host reboot, or a service/gateway restart that takes the process down — bypasses the finally and orphans both artifacts. The archive temp is full-size (multi-GB for a real backup), so each interrupted run permanently leaks a copy of the entire backup into tmpdir.
Nothing ever reaps them. A subsequent backup create creates a fresh mkdtemp dir and a fresh UUID-named temp; it does not scan tmpdir for stale openclaw-backup-* / *.<uuid>.tmp left by a prior crashed run. So the leak accumulates across interrupted runs until something external cleans tmpdir.
Why it matters
On a single-volume host where tmpdir shares the root filesystem, repeated interrupted backups silently refill the disk with full-size orphans. We hit this in production three times before tracing it; each incident pinned ~2.7 GB until manually cleared. The failure mode is disk-pressure, and it compounds: a disk-pressure kill during a backup leaves an orphan that brings the disk closer to the next kill.
Reproduction
- Start
backup create on a state dir large enough that archiving takes a few seconds.
kill -9 the process (or trigger an OOM/reboot) after staging begins but before publish.
- Inspect
tmpdir (os.tmpdir(), typically /tmp):
openclaw-backup-<rand>/ staging dir remains (manifest + sqlite snapshots).
<outputPath>.<uuid>.tmp full-size archive remains.
- Run
backup create again to completion. Confirm the prior run's orphans are still present — the new run does not sweep them.
Root cause (verified against source, v2026.6.8)
In the createBackupArchive path:
tempDir = fs.mkdtemp(path.join(tempRoot, "openclaw-backup-"))
tempArchivePath = ${outputPath}.${randomUUID()}.tmp
- the tar write, manifest write, and publish all run inside one
try, with cleanup of both temps only in the matching finally.
There is no startup pass that lists tmpdir and removes stale openclaw-backup-* dirs or *.<uuid>.tmp archives from earlier runs. The finally is the only cleanup, and it cannot fire when the process is hard-killed.
(Note: publish is hardlink-then-unlink-temp with a copyFile(COPYFILE_EXCL) fallback — not an atomic rename — but that detail isn't the bug; the bug is that the in-process finally is the only thing that ever removes the temps.)
Proposed fix
On backup create startup — before staging the new run — sweep the chosen temp root for the tool's own stale artifacts:
- directories matching the
openclaw-backup- mkdtemp prefix, and
- files matching
*.<uuid>.tmp siblings of the resolved output path,
removing those not currently held open / older than a small grace window (to avoid racing a concurrent run). Scope the match tightly to the tool's own naming so it can never touch unrelated files in a shared tmpdir. This makes the next run self-heal after any hard kill and fixes the leak at the source for every caller, rather than each deployment having to bolt on an external janitor.
There's accepted precedent for this exact pattern: #92874 (the memory backend's reindex temps) was the same crash-unsafe-temp-on-hard-kill bug class, and it was fixed (merged in #92891) by adding a startup sweep that removes the tool's own stale temp files left by a hard-killed run, guarded by a staleness window so it can't race a concurrent operation. backup create has the identical bug class but no such sweep; adopting the same approach here — ideally a shared startup-sweep-own-artifacts helper so both live in one place — would retire it at the source.
Workaround (for anyone hitting this before the fix)
A periodic/boot-time janitor that trashes, under tmpdir, the openclaw-backup-* staging dirs and the intermediate archives matching the verified ${outputPath}.${uuid}.tmp naming — i.e. the *.tar.gz.*.tmp suffix (this matches the default <timestamp>-openclaw-backup.tar.gz.<uuid>.tmp and any custom --output name, so a copy-paste reader doesn't get an empty match) — older than ~60 min and not held open by lsof. (This is what we shipped locally; it works but every deployment shouldn't need it.)
Version: openclaw 2026.6.8 (Node v24.13.0, Linux)
Summary
backup createstages its work in two temp artifacts underos.tmpdir():openclaw-backup-<rand>/(fs.mkdtemp), holding the manifest and sanitized sqlite snapshots, and<outputPath>.<uuid>.tmp, which receives the full gzip tar stream before being published to the final path.Both are removed in a single
try { … } finally { fs.rm(tempArchivePath); fs.rm(tempDir) }. That cleanup is in-process only. A hard kill during the run —SIGKILL, OOM, host reboot, or a service/gateway restart that takes the process down — bypasses thefinallyand orphans both artifacts. The archive temp is full-size (multi-GB for a real backup), so each interrupted run permanently leaks a copy of the entire backup intotmpdir.Nothing ever reaps them. A subsequent
backup createcreates a freshmkdtempdir and a fresh UUID-named temp; it does not scantmpdirfor staleopenclaw-backup-*/*.<uuid>.tmpleft by a prior crashed run. So the leak accumulates across interrupted runs until something external cleanstmpdir.Why it matters
On a single-volume host where
tmpdirshares the root filesystem, repeated interrupted backups silently refill the disk with full-size orphans. We hit this in production three times before tracing it; each incident pinned ~2.7 GB until manually cleared. The failure mode is disk-pressure, and it compounds: a disk-pressure kill during a backup leaves an orphan that brings the disk closer to the next kill.Reproduction
backup createon a state dir large enough that archiving takes a few seconds.kill -9the process (or trigger an OOM/reboot) after staging begins but before publish.tmpdir(os.tmpdir(), typically/tmp):openclaw-backup-<rand>/staging dir remains (manifest + sqlite snapshots).<outputPath>.<uuid>.tmpfull-size archive remains.backup createagain to completion. Confirm the prior run's orphans are still present — the new run does not sweep them.Root cause (verified against source, v2026.6.8)
In the
createBackupArchivepath:tempDir = fs.mkdtemp(path.join(tempRoot, "openclaw-backup-"))tempArchivePath = ${outputPath}.${randomUUID()}.tmptry, with cleanup of both temps only in the matchingfinally.There is no startup pass that lists
tmpdirand removes staleopenclaw-backup-*dirs or*.<uuid>.tmparchives from earlier runs. Thefinallyis the only cleanup, and it cannot fire when the process is hard-killed.(Note: publish is hardlink-then-unlink-temp with a
copyFile(COPYFILE_EXCL)fallback — not an atomic rename — but that detail isn't the bug; the bug is that the in-processfinallyis the only thing that ever removes the temps.)Proposed fix
On
backup createstartup — before staging the new run — sweep the chosen temp root for the tool's own stale artifacts:openclaw-backup-mkdtemp prefix, and*.<uuid>.tmpsiblings of the resolved output path,removing those not currently held open / older than a small grace window (to avoid racing a concurrent run). Scope the match tightly to the tool's own naming so it can never touch unrelated files in a shared
tmpdir. This makes the next run self-heal after any hard kill and fixes the leak at the source for every caller, rather than each deployment having to bolt on an external janitor.There's accepted precedent for this exact pattern: #92874 (the memory backend's reindex temps) was the same crash-unsafe-temp-on-hard-kill bug class, and it was fixed (merged in #92891) by adding a startup sweep that removes the tool's own stale temp files left by a hard-killed run, guarded by a staleness window so it can't race a concurrent operation.
backup createhas the identical bug class but no such sweep; adopting the same approach here — ideally a shared startup-sweep-own-artifacts helper so both live in one place — would retire it at the source.Workaround (for anyone hitting this before the fix)
A periodic/boot-time janitor that trashes, under
tmpdir, theopenclaw-backup-*staging dirs and the intermediate archives matching the verified${outputPath}.${uuid}.tmpnaming — i.e. the*.tar.gz.*.tmpsuffix (this matches the default<timestamp>-openclaw-backup.tar.gz.<uuid>.tmpand any custom--outputname, so a copy-paste reader doesn't get an empty match) — older than ~60 min and not held open bylsof. (This is what we shipped locally; it works but every deployment shouldn't need it.)