Summary

Propose an improved backup architecture for OpenClaw that addresses current issues with file locking, symlink handling, exclusion rule configurability, and backup size optimization.

Problem Statement

The current openclaw backup create command has several known issues:

• Bug: openclaw backup create fails with ENOENT when session file is cleaned up during backup #67417: Fails with ENOENT when session files are cleaned up during backup (race condition)
• [Bug]: openclaw backup create --verify fails with duplicate entry path when workspace contains symlinks #57515: Duplicate path errors when workspace contains symlinks
• [Feature]: Exclude node_modules from extensions in backup archive #64144: No way to exclude node_modules or other large directories from extensions
• No configurable exclusion rules — users cannot customize what gets backed up
• No service awareness — gateway remains running during backup, causing SQLite locks

Proposed Architecture

1. Service-Aware Backup Lifecycle

1. Detect locked files (SQLite, WAL files)
2. If locked → openclaw gateway stop
3. Create tar.gz archive
4. Restore → openclaw gateway start
5. If user rejects service control → fall back to best-effort with warning

This solves #67417 by eliminating the race condition between backup enumeration and session cleanup.

2. Configurable Exclusion Rules

Exclusion rules should be stored in a config file, not hardcoded:

{
  "excluded_extensions": ["log", "tmp", "pyc", "bak", "swp"],
  "excluded_folders": [
    "node_modules", ".git", "__pycache__", ".venv",
    "extensions", "browser", "delivery-queue", "flows",
    "skills-backup"
  ],
  "excluded_files": [".DS_Store", "Thumbs.db"]
}

Why this matters: Every OpenClaw installation is different. Some users have 100+ skills, some run in environments with node_modules or browser caches. Without configurability, backups either include gigabytes of rebuildable data or exclude things users actually need.

3. Symlink-Aware Traversal

Skip symbolic links and junction points entirely using FileAttributes.ReparsePoint (Windows) or lstat checks (Unix). This prevents:

• Duplicate paths in the archive ([Bug]: openclaw backup create --verify fails with duplicate entry path when workspace contains symlinks #57515)
• Infinite recursion through circular symlinks
• Unintentional backup of linked directories (e.g., projects -> /d/Research/Projects)

4. Output Format

Use tar.gz for universal compatibility:

• Zero external dependencies on most platforms
• Paths stored as absolute (e.g., C:\Users\...\) for disaster recovery
• Restore command: tar -xzf backup.tar.gz
• Compressed size typically 4-10x smaller than raw data

5. Two Backup Modes

Reference Implementation

I built a standalone tool implementing this design:

• Repository: JIRBOY/openclaw-backup
• Stack: .NET 8, zero external dependencies (System.Formats.Tar + GZipStream)
• Features: Service management (gateway stop/start), UAC auto-elevation, XML-configurable exclusions, backup rotation, dry-run mode
• Real-world results on a typical installation:
  • Naive scan: 13,400 files, 256 MB
  • With exclusion rules: 1,995 files, 8.6 MB compressed

Related Issues

• Bug: openclaw backup create fails with ENOENT when session file is cleaned up during backup #67417 — ENOENT race condition (solved by service-aware backup)
• [Bug]: openclaw backup create --verify fails with duplicate entry path when workspace contains symlinks #57515 — Symlink duplicate paths (solved by symlink-aware traversal)
• [Feature]: Exclude node_modules from extensions in backup archive #64144 — Exclude node_modules (solved by configurable exclusions)
• [Feature]: Backup & Restore for settings and migration #26515 — Backup & Restore for settings (addressed by config-only mode)
• Add backup/restore utility for config, cron jobs, and session history #13616 — Backup/restore utility (addressed by full mode)

Impact

This would give OpenClaw users:

1. Reliable backups — no more ENOENT failures or locked file errors
2. Smaller archives — configurable exclusions prevent bloated backups
3. Cross-platform restore — tar.gz works everywhere
4. Disaster recovery confidence — knowing that a single archive can restore the entire setup