Skip to content

Memory: Session files loaded entirely into memory via readFileSync #69451

@ksxh0524

Description

@ksxh0524

Problem

OpenClaw gateway loads ALL session files into memory at startup:

  1. sessions.json (42MB in our case) is loaded via readFileSync + JSON.parse
  2. All JSONL session files (831 files, 389MB) are loaded via readFileSync per session access
  3. No lazy loading or streaming mechanism exists

This causes the gateway process to consume 3.5GB+ RSS on a 16GB machine after running for a few weeks.

Impact

  • Gateway RSS grows linearly with session history
  • No pagination or streaming for session messages
  • sessions.json is a single 42MB file that gets parsed entirely on every access
  • Memory never shrinks because V8 GC is not aggressive enough

Environment

  • OpenClaw version: latest
  • OS: macOS (ARM64)
  • Session count: 831 files
  • sessions.json size: 42MB

Suggested Fix

  1. Lazy-load session JSONL files only when needed
  2. Stream/paginate session messages instead of readFileSync
  3. Split sessions.json into per-session metadata files
  4. Add configurable memory limits
  5. Implement LRU cache for active sessions only

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority user-facing bug, regression, or broken workflow.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.clawsweeper:needs-maintainer-reviewClawSweeper marked this issue as needing maintainer review before automation.clawsweeper:needs-product-decisionClawSweeper marked this issue as needing a product or behavior decision.clawsweeper:no-new-fix-prClawSweeper does not recommend queueing a new automated fix PR for this issue.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.impact:session-stateSession, memory, transcript, context, or agent state can drift or corrupt.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.staleMarked as stale due to inactivity

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions