Skip to content

Plugin discovery loads all dist/extensions/ manifests at boot regardless of tools.allow (~500 MB structural heap) #70533

@jpippo364

Description

@jpippo364

Summary

Plugin discovery reads every manifest under dist/extensions/ via readFileSync and runs each through zod validation at boot — even when tools.allow restricts the enabled plugin set to a small subset. For an install with 104 bundled extensions and a 6-plugin allowlist, this contributes ~500 MB of resident heap that scales with bundle size, not user config.

Ask: apply the tools.allow filter at discovery time (before reading/validating the manifest), not only at enable time.

Environment

  • OpenClaw version: 2026.4.15 (also reproducible on 2026.3.24)
  • Node.js: v22 (macOS, /usr/local/opt/node)
  • OS: macOS 15.4 (Darwin 25.4.0)
  • Install method: npm i -g openclaw
  • Bundled extensions: 104 directories under dist/extensions/
  • Active plugins in config: 6 (tools.allow restricts to icecream-ito + a handful of providers)

Evidence

Captured a V8 sampling heap profile (node --heap-prof --heap-prof-interval=524288) against a warm, live gateway with workload. Aggregated top contributors by source file:

MB % Location What it is
127 21.3% node_modules/openclaw/[zod] Schema validation — every plugin's tool/config schemas compiled
109 18.4% node:fs readFileSync Plugin manifest + skills files read at boot
89 14.9% node_modules/openclaw/[source-map] Source-map parsing (separate issue — also exacerbated by N manifests)
36 6.0% node:vm Script Module compilation
22 3.8% node_modules/openclaw/[jiti] Runtime TS transpile of manifests
20 3.3% node_modules/openclaw/[json5] Config parsing
17 2.8% node_modules/openclaw core Runtime
10 1.7% icecream-ito-plugin User's actual plugin

Combined zod + readFileSync + jiti + json5 + source-map + vm = ~75% of sampled heap (~450 MB) — all module-load overhead proportional to the number of bundled extensions, not the allowlist.

The user's plugin itself is 1.7%. The workload-side state (sessions, conversation history, search indices) is <5% combined and reclaimable by GC. The 500 MB floor is structural and persists across idle.

Current behavior

Discovery walks dist/extensions/*, reads each manifest with readFileSync, transpiles via jiti where needed, and validates with zod — before the tools.allow gate is consulted. With 104 bundled extensions, this happens 104 times at every boot.

The allowlist only prevents instantiation/registration later, not the discovery-time work that built the validated schema graph. Zod schemas, once compiled, are retained for the process lifetime.

Expected behavior

When tools.allow (or equivalent) is non-empty:

  1. Enumerate extension directory names only (cheap readdir).
  2. Filter against the allowlist.
  3. Only then: readFileSync + jiti + zod validate the manifests that survive the filter.

For the 6-of-104 case, this would save ~94% of the discovery cost (~470 MB).

Impact

Reproduction

  1. Install openclaw globally with all bundled extensions present (104).
  2. Configure tools.allow with a small subset (e.g., 6 plugins).
  3. Boot the gateway and wait for settle (~5 min).
  4. ps -o rss= on the gateway PID → resident ~1.2–1.3 GB.
  5. Run node --heap-prof --heap-prof-interval=524288 against the same config and aggregate by file-level location — readFileSync, zod, jiti, json5 dominate.

Heap profile (1.2 MB, loadable in Chrome DevTools → Memory → Load profile) and the aggregator script are available on request — not attaching inline due to GitHub issue size limits.

Workaround

None. The 104 extensions ship inside the openclaw npm package — users can't selectively remove them without breaking the install. Only upstream can filter at discovery.

Suggested fix sketch

In the plugin discovery path, approximately:

const names = await fs.readdir(extensionsDir);
const allowed = config.tools?.allow
  ? names.filter(n => config.tools.allow.includes(n))
  : names;
const manifests = await Promise.all(allowed.map(loadAndValidateManifest));

The loadAndValidateManifest step is where readFileSync + zod compile happens today. Gating that behind the allowlist check is the intended change.

Happy to test a fix against Ito's live config and report before/after RSS numbers.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High-priority user-facing bug, regression, or broken workflow.clawsweeper:fix-shape-clearClawSweeper found a clear likely implementation shape for this issue.clawsweeper:queueable-fixClawSweeper marked this issue as an existing queue_fix_pr work candidate.clawsweeper:source-reproClawSweeper found a high-confidence source-level issue reproduction.impact:crash-loopCrash, hang, restart loop, or process-level availability failure.issue-rating: 🦞 diamond lobsterVery strong issue quality with high-confidence source-level or clear reproduction.

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions