perf(plugins): memoize metadata snapshots per process#81570
Conversation
|
Codex review: needs maintainer review before merge. Summary Reproducibility: yes. at source/proof level: current main rebuilds plugin metadata snapshots for each load, and the PR body supplies beta6 terminal timing plus diagnostics cache-hit proof. I did not run local tests because this was a read-only review. Real behavior proof Next step before merge Security Review detailsBest possible solution: Land this scoped hotfix after maintainer review, and keep the broader freshness matrix in #81064 or a narrower follow-up. Do we have a high-confidence way to reproduce the issue? Yes, at source/proof level: current main rebuilds plugin metadata snapshots for each load, and the PR body supplies beta6 terminal timing plus diagnostics cache-hit proof. I did not run local tests because this was a read-only review. Is this the best way to solve the issue? Yes for the release-sized hotfix: the single-entry process memo is narrowly keyed, clone-isolates returned snapshots, and stays within the documented snapshot boundary. The wider freshness hardening is correctly left to the related broader PR or a follow-up. What I checked:
Likely related people:
Remaining risk / open question:
Codex review notes: model gpt-5.5, reasoning high; reviewed against 462a05621014. |
07786f8 to
61bf711
Compare
95fa019 to
3d05063
Compare
3d05063 to
f592933
Compare
fa3cddf to
ba05b66
Compare
ba05b66 to
676e3df
Compare
a076fb3 to
81edbf3
Compare
81edbf3 to
9169cf3
Compare
9169cf3 to
be937da
Compare
be937da to
fb8e1ce
Compare
|
Follow-up on the earlier clawsweeper P2s:
Current head |
|
Thanks @Kaspre. I landed the cleaned version directly on Proof on the landed commit:
This will be included in the next release. Closing this PR as superseded by the landed main commit. |
Summary
loadPluginMetadataSnapshotso repeated same-process CLI startup callers reuse metadata snapshot work.AI-assisted: yes, with Codex.
Root cause
Beta6 still calls
loadPluginMetadataSnapshothundreds of times during CLI startup. The lower-level manifest scan cache helps, but callers still repeatedly rebuild registry snapshot state, manifest registry state, owner maps, id normalizers, and metrics.The earlier slim branch avoided some broad caching risk, but it skipped policy-stale derived snapshots. On this beta6 install, many validation callers use
config: {}, which producespersisted-registry-stale-policy; skipping those snapshots left the hot path intact.Scope notes
This is the release-sized hotfix slice. It intentionally does not attempt the full freshness matrix from #81064. In particular, arbitrary direct plugin file edits in the same process are not exhaustively detected unless the persisted registry / managed npm fingerprint changes or the process restarts. That broader P2 hardening should stay with #81064 or a follow-up.
GitHub file API after the final recut reports 4 files changed, 468 additions, and 31 deletions (
size: M).Real behavior proof
Behavior or issue addressed: repeated in-process plugin metadata snapshot construction dominates beta6 CLI startup on installs with many plugins.
Real environment tested: local production OpenClaw install on Linux/WSL2, upgraded to
OpenClaw 2026.5.12-beta.6 (c4292e0). The local CLI exit-fix patch was not applied for the final proof. The after-fix timing proof used #81570 production headfb8e1ce931def59793d3218ffef7c0393f8deeddapplied to beta6 commitc4292e053d7c86361700b3acb59cb51e4a310f2d, built from a fresh beta6 proof worktree, and temporarily overlaid as a completedist/onto the same install. The current PR head is9de96674aff0415862d1ef88be3f5722a53e61a0; commits after the timing proof are limited to a memo-key Nix-mode input simplification and test fixture/size trims. The install was restored to beta6 afterward.Exact steps or command run after this patch:
pnpm -C <slim-pr-worktree> format:check src/plugins/plugin-metadata-snapshot.ts src/plugins/plugin-metadata-snapshot.memo.test.ts src/plugins/plugin-registry-snapshot.ts src/plugins/plugin-registry-snapshot.test.ts;pnpm -C <slim-pr-worktree> test src/plugins/plugin-metadata-snapshot.memo.test.ts src/plugins/plugin-registry-snapshot.test.ts src/plugins/plugin-registry.test.ts;timeout 90s pnpm tsgo:core;pnpm -C <beta6-proof-worktree> format:check src/plugins/plugin-metadata-snapshot.ts src/plugins/plugin-registry-snapshot.ts src/plugins/plugin-metadata-snapshot.memo.test.ts src/plugins/plugin-registry-snapshot.test.ts;pnpm -C <beta6-proof-worktree> test src/plugins/plugin-metadata-snapshot.memo.test.ts src/plugins/plugin-registry-snapshot.test.ts src/plugins/plugin-registry.test.ts;timeout 240s pnpm build;/usr/bin/time -p timeout 240s openclaw plugins list;/usr/bin/time -p timeout 240s openclaw gateway status;OPENCLAW_DIAGNOSTICS=timeline OPENCLAW_DIAGNOSTICS_TIMELINE_PATH=/tmp/openclaw-beta6-81570-fb8-timeline.jsonl timeout 120s openclaw plugins list --json.Evidence after fix: unpatched beta6 baseline in the same install was
openclaw plugins listreal 80.63andopenclaw gateway statusreal 64.15. With #81570 headfb8e1ceoverlaid,openclaw plugins listcompleted withPlugins (18/104 enabled)inreal 14.69,user 14.25,sys 1.53;openclaw gateway statuscompleted inreal 12.69,user 12.28,sys 1.69. The timeline run recorded 906plugins.metadata.scanspans with 896 cache hits, total metadata span time9411.2ms, max291.8ms, and no stderr.Observed result after fix: beta6 CLI metadata startup work collapses to process-local cache hits after the first few distinct contexts.
plugins listimproves from 80.63s to 14.69s, andgateway statusimproves from 64.15s to 12.69s on the same install.What was not tested: current head
9de96674aff0415862d1ef88be3f5722a53e61a0was not retimed in the local beta6 install after the post-proof memo-key Nix-mode input simplification and test-only trim commits; hosted non-draft CI is green on that exact head. The comprehensive plugin discovery freshness matrix from #81064 is intentionally out of scope for this release-sized PR; this PR only includes compact coverage for the vanished install-record case needed to keep beta6 on the persisted fast path.Verification
pnpm -C <slim-pr-worktree> format:check src/plugins/plugin-metadata-snapshot.ts src/plugins/plugin-metadata-snapshot.memo.test.ts src/plugins/plugin-registry-snapshot.ts src/plugins/plugin-registry-snapshot.test.tspnpm -C <slim-pr-worktree> test src/plugins/plugin-metadata-snapshot.memo.test.ts src/plugins/plugin-registry-snapshot.test.ts src/plugins/plugin-registry.test.tstimeout 90s pnpm tsgo:corepnpm -C <beta6-proof-worktree> format:check src/plugins/plugin-metadata-snapshot.ts src/plugins/plugin-registry-snapshot.ts src/plugins/plugin-metadata-snapshot.memo.test.ts src/plugins/plugin-registry-snapshot.test.tspnpm -C <beta6-proof-worktree> test src/plugins/plugin-metadata-snapshot.memo.test.ts src/plugins/plugin-registry-snapshot.test.ts src/plugins/plugin-registry.test.tstimeout 240s pnpm buildfrom the beta6 proof worktreegit diff --check9de96674aff0415862d1ef88be3f5722a53e61a0:Real behavior proof,check-lint,check-test-types,check-prod-types,check,build-artifacts,build-smoke,checks-node-core,checks-node-core-fast,checks-fast-protocol,Critical Quality (plugin-boundary),Critical Quality (network-runtime-boundary), selectedSecurity Highshards, and the remaining required PR checks.