Skip to content

fix(memory-lancedb): [P1] add missing runtime deps — plugin broken on every global install#22692

Closed
mahsumaktas wants to merge 3 commits intoopenclaw:mainfrom
mahsumaktas:fix/memory-lancedb-missing-deps
Closed

fix(memory-lancedb): [P1] add missing runtime deps — plugin broken on every global install#22692
mahsumaktas wants to merge 3 commits intoopenclaw:mainfrom
mahsumaktas:fix/memory-lancedb-missing-deps

Conversation

@mahsumaktas
Copy link
Copy Markdown
Contributor

@mahsumaktas mahsumaktas commented Feb 21, 2026

Severity: P1 — memory-lancedb is non-functional on global npm installs (all platforms)

The memory-lancedb extension fails to load on any globally installed OpenClaw instance. memory_store, memory_recall, and all long-term memory operations are broken.

memory-lancedb: failed to load LanceDB.
Error: Cannot find module 'apache-arrow'
Require stack:
  - @lancedb/lancedb/dist/arrow.js

Root Cause

@lancedb/lancedb@0.26.2 has runtime dependencies that aren't explicitly listed in extensions/memory-lancedb/package.json:

Dependency Role Why it's missing
apache-arrow Peer dep of @lancedb/lancedb Peer deps aren't auto-installed; pnpm hoists to .ignored/
flatbuffers Dep of apache-arrow Transitive dep, lost when parent is unresolvable
reflect-metadata Dep of @lancedb/lancedb (embedding registry) Hoisted to .ignored/ by pnpm workspace
pnpm's workspace hoisting places these under node_modules/.ignored/ instead of the standard node_modules/ path. When the extension's code does require('apache-arrow') at runtime, Node.js resolution can't find it.

Note on native platform bindings: @lancedb/lancedb already declares all platform-specific binaries (@lancedb/lancedb-darwin-arm64, @lancedb/lancedb-linux-x64-gnu, etc.) as its own optionalDependencies. These propagate correctly through the package manager — no explicit re-declaration needed in this extension's package.json.

Fix

Single-file change: promote implicit peer/transitive dependencies to explicit entries in extensions/memory-lancedb/package.json.

  "dependencies": {
    "@lancedb/lancedb": "^0.26.2",
    "@sinclair/typebox": "0.34.48",
+   "apache-arrow": ">=15.0.0 <=18.1.0",
+   "flatbuffers": "^24.3.25",
    "openai": "^6.22.0",
+   "reflect-metadata": "^0.2.2"
  },

Version ranges are exact matches from npm registry:

  • apache-arrow: matches @lancedb/lancedb@0.26.2's peer dep range (>=15.0.0 <=18.1.0)
  • flatbuffers: matches apache-arrow@18.x's dependency (^24.3.25)
  • reflect-metadata: matches @lancedb/lancedb@0.26.2's dependency (^0.2.2)

Why this approach

Alternative Problem
shamefully-hoist=true in .npmrc Global setting, affects all packages, defeats pnpm's isolation
pnpm.overrides Doesn't fix global npm install (only pnpm workspace)
Postinstall script Fragile, platform-detection complexity, harder to audit
Explicit deps in package.json Correct, portable, works with npm/pnpm/yarn, zero side effects

Scope

  • 1 file changed: extensions/memory-lancedb/package.json
  • +3 dependencies (apache-arrow, flatbuffers, reflect-metadata)
  • Zero code changes — only dependency declarations
  • Non-breaking: existing installs where deps happen to resolve will continue to work

Reproduction

npm install -g openclaw     # any platform
openclaw config set plugins.entries.memory-lancedb.enabled true --json
openclaw config set plugins.entries.memory-lancedb.config.embedding.apiKey "sk-..." --json
openclaw gateway restart
# → memory-lancedb: failed to load LanceDB. Error: Cannot find module 'apache-arrow'

Verified

Tested on macOS ARM64 (M4) with OpenClaw v2026.2.19-2. After manually installing these deps into the extension's node_modules, memory-lancedb loads and operates correctly:

memory-lancedb: initialized (db: ~/.openclaw/memory/lancedb, model: text-embedding-3-small)

Note: exports field gap in @lancedb/lancedb

There is a separate upstream issue: @lancedb/lancedb's package.json exports field doesn't expose ./dist/arrow, which OpenClaw's extension code requires. This needs either an upstream LanceDB fix or a build-time patch in OpenClaw. Filed separately — this PR focuses on the dependency resolution fix which addresses the most common failure mode.

Fixes #22687
Related: #19466 (Docker variant of the same issue)

Greptile Summary

Promotes implicit peer and transitive dependencies to explicit entries in extensions/memory-lancedb/package.json to fix global npm install failures. The @lancedb/lancedb@0.26.2 package has peer dependencies (apache-arrow) and runtime dependencies (reflect-metadata) that pnpm's workspace hoisting places under node_modules/.ignored/, making them unresolvable at runtime. This PR correctly adds these as explicit dependencies with version ranges that match the upstream package requirements.

Key changes:

  • Adds apache-arrow (>=15.0.0 <=18.1.0) matching @lancedb/lancedb's peer dependency range
  • Adds flatbuffers (^24.3.25) as transitive dependency of apache-arrow
  • Adds reflect-metadata (^0.2.2) matching @lancedb/lancedb's dependency
  • Adds all 8 platform-specific native bindings as optional dependencies pinned to 0.26.2

This approach aligns with the repository guidelines in AGENTS.md:12 which states "runtime deps must live in dependencies" for plugin installations.

Confidence Score: 5/5

  • This PR is safe to merge with minimal risk - it only adds dependency declarations without any code changes.
  • The PR correctly addresses a critical runtime failure by adding missing dependencies to the package.json. All added dependencies exist in the lockfile with compatible versions, the approach follows repository guidelines for plugin dependencies, and the change is non-breaking (existing installs where deps happen to resolve will continue to work). The version ranges are carefully matched to upstream requirements, and all 8 platform-specific native bindings are included.
  • No files require special attention - the single changed file has straightforward dependency additions.

Last reviewed commit: d4b3756

…installs

`@lancedb/lancedb` declares `apache-arrow` as a peer dependency and
`reflect-metadata` as a regular dependency, but pnpm workspace hoisting
places the resolved packages under `node_modules/.ignored/` instead of
the standard resolution path. When OpenClaw is installed globally via
`npm install -g openclaw`, Node.js cannot resolve these at runtime:

  Error: Cannot find module 'apache-arrow'
  Require stack:
  - @lancedb/lancedb/dist/arrow.js

This breaks `memory_store`, `memory_recall`, and all memory operations
on every platform.

Fix: promote the implicit peer/transitive dependencies to explicit
entries in the extension's own package.json so they are always installed
alongside @lancedb/lancedb regardless of hoisting strategy:

- apache-arrow (peer dep of @lancedb/lancedb, range >=15.0.0 <=18.1.0)
- flatbuffers (dep of apache-arrow, lost during hoisting)
- reflect-metadata (dep of @lancedb/lancedb embedding registry)
- Platform-specific native bindings as optionalDependencies

Fixes #22687
Related: #19466 (Docker variant of the same issue)
Mahsum added 2 commits February 21, 2026 17:34
…e.json

Platform native bindings are already optional deps of @lancedb/lancedb itself.
Listing them explicitly causes pnpm frozen-lockfile to fail in CI because
the lockfile is platform-specific (generated on macOS ARM64 but CI runs on Linux).
@openclaw-barnacle
Copy link
Copy Markdown

This pull request has been automatically marked as stale due to inactivity.
Please add updates or it will be closed.

@openclaw-barnacle openclaw-barnacle Bot added the stale Marked as stale due to inactivity label Feb 28, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

extensions: memory-lancedb Extension: memory-lancedb size: XS stale Marked as stale due to inactivity

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: memory-lancedb fails on macOS (pnpm global install) — apache-arrow + native bindings missing

1 participant