Skip to content

fix(ts-sdk): replace sqlite3 with better-sqlite3 to fix native binding resolution#4270

Merged
deshraj merged 3 commits intomainfrom
fix/replace-sqlite3-with-better-sqlite3
Mar 9, 2026
Merged

fix(ts-sdk): replace sqlite3 with better-sqlite3 to fix native binding resolution#4270
deshraj merged 3 commits intomainfrom
fix/replace-sqlite3-with-better-sqlite3

Conversation

@utkarsh240799
Copy link
Copy Markdown
Contributor

@utkarsh240799 utkarsh240799 commented Mar 9, 2026

Summary

  • Replace sqlite3 with better-sqlite3 in both SQLiteManager (history store) and MemoryVectorStore (in-memory vector store) to fix native binding resolution failures under OpenClaw's jiti loader
  • Update all build configs (tsup.config.ts, package.json, src/oss/package.json) so the package builds cleanly
  • Add 24 tests covering both migrated modules end-to-end

Problem

The sqlite3 package uses the bindings module to locate its native .node addon at runtime by walking the V8 call stack to find the caller's __dirname. When OpenClaw loads the mem0 plugin through jiti (JIT TypeScript/ESM loader), V8 stack frames have synthetic filenames from jiti's transpiler, causing bindings to resolve the wrong module_root:

Error: Could not locate the bindings file. Tried:
 → /app/node_modules/.pnpm/jiti@2.6.1/node_modules/jiti/build/node_sqlite3.node
 → ...

The actual binary exists in the plugin's extension directory but bindings never searches there. This is architecture-independent — it affects x86_64, ARM64, macOS, and Linux equally.

Additionally, the published dist/oss/index.mjs had two unconditional top-level import sqlite3 from "sqlite3" statements (from SQLiteManager and MemoryVectorStore), so sqlite3 was evaluated eagerly at ESM parse time with no way to conditionally skip it.

Solution

Replace sqlite3 with better-sqlite3 which:

  • Ships prebuilt binaries via prebuild-install (no compile step needed)
  • Uses a different native loading mechanism that doesn't rely on bindings module path walking
  • Is synchronous (simpler API, eliminates the async callback wrapper boilerplate)
  • Has broader platform coverage out of the box (x64, ARM64, musl/glibc)

Changes

SQLiteManager.ts

  • Rewritten to use better-sqlite3's synchronous API
  • Prepared statements cached in constructor (stmtInsert, stmtSelect) for performance
  • init() is now synchronous — no more .catch(console.error) race condition
  • undefined params explicitly coerced to null for better-sqlite3 compatibility

MemoryVectorStore (memory.ts)

  • Rewritten to use better-sqlite3's synchronous API
  • Removed run(), all(), getOne() async Promise wrappers
  • Batch insert() wrapped in a transaction for atomicity and performance
  • Buffer handling updated for better-sqlite3's Uint8Array return type

tsup.config.ts

  • External array: sqlite3better-sqlite3

package.json

  • peerDependencies: sqlite3: 5.1.7better-sqlite3: ^11.9.1
  • pnpm.onlyBuiltDependencies: sqlite3better-sqlite3
  • devDependencies: Added @types/better-sqlite3: ^7.6.13
  • Removed @types/sqlite3 from peerDependencies

src/oss/package.json

  • sqlite3better-sqlite3 in dependencies
  • Removed @types/sqlite3

Tests (new: better-sqlite3-migration.test.ts)

24 tests covering:

  • SQLiteManager: in-memory, file persistence, addHistory, getHistory (DESC order), memory_id isolation, NULL handling, reset, rapid insertions
  • MemoryVectorStore: insert/get, search with cosine similarity sorting, filters, limit, dimension mismatch errors, update, delete, deleteCol, list, getUserId/setUserId, upsert behavior, file persistence

Migration

Users of the mem0ai npm package who use the OSS mode with sqlite history will need to:

  1. npm uninstall sqlite3 / pnpm remove sqlite3
  2. npm install better-sqlite3 / pnpm add better-sqlite3

No code changes required — the API is unchanged.

Test plan

  • All 24 new tests pass (npx jest --testPathPattern=better-sqlite3-migration)
  • Type-check passes (npx tsc --noEmit — zero new errors)
  • tsup build succeeds (ESM, CJS, DTS all generated cleanly)
  • better-sqlite3 correctly externalized in build output (verified in dist/oss/index.mjs)
  • Pre-existing test failures unchanged (3 suites fail on main with identical errors)
  • Test with OpenClaw plugin loading (manual)
  • Test on ARM64 (if available)

OpenClaw End-to-End Test Results

Tested the full mem0 OSS plugin lifecycle through OpenClaw's gateway (jiti-based TypeScript plugin loader) to verify the better-sqlite3 migration works in the exact environment that triggered the original sqlite3 binding failures.

Setup

Results

Test Status Details
Plugin registration openclaw-mem0: registered (mode: open-source, user: utkarsh, graph: false, autoRecall: true, autoCapture: true)
Plugin initialization openclaw-mem0: initialized — no sqlite3 binding errors, no bindings module path resolution failures
Memory capture (auto) openclaw-mem0: auto-captured 2 memories — sent "my favorite programming language is Rust and I prefer dark mode in all my editors", 2 memories extracted and stored
Memory recall #1 Asked "What is my favorite programming language?" → "Your favorite programming language is Rust."
Memory recall #2 Asked "Do I prefer light or dark mode?" → "You prefer dark mode in all your editors."
Vector store persistence mem0-vectors-test.db created (40KB), data persisted across gateway restarts
Gateway error log Zero better-sqlite3 errors after correct setup (no binding failures, no module resolution errors)

Fixes

Related

…g resolution

The `sqlite3` package uses the `bindings` module to locate its native
`.node` addon at runtime by walking up from `__dirname` of the caller.
When OpenClaw loads the mem0 plugin through jiti (JIT TypeScript/ESM
loader), the caller `__dirname` resolves to jiti's own location, but
the actual `node_sqlite3.node` binary lives in the plugin's extension
directory. The `bindings` module never searches there, causing:

  Error: Could not locate the bindings file. Tried:
  → ...jiti/build/node_sqlite3.node

Replace `sqlite3` with `better-sqlite3` which ships prebuilt binaries
via `prebuild-install` and uses a different native loading mechanism
that does not rely on `bindings` module path walking.

Changes:
- Rewrite SQLiteManager to use better-sqlite3 synchronous API with
  cached prepared statements for addHistory/getHistory
- Rewrite MemoryVectorStore to use better-sqlite3 synchronous API
  with transaction-wrapped batch inserts
- Update tsup.config.ts external: sqlite3 → better-sqlite3
- Update peerDependencies and onlyBuiltDependencies in package.json
- Update src/oss/package.json dependencies
- Add @types/better-sqlite3 to devDependencies
- Add 24 tests covering SQLiteManager, MemoryVectorStore, file
  persistence, cosine similarity search, filters, and userId mgmt

Fixes #4107
Fixes #4050
Fixes #4172
Fixes #4156

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@CLAassistant
Copy link
Copy Markdown

CLAassistant commented Mar 9, 2026

CLA assistant check
All committers have signed the CLA.

@utkarsh240799 utkarsh240799 requested a review from whysosaket March 9, 2026 08:13
utkarsh240799 and others added 2 commits March 9, 2026 19:57
Upgrade to latest better-sqlite3 v12 for improved Node.js support
(20.x, 22.x, 23.x, 24.x, 25.x). Includes prettier formatting fixes
for test and vector store files.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Add v2.3.0 changelog entry for the better-sqlite3 migration,
documenting the bug fixes, performance improvements, and
Node 20+ requirement for OSS sqlite features.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@deshraj deshraj merged commit 36537c8 into main Mar 9, 2026
4 checks passed
@deshraj deshraj deleted the fix/replace-sqlite3-with-better-sqlite3 branch March 9, 2026 18:00
utkarsh240799 added a commit that referenced this pull request Mar 10, 2026
Bump plugin version to 0.3.0 to reflect the upstream mem0ai
dependency update with sqlite3 to better-sqlite3 migration (#4270).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
jamebobob pushed a commit to jamebobob/mem0-vigil-recall that referenced this pull request Mar 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

4 participants