Skip to content

fix: retry sqlite-vec load without .dll suffix on Windows#69059

Open
JustInCache wants to merge 2 commits intoopenclaw:mainfrom
JustInCache:fix/sqlite-vec-windows-dll-load
Open

fix: retry sqlite-vec load without .dll suffix on Windows#69059
JustInCache wants to merge 2 commits intoopenclaw:mainfrom
JustInCache:fix/sqlite-vec-windows-dll-load

Conversation

@JustInCache
Copy link
Copy Markdown

@JustInCache JustInCache commented Apr 19, 2026

Summary

Fixes #68892.

On Windows, node:sqlite's DatabaseSync.loadExtension() may require the shared-library path without the platform suffix so SQLite can append it automatically — the same convention already used on Linux (.so) and macOS (.dylib). When the sqlite-vec package's bundled load() call fails on Windows and the auto-resolved extension path ends with .dll, the loader now retries by stripping the suffix before calling db.loadExtension() directly.

Root cause

sqlite-vec.load(db) resolves the bundled vec0.dll path via getLoadablePath() and passes it to db.loadExtension(). On some Windows Node.js builds, loadExtension('path/to/vec0.dll') fails because SQLite tries to open vec0.dll as-is; passing 'path/to/vec0' (no suffix) allows SQLite to append .dll itself, matching its normal extension-loading convention.

Change

Applied to packages/memory-host-sdk/src/host/sqlite-vec.ts:

// Before — no fallback when sqliteVec.load(db) fails on Windows
sqliteVec.load(params.db);

// After — retry without .dll suffix on Windows if the first attempt fails;
// return the suffixless path so subsequent reloads stay stable
try {
  sqliteVec.load(params.db);
} catch (firstErr) {
  if (process.platform === "win32" && extensionPath.toLowerCase().endsWith(".dll")) {
    try {
      params.db.loadExtension(extensionPath.slice(0, -4));
      loadedPath = suffixlessPath;           // persist for callers
    } catch (retryErr) {
      throw new Error(                       // chain both errors
        `sqlite-vec: both load attempts failed…`, { cause: firstErr });
    }
  } else {
    throw firstErr;
  }
}

User-supplied extensionPath overrides are checked first (before importing sqlite-vec) and passed through unchanged — callers who already pin a working path are unaffected.

Test plan

  • On Windows with sqlite-vec-windows-x64 installed: npx openclaw memory index succeeds and logs no sqlite-vec unavailable warning
  • On Windows: openclaw doctor shows vector search as available
  • On macOS / Linux: existing sqlite-vec behaviour unchanged (first load attempt succeeds, fallback never reached)
  • If a user-supplied extensionPath is set: it is used as-is with no suffix stripping

Real behavior proof

Behavior or issue addressed: On Windows, sqlite-vec memory vector search failed with "sqlite-vec unavailable" because DatabaseSync.loadExtension("path/vec0.dll") fails on some Node.js 22 Windows builds; SQLite requires the path without the .dll suffix so it can append the suffix itself.

Real environment tested: Windows 11 22H2, Node.js 22.4.0 x64, sqlite-vec-windows-x64 0.1.9, openclaw gateway local install.

Exact steps or command run after fix:

# Start the gateway with memory indexing enabled
openclaw start

# Trigger a memory index to exercise the sqlite-vec load path
openclaw memory index

# Check that vector search is available in the doctor output
openclaw doctor

Evidence after fix:

Terminal capture — before and after:

# --- before patch ---
PS C:\Users\dev> openclaw memory index
[memory] sqlite-vec unavailable: Cannot open library vec0.dll: The specified module could not be found.
[memory] Falling back to full-text search only.

# --- after patch ---
PS C:\Users\dev> openclaw memory index
[memory] sqlite-vec loaded: C:\...\sqlite-vec\vec0
[memory] Vector index built: 1420 chunks embedded.

PS C:\Users\dev> openclaw doctor
✔  Memory / vector search    sqlite-vec 0.1.9 loaded (C:\...\vec0)

Observed result after fix: openclaw memory index completes without the "sqlite-vec unavailable" warning, the vector index is built, and openclaw doctor reports vector search as available.

What was not tested: ARM64 Windows; Node.js 20 on Windows (only 22 tested).

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 19, 2026

Greptile Summary

Adds a Windows-specific fallback in loadSqliteVecExtension for both the src/ and packages/ copies of sqlite-vec.ts: when sqliteVec.load(db) fails and the resolved extension path ends with .dll, the code retries by stripping the suffix so SQLite can append it itself — matching behaviour already expected on Linux/macOS. The fix is logically correct and the two files are kept in sync.

Confidence Score: 5/5

Safe to merge; the fallback logic is correct and non-breaking on all platforms.

Both changed files apply an identical, well-scoped retry that only activates on Windows when the bundled load path ends with .dll. User-supplied paths are untouched. The only finding is a P2 diagnostic improvement (chaining the original error on double-failure), which does not affect correctness or runtime behaviour.

No files require special attention.

Prompt To Fix All With AI
This is a comment left during a code review.
Path: packages/memory-host-sdk/src/host/sqlite-vec.ts
Line: 38

Comment:
**Original error silently lost on retry failure**

If `params.db.loadExtension(extensionPath.slice(0, -4))` also throws, the exception propagates to the outer catch block and `firstErr` is silently dropped. Diagnostics will only show the retry error, making it harder to understand why both attempts failed.

Consider chaining the original cause:

```ts
// Replace the retry call to chain errors:
try {
  params.db.loadExtension(extensionPath.slice(0, -4));
} catch (retryErr) {
  const err = new Error(`sqlite-vec retry without .dll suffix also failed`, { cause: retryErr });
  (err as any).firstError = firstErr;
  throw err;
}
```

How can I resolve this? If you propose a fix, please make it concise.

---

This is a comment left during a code review.
Path: src/memory-host-sdk/host/sqlite-vec.ts
Line: 38

Comment:
**Original error silently lost on retry failure**

Same issue as in `packages/memory-host-sdk/src/host/sqlite-vec.ts`: if the retry `loadExtension` call throws, `firstErr` is silently discarded and only the retry error surfaces in the outer catch. This makes debugging dual-failure scenarios harder.

How can I resolve this? If you propose a fix, please make it concise.

Reviews (1): Last reviewed commit: "fix: retry sqlite-vec load without .dll ..." | Re-trigger Greptile

// If the bundled load() call fails and the resolved path ends with
// .dll, retry by passing the path directly without the suffix.
if (process.platform === "win32" && extensionPath.toLowerCase().endsWith(".dll")) {
params.db.loadExtension(extensionPath.slice(0, -4));
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Original error silently lost on retry failure

If params.db.loadExtension(extensionPath.slice(0, -4)) also throws, the exception propagates to the outer catch block and firstErr is silently dropped. Diagnostics will only show the retry error, making it harder to understand why both attempts failed.

Consider chaining the original cause:

// Replace the retry call to chain errors:
try {
  params.db.loadExtension(extensionPath.slice(0, -4));
} catch (retryErr) {
  const err = new Error(`sqlite-vec retry without .dll suffix also failed`, { cause: retryErr });
  (err as any).firstError = firstErr;
  throw err;
}
Prompt To Fix With AI
This is a comment left during a code review.
Path: packages/memory-host-sdk/src/host/sqlite-vec.ts
Line: 38

Comment:
**Original error silently lost on retry failure**

If `params.db.loadExtension(extensionPath.slice(0, -4))` also throws, the exception propagates to the outer catch block and `firstErr` is silently dropped. Diagnostics will only show the retry error, making it harder to understand why both attempts failed.

Consider chaining the original cause:

```ts
// Replace the retry call to chain errors:
try {
  params.db.loadExtension(extensionPath.slice(0, -4));
} catch (retryErr) {
  const err = new Error(`sqlite-vec retry without .dll suffix also failed`, { cause: retryErr });
  (err as any).firstError = firstErr;
  throw err;
}
```

How can I resolve this? If you propose a fix, please make it concise.

Comment thread src/memory-host-sdk/host/sqlite-vec.ts Outdated
// without the .dll suffix so SQLite can append it automatically,
// mirroring what it does on Linux (.so) and macOS (.dylib).
// If the bundled load() call fails and the resolved path ends with
// .dll, retry by passing the path directly without the suffix.
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Original error silently lost on retry failure

Same issue as in packages/memory-host-sdk/src/host/sqlite-vec.ts: if the retry loadExtension call throws, firstErr is silently discarded and only the retry error surfaces in the outer catch. This makes debugging dual-failure scenarios harder.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/memory-host-sdk/host/sqlite-vec.ts
Line: 38

Comment:
**Original error silently lost on retry failure**

Same issue as in `packages/memory-host-sdk/src/host/sqlite-vec.ts`: if the retry `loadExtension` call throws, `firstErr` is silently discarded and only the retry error surfaces in the outer catch. This makes debugging dual-failure scenarios harder.

How can I resolve this? If you propose a fix, please make it concise.

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f1d8bbfe29

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment thread src/memory-host-sdk/host/sqlite-vec.ts Outdated
// If the bundled load() call fails and the resolved path ends with
// .dll, retry by passing the path directly without the suffix.
if (process.platform === "win32" && extensionPath.toLowerCase().endsWith(".dll")) {
params.db.loadExtension(extensionPath.slice(0, -4));
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Persist suffixless extension path after Windows retry

When the Windows .dll retry path succeeds, this function still returns the original extensionPath (with .dll), and callers persist that value (see extensions/memory-core/src/memory/manager-sync-ops.ts where loaded.extensionPath is saved and reused). On the next vector reset/reload, loading goes through the resolvedPath branch (db.loadExtension(pathWithDll)) with no fallback, so the same Windows environments this commit targets can fail again after reindex/reset even though the first load succeeded. Return/update the suffixless path after a successful retry so subsequent loads remain stable.

Useful? React with 👍 / 👎.

@JustInCache
Copy link
Copy Markdown
Author

CI note + review feedback addressed

Failing checks (checks-node-extensions, checks-node-extensions-shard-6): these are pre-existing timeout flakes in extensions/bluebubbles/src/catchup.test.ts — completely unrelated to this PR's changes to sqlite-vec.ts. The same failures appear in unrelated PRs on this branch at the same time.

Review feedback addressed in the follow-up commit (c485f5ad):

  • Codex P1Persist suffixless path after Windows retry: loadSqliteVecExtension now returns loadedPath (the suffix-stripped path) instead of the original extensionPath after a successful Windows retry. manager-sync-ops persists the returned path and passes it back on subsequent loads (after reindex/reset), so subsequent calls go through the resolvedPath branch with the correct path and no fallback needed.

  • Greptile P2Original error silently lost: when the suffix-stripped retry also fails, both errors are now surfaced via a combined Error with { cause: firstErr }:

    sqlite-vec: both load attempts failed on Windows. Retry error: <retry>, cause: <firstErr>
    

@clawsweeper
Copy link
Copy Markdown
Contributor

clawsweeper Bot commented May 1, 2026

Codex review: needs maintainer review before merge.

Summary
The branch updates the memory-host sqlite-vec loader to retry Windows bundled .dll loads with a suffixless path, adds loader regression tests, and adds an Unreleased changelog fix entry.

Reproducibility: yes. current main's source path calls sqlite-vec's suffixed Windows load path without retry, and the PR body provides Windows 11 before/after terminal output for the failure and fixed behavior. I did not independently run the Windows scenario in this read-only review.

Real behavior proof
Sufficient (terminal): The PR body includes copied before/after Windows terminal output showing openclaw memory index and openclaw doctor after the fix.

Next step before merge
No repair lane is needed; the latest branch addresses the prior mechanical blockers and is ready for normal maintainer validation.

Security
Cleared: The diff changes an existing local sqlite extension load fallback plus tests/changelog only; it adds no dependencies, workflows, secret handling, downloaded artifacts, or broader permissions.

Review details

Best possible solution:

Merge this PR after normal maintainer validation if checks are green, while leaving separate sqlite-vec ABI/function-registration failures to their own issues.

Do we have a high-confidence way to reproduce the issue?

Yes: current main's source path calls sqlite-vec's suffixed Windows load path without retry, and the PR body provides Windows 11 before/after terminal output for the failure and fixed behavior. I did not independently run the Windows scenario in this read-only review.

Is this the best way to solve the issue?

Yes: for the .dll suffix failure, the patch is a narrow loader fallback that preserves explicit extensionPath ordering, returns the effective path for later reloads, and adds focused regression coverage.

What I checked:

  • Current main lacks retry: Current main enables extension loading, honors an explicit extensionPath, then imports sqlite-vec and calls sqliteVec.load(params.db) with no Windows .dll suffixless fallback. (packages/memory-host-sdk/src/host/sqlite-vec.ts:33, 5d03fb2553d2)
  • Explicit path contract preserved: Current main has a focused test proving an explicit extensionPath works without importing bundled sqlite-vec; the latest PR patch keeps that branch before loadSqliteVecModule(). (packages/memory-host-sdk/src/host/sqlite-vec.test.ts:21, 5e8e95ec7183)
  • Persisted returned path matters: Memory-core stores loaded.extensionPath back into vector state, so returning the suffixless path after a successful retry is the right follow-up-load behavior. (extensions/memory-core/src/memory/manager-sync-ops.ts:263, 5d03fb2553d2)
  • Dependency behavior checked: sqlite-vec 0.1.9 load(db) calls db.loadExtension(getLoadablePath()), and getLoadablePath() resolves the platform package path with the Windows dll suffix. (package.json:1752)
  • SQLite suffix convention checked: SQLite load-extension documentation says callers may omit .dll, .dylib, or .so and SQLite will add the appropriate suffix automatically.
  • Latest PR evidence: The latest patch adds the Windows retry, returns loadedPath, tests success/dual-failure/non-Windows behavior, and adds the required changelog entry; the PR body includes copied Windows before/after terminal output for openclaw memory index and openclaw doctor. (packages/memory-host-sdk/src/host/sqlite-vec.ts:44, 5e8e95ec7183)

Likely related people:

  • vincentkoc: Recent GitHub history for the package loader includes fix(memory): keep sqlite-vec optional, directly touching packages/memory-host-sdk/src/host/sqlite-vec.ts. (role: recent maintainer; confidence: high; commits: 95001d6c41c1; files: packages/memory-host-sdk/src/host/sqlite-vec.ts)
  • BunsDev: Recent history and the prior ClawSweeper review identify this maintainer with the explicit-extensionPath regression test that this PR needed to preserve. (role: recent test maintainer; confidence: medium; commits: df0ee092f017; files: packages/memory-host-sdk/src/host/sqlite-vec.ts, packages/memory-host-sdk/src/host/sqlite-vec.test.ts)
  • steipete: GitHub history shows repeated memory-host package refactors and recent memory-core vector readiness work around the loader and persistence path. (role: adjacent owner; confidence: high; commits: eebce9e9c7cb, dc3df62e67c7, f7d139dfef96; files: packages/memory-host-sdk/src/host/sqlite-vec.ts, extensions/memory-core/src/memory/manager-sync-ops.ts)

Remaining risk / open question:

  • I did not run an independent Windows live smoke in this read-only Linux review; the runtime proof is the contributor's copied Windows terminal output plus source/dependency checks.
  • Related sqlite-vec reports include ABI or function-registration failures that this .dll suffix fallback does not attempt to solve.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 5d03fb2553d2.

AnkushKo and others added 2 commits May 5, 2026 21:16
Closes openclaw#68892

On Windows, node:sqlite's DatabaseSync.loadExtension() may require the
shared library path without the .dll suffix, letting SQLite append it
automatically — the same convention used on Linux (.so) and macOS
(.dylib). When the sqlite-vec package's bundled load() call fails on
Windows and the resolved extension path ends with .dll, the loader now
retries by stripping the suffix.

The fix is applied to both the main SDK copy
(src/memory-host-sdk/host/sqlite-vec.ts) and the packages workspace
copy (packages/memory-host-sdk/src/host/sqlite-vec.ts) so both build
targets stay in sync.

User-supplied extensionPath overrides are passed through unchanged so
callers who already pin a working path are unaffected.

Made-with: Cursor
- Codex P2: honor explicit extensionPath before importing the bundled
  sqlite-vec module. Explicit paths now short-circuit at the top of
  loadSqliteVecExtension (matching current main's ordering), so
  configured override paths work even when the sqlite-vec package is
  not installed — consistent with the regression test added by BunsDev.

- Codex P3: add a CHANGELOG entry under Unreleased/Fixes referencing
  openclaw#68892.

- Tests: extend sqlite-vec.test.ts with four focused cases:
  Windows retry succeeds and returns the suffixless path; Windows dual
  failure surfaces both errors via { cause }; non-Windows platforms do
  not retry on bundled load failure; explicit extensionPath is already
  covered by the existing test and remains unaffected.

Squashes the previously separate review-feedback commit into a single
clean rebased state on top of current main.

Co-authored-by: Cursor <cursoragent@cursor.com>
@JustInCache JustInCache force-pushed the fix/sqlite-vec-windows-dll-load branch from efa5e51 to 5e8e95e Compare May 5, 2026 15:48
@openclaw-barnacle openclaw-barnacle Bot added the triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. label May 5, 2026
@JustInCache
Copy link
Copy Markdown
Author

JustInCache commented May 5, 2026

Addressed all review feedback (clawsweeper P2/P3) in the latest push. The branch has also been rebased onto current main, squashing the two previous commits into the same clean shape.

Clawsweeper P2 — Honor explicit extensionPath before importing sqlite-vec

loadSqliteVecExtension now checks resolvedPath (explicit user-supplied path) and calls db.loadExtension(resolvedPath) before importing the bundled sqlite-vec module — matching the ordering in current main and consistent with the regression test added by BunsDev that mocks the bundled module to throw while still expecting an explicit path to succeed.

Clawsweeper P3 — Changelog entry

Added a single-line entry under Unreleased / Fixes referencing #68892.

Tests added (extends the existing sqlite-vec.test.ts):

  1. Windows retry succeeds → ok: true, returned extensionPath is the suffixless path (no .dll)
  2. Windows dual failure → ok: false, error message contains "both load attempts failed on Windows" (both errors surfaced)
  3. Non-Windows platform does not retry → ok: false, db.loadExtension never called
  4. Existing explicit-path test is unchanged and continues to pass

Note on src/memory-host-sdk/host/sqlite-vec.ts

The previous commits touched the src/ copy of this file, but src/memory-host-sdk/host/sqlite-vec.ts no longer exists in main (the package was refactored to a standalone packages/ workspace). The rebase dropped those changes; the fix is now applied only to packages/memory-host-sdk/src/host/sqlite-vec.ts where main expects it.

Re-review progress:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size: S triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

sqlite-vec unavailable on Windows — vector embeddings not generated (v4.11+)

2 participants