Skip to content

feat(memory): add openai-compatible embeddings provider#84930

Closed
dutifulbob wants to merge 1 commit into
openclaw:mainfrom
dutifulbob:feat/openai-compatible-embeddings-contract
Closed

feat(memory): add openai-compatible embeddings provider#84930
dutifulbob wants to merge 1 commit into
openclaw:mainfrom
dutifulbob:feat/openai-compatible-embeddings-contract

Conversation

@dutifulbob

@dutifulbob dutifulbob commented May 21, 2026

Copy link
Copy Markdown
Contributor

Summary

OpenClaw did not have a clean generic provider for self-hosted OpenAI-compatible embedding servers.
This adds an explicit openai-compatible memory embedding provider for local or self-hosted /v1/embeddings servers, without borrowing cloud OpenAI config or making vendor-specific warmup calls.
This builds on #80479 by Soham Patankar / @yaanfpv and adds the missing direct HTTP contract coverage from the implementation plan.

What Changed

The new provider is a bundled plugin that users opt into from memory-lancedb config.
It uses the existing remote embedding engine, but keeps auth, base URL, model, SSRF policy, dimensions, and cache-key behavior scoped to this provider.

  • Added extensions/openai-compatible-embeddings with provider registration, provider creation, adapter runtime metadata, manifest, package metadata, and focused tests.
  • Required embedding.baseUrl and embedding.model, with optional embedding.apiKey used only as a bearer token for this provider.
  • Avoided auto-selection, global models.providers.* inheritance, and LM Studio-style preload/warmup calls.
  • Preserved embedding.dimensions by sending OpenAI-compatible dimensions in request bodies and including it in cache keys.
  • Added direct local HTTP server tests for request path, headers, body shape, dimensions, no warmup, auth omission, parser errors, and missing config errors.
  • Added fixture-style compatibility tests for Ollama, llama.cpp llama-server, vLLM, LocalAI, TGI-compatible servers, and llamafile responses through the same generic path.
  • Updated memory-lancedb docs, changelog, labeler, and lockfile workspace importer.

Real behavior proof

Behavior addressed: explicit OpenAI-compatible memory embeddings for self-hosted /v1/embeddings servers without cloud credential inheritance or vendor warmup calls.

Real environment tested: local OpenClaw checkout on Linux, local HTTP fixture server, and local Ollama 0.24.0 at 127.0.0.1:11434 with the already-installed qwen2.5:3b model. No model was downloaded for this PR.

Exact steps or command run after this patch: node --import tsx --input-type=module smoke script that imported createOpenAICompatibleEmbeddingProvider, configured { model: "qwen2.5:3b", remote: { baseUrl: "http://127.0.0.1:11434/v1" } }, then called provider.embedQuery("hello from openclaw") and provider.embedBatch(["first memory", "second memory"]) against the local Ollama /v1/embeddings endpoint.

Evidence after fix: terminal output from the real local Ollama smoke was:

queryDims 2048
batchCount 2
batchDims [2048,2048]
finite true

Observed result after fix: provider creation made no warmup request; the first real query and batch calls succeeded through the generic /v1/embeddings path, returned finite vectors, and preserved the local Ollama model's 2048-dimensional embedding shape for both query and batch calls.

What was not tested: live llama.cpp, vLLM, TGI, LocalAI, and llamafile servers; those are covered by OpenAI-shaped compatibility fixtures, not live runtime instances.

Verification

  • node scripts/run-vitest.mjs extensions/openai-compatible-embeddings passed: 2 files, 14 tests.
  • node scripts/run-tsgo.mjs -p tsconfig.extensions.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/extensions.tsbuildinfo passed.
  • node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.extensions.test.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/extensions-test.tsbuildinfo passed.
  • node scripts/run-oxlint.mjs --tsconfig config/tsconfig/oxlint.extensions.json extensions/openai-compatible-embeddings passed.
  • PNPM_CONFIG_PM_ON_FAIL=ignore PNPM_CONFIG_VERIFY_DEPS_BEFORE_RUN=false pnpm plugins:sync:check passed.
  • ./node_modules/.bin/oxfmt --check --threads=1 ... passed for touched files.
  • node scripts/check-extension-plugin-sdk-boundary.mjs --mode=plugin-sdk-internal passed.
  • node scripts/check-extension-plugin-sdk-boundary.mjs --mode=src-outside-plugin-sdk passed.
  • node scripts/check-extension-plugin-sdk-boundary.mjs --mode=relative-outside-package passed.
  • node --import tsx scripts/check-no-extension-src-imports.ts passed.
  • node scripts/check-docs-mdx.mjs docs/plugins/memory-lancedb.md passed.
  • pnpm check:changelog-attributions passed.
  • node scripts/check-plugin-sdk-subpath-exports.mjs passed.
  • node scripts/check-plugin-extension-import-boundary.mjs passed.
  • git diff --check passed.
  • pnpm docs:list completed and confirmed the relevant docs area.
  • codex review --base main is being rerun after the review fixes.

Known local tooling notes:

  • pnpm install --lockfile-only was blocked by the repo minimum-release-age guard for unrelated web-tree-sitter@0.26.9, so the new workspace importer was added directly to pnpm-lock.yaml.
  • A local markdownlint-cli2 run expanded to the full docs tree plus historical CHANGELOG.md and reported hundreds of pre-existing changelog/docs lint errors, so it was not useful as changed-file proof.
  • pnpm changed:lanes --json without a base uses this checkout's divergent origin/main; node scripts/changed-lanes.mjs --base upstream/main --json correctly scoped this branch to extensions, docs, tooling, and all-lane CI.

Risks

The provider intentionally supports the common OpenAI-compatible embeddings contract, not every vendor quirk.
If a server claims compatibility but returns a non-OpenAI response shape, the shared remote parser will fail with the provider-specific error prefix.

  • No model install, model discovery, or auto-selection is included.
  • No cloud fallback is added.
  • Live non-Ollama runtimes still need optional runtime validation if maintainers want provider-specific proof before merge.

@github-actions

Copy link
Copy Markdown
Contributor

Dependency Changes Detected

This PR changes dependency-related files. Maintainers should confirm these changes are intentional.

Changed files:

  • extensions/openai-compatible-embeddings/package.json
  • pnpm-lock.yaml

Maintainer follow-up:

  • Review whether the dependency changes are intentional.
  • Inspect resolved package deltas when lockfile or workspace dependency policy changes are present.
  • Run pnpm deps:changes:report -- --base-ref origin/main --markdown /tmp/dependency-changes.md --json /tmp/dependency-changes.json locally for detailed release-style evidence.

@github-actions github-actions Bot added the dependencies-changed PR changes dependency-related files label May 21, 2026
@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation size: L triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 21, 2026
@clawsweeper

clawsweeper Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor

Codex review: needs changes before merge.

Workflow note: Future ClawSweeper reviews update this same comment in place.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

Summary
The PR adds a bundled openai-compatible memory embedding provider plugin, docs, tests, labeler routing, a changelog entry, and workspace lockfile metadata for self-hosted /v1/embeddings servers.

Reproducibility: yes. for the review finding by source inspection: memory-lancedb defaults an omitted model before calling the new provider, so the new missing-model check cannot fire in the documented config path. Not applicable as a bug reproduction for the feature request itself.

PR rating
Overall: 🦐 gold shrimp
Proof: 🦞 diamond lobster
Patch quality: 🦐 gold shrimp
Summary: Strong proof and a clean feature shape, but the missing-model contract bug keeps the patch from being merge-ready.

Rank-up moves:

  • Require explicit embedding.model for openai-compatible through memory-lancedb config and add a regression test for the omitted-model case.
  • Run the normal dependency-change report for the new workspace importer before landing.
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

Real behavior proof
Sufficient (live_output): The PR body includes after-fix live output from a local Ollama /v1/embeddings smoke plus local HTTP fixture coverage; non-Ollama runtimes remain fixture-only.

Risk before merge

  • A memory-lancedb config that selects openai-compatible and omits embedding.model will currently send text-embedding-3-small to the local server instead of failing fast with the promised missing-model error.
  • The dependency bot flagged package metadata and lockfile changes; the observed lockfile delta is workspace-only, but maintainers should still run the normal dependency-change review before landing.

Maintainer options:

  1. Decide the mitigation before merge
    Land this provider after enforcing the explicit model contract at the memory-lancedb config boundary while keeping the opt-in, no-warmup, SSRF-guarded remote path and direct HTTP coverage.
  2. Pause or close
    Do not merge this PR until maintainers decide whether the risk is worth taking.

Next step before merge
There is one narrow code/test blocker that automation can attempt: enforce the explicit model requirement for this provider before normal maintainer review resumes.

Security
Cleared: No concrete security or supply-chain regression found; the new network path is opt-in, uses the existing SSRF policy helper, and adds no third-party runtime dependency.

Review findings

  • [P2] Enforce the model requirement before defaulting — extensions/openai-compatible-embeddings/embedding-provider.ts:84-88
Review details

Best possible solution:

Land this provider after enforcing the explicit model contract at the memory-lancedb config boundary while keeping the opt-in, no-warmup, SSRF-guarded remote path and direct HTTP coverage.

Do we have a high-confidence way to reproduce the issue?

Yes for the review finding by source inspection: memory-lancedb defaults an omitted model before calling the new provider, so the new missing-model check cannot fire in the documented config path. Not applicable as a bug reproduction for the feature request itself.

Is this the best way to solve the issue?

No, not as-is: the provider architecture is reasonable, but the explicit model requirement needs to be enforced where memory-lancedb parses or passes config. The safer fix is a provider-specific parser/test change rather than relying on the factory to distinguish defaults from user input.

Label changes:

  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix live output from a local Ollama /v1/embeddings smoke plus local HTTP fixture coverage; non-Ollama runtimes remain fixture-only.
  • add rating: 🦐 gold shrimp: Current PR rating is 🦐 gold shrimp because proof is 🦞 diamond lobster, patch quality is 🦐 gold shrimp, and Strong proof and a clean feature shape, but the missing-model contract bug keeps the patch from being merge-ready.
  • add status: 🛠️ actively grinding: The PR author has acted after the latest ClawSweeper review and work remains. Sufficient (live_output): The PR body includes after-fix live output from a local Ollama /v1/embeddings smoke plus local HTTP fixture coverage; non-Ollama runtimes remain fixture-only.
  • remove rating: 🐚 platinum hermit: Current PR rating is rating: 🦐 gold shrimp, so this older rating label is no longer current.
  • remove status: 👀 ready for maintainer look: Current PR status label is status: 🛠️ actively grinding.

Label justifications:

  • P2: This is a normal-priority user-facing provider feature with a focused memory-plugin scope and one fixable correctness blocker.
  • rating: 🦐 gold shrimp: Current PR rating is 🦐 gold shrimp because proof is 🦞 diamond lobster, patch quality is 🦐 gold shrimp, and Strong proof and a clean feature shape, but the missing-model contract bug keeps the patch from being merge-ready.
  • status: 🛠️ actively grinding: The PR author has acted after the latest ClawSweeper review and work remains. Sufficient (live_output): The PR body includes after-fix live output from a local Ollama /v1/embeddings smoke plus local HTTP fixture coverage; non-Ollama runtimes remain fixture-only.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix live output from a local Ollama /v1/embeddings smoke plus local HTTP fixture coverage; non-Ollama runtimes remain fixture-only.

Full review comments:

  • [P2] Enforce the model requirement before defaulting — extensions/openai-compatible-embeddings/embedding-provider.ts:84-88
    memory-lancedb fills a missing embedding.model with text-embedding-3-small before this factory runs, so this check never fires for the documented config path. A config with provider: "openai-compatible" and baseUrl but no model will send the OpenAI default model name to the local server instead of failing fast with the promised embedding.model error. Please require an explicit model for this provider at the memory-lancedb config boundary, or thread whether the model was user-specified and add regression coverage.
    Confidence: 0.9

Overall correctness: patch is incorrect
Overall confidence: 0.86

Acceptance criteria:

  • node scripts/run-vitest.mjs extensions/openai-compatible-embeddings
  • node scripts/run-vitest.mjs extensions/memory-lancedb/index.test.ts extensions/memory-lancedb/config.test.ts
  • node scripts/run-tsgo.mjs -p tsconfig.extensions.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/extensions.tsbuildinfo
  • node scripts/check-docs-mdx.mjs docs/plugins/memory-lancedb.md
  • git diff --check

What I checked:

Likely related people:

  • frankekn: The current main memory-lancedb config, provider creation path, and remote embedding parser lines inspected by blame point to squash commit 2585249, whose metadata credits and reviews frankekn. (role: recent area contributor; confidence: medium; commits: 258524973798; files: extensions/memory-lancedb/index.ts, extensions/memory-lancedb/config.ts, packages/memory-host-sdk/src/host/embeddings-remote-fetch.ts)
  • yaanfpv: The closed companion PR proposed the same openai-compatible embeddings provider and this PR body credits that prior work as the basis for the current branch. (role: prior related implementation proposer; confidence: medium; commits: 229abb496db2, 727c853ff232; files: extensions/openai-compatible-embeddings, docs/plugins/memory-lancedb.md)

Codex review notes: model gpt-5.5, reasoning high; reviewed against bde07ddb1552.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. labels May 21, 2026
@clawsweeper

clawsweeper Bot commented May 21, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🔥 Warming up: real-behavior proof passed; findings, security review, or rank-up moves are still in progress.

Hatch command

Comment @clawsweeper hatch when this PR is hatchable.

Hatchability rules:

  • Merged PRs are hatchable.
  • Open PRs are hatchable when they are status: 👀 ready for maintainer look, status: 🚀 automerge armed, or labeled clawsweeper:automerge.
  • Closed unmerged PRs are hatchable only when one of those hatchable labels is still present in the durable record.
What is this egg doing here?
  • Eggs appear after the PR passes real-behavior proof. It is here for vibes, not verdicts: it does not change labels, ratings, merge decisions, or automation.
  • The shell reacts to review momentum: open follow-up work warms it up, re-review makes it wobble, and a clean final review lets it hatch.
  • Hatchability usually comes from sufficient real-behavior proof, no blocking P0/P1/P2 findings, no security attention needed, and clean correctness. A merged PR is already final, so merge makes the egg hatchable independently.
  • The hatch is seeded from this repository and PR number, so the same PR keeps the same creature; the reviewed head SHA can only change safe visual details.
  • Rarity is just collectible sparkle: 🥚 common, 🌱 uncommon, 💎 rare, ✨ glimmer, and 🌈 legendary.

@dutifulbob dutifulbob force-pushed the feat/openai-compatible-embeddings-contract branch from 825ce3b to 1453ad9 Compare May 21, 2026 11:32
@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed proof: sufficient ClawSweeper judged the real behavior proof convincing. triage: needs-real-behavior-proof Candidate: external PR needs after-fix proof from a real setup. labels May 21, 2026
@dutifulbob dutifulbob force-pushed the feat/openai-compatible-embeddings-contract branch from 1453ad9 to b818c1d Compare May 21, 2026 11:37
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 21, 2026
Co-authored-by: Soham Patankar <102520430+yaanfpv@users.noreply.github.com>
@dutifulbob dutifulbob force-pushed the feat/openai-compatible-embeddings-contract branch from b818c1d to 727c853 Compare May 21, 2026 11:42
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 21, 2026
@dutifulbob

Copy link
Copy Markdown
Contributor Author

Final implementation/proof update for the maintainer pass.

Original PR/author credit:

What changed after review:

  • Added the bundled openai-compatible-embeddings plugin using the existing memory embedding provider contract and shared remote embedding HTTP path.
  • Kept the provider explicit-only: embedding.provider: "openai-compatible", embedding.baseUrl, and embedding.model are required for use.
  • Added embedding.dimensions forwarding and included dimensions in the cache key.
  • Regenerated plugin inventory/reference docs after the generated-doc check caught the stale inventory.

Local verification on latest head 727c853ff232be53a25b80938213b6c78252f08c:

  • node scripts/run-vitest.mjs extensions/openai-compatible-embeddings -> 2 files, 14 tests passed.
  • pnpm test:extension openai-compatible-embeddings -> 2 files, 14 tests passed.
  • node scripts/run-vitest.mjs run --config test/vitest/vitest.extension-openai-compatible-embeddings.config.ts -> 2 files, 14 tests passed.
  • node scripts/run-vitest.mjs src/plugins/contracts/memory-embedding-provider.contract.test.ts -> 4 tests passed.
  • node scripts/run-vitest.mjs src/plugins/bundled-plugin-naming.test.ts -> 6 tests passed.
  • node scripts/run-tsgo.mjs -p tsconfig.extensions.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/extensions.tsbuildinfo -> passed.
  • node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.extensions.test.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/extensions-test.tsbuildinfo -> passed.
  • pnpm check:test-types -> passed.
  • node scripts/run-tsgo.mjs -p extensions/openai-compatible-embeddings/tsconfig.json --incremental false -> passed.
  • node scripts/run-oxlint.mjs --tsconfig config/tsconfig/oxlint.extensions.json extensions/openai-compatible-embeddings -> passed.
  • pnpm plugins:sync:check -> passed.
  • pnpm plugins:inventory:check -> passed.
  • node scripts/check-docs-mdx.mjs docs/plugins/memory-lancedb.md docs/plugins/plugin-inventory.md docs/plugins/reference.md docs/plugins/reference/openai-compatible-embeddings.md -> passed.
  • Extension/plugin SDK boundary checks -> passed.
  • git diff --check -> passed.
  • codex review --base main -> no actionable correctness issues.
  • pnpm check:changed -> passed in this checkout.

Real local provider proof:

  • Environment: local Ollama 0.24.0 on 127.0.0.1:11434.
  • Model used: existing installed qwen2.5:3b; no model was downloaded for this PR.
  • Smoke imported createOpenAICompatibleEmbeddingProvider({ model: "qwen2.5:3b", remote: { baseUrl: "http://127.0.0.1:11434/v1" } }).
  • Observed output:
    • queryDims 2048
    • batchCount 2
    • batchDims [2048,2048]
    • finite true

Dependency delta:

  • pnpm deps:changes:report -- --base-ref upstream/main --markdown /tmp/dependency-changes-84930.md --json /tmp/dependency-changes-84930.json -> 0 added, 0 removed, 0 changed resolved packages; 2 dependency file changes (extensions/openai-compatible-embeddings/package.json, pnpm-lock.yaml).

GitHub verification:

  • Latest SHA CI is green, including Real behavior proof, docs, dependency checks, selected CodeQL quality shard, CI node shards, build artifacts, and Windows node test.

Known proof gap:

  • Live non-Ollama OpenAI-compatible runtimes were covered by local HTTP fixtures, not live runtime processes.

@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. status: 🛠️ actively grinding The PR author has acted after the latest ClawSweeper review and work remains. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels May 21, 2026
@osolmaz osolmaz marked this pull request as draft May 21, 2026 11:54
@osolmaz

osolmaz commented May 21, 2026

Copy link
Copy Markdown
Member

do not merge, this was inspired by another user and is not a good architecture

@clawsweeper clawsweeper Bot removed the rating: 🦐 gold shrimp Decent PR readiness signal, but merge confidence is limited. label May 21, 2026
@clawsweeper clawsweeper Bot added rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. and removed status: 🛠️ actively grinding The PR author has acted after the latest ClawSweeper review and work remains. labels May 21, 2026
@dutifulbob

Copy link
Copy Markdown
Contributor Author

Closing this as superseded by #84947.

The replacement PR separates the architecture work into the general embeddingProviders contract first, so the OpenAI-compatible provider can be rebuilt on top of that contract instead of landing as a memory-specific provider/plugin shape.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies-changed PR changes dependency-related files docs Improvements or additions to documentation P2 Normal backlog priority with limited blast radius. proof: sufficient ClawSweeper judged the real behavior proof convincing. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. size: L status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants