Skip to content

feat(embeddings): add OpenAI-compatible core provider#85269

Merged
mbelinky merged 20 commits into
openclaw:mainfrom
dutifulbob:feat/openai-generic-embedding-provider
May 27, 2026
Merged

feat(embeddings): add OpenAI-compatible core provider#85269
mbelinky merged 20 commits into
openclaw:mainfrom
dutifulbob:feat/openai-generic-embedding-provider

Conversation

@dutifulbob

@dutifulbob dutifulbob commented May 22, 2026

Copy link
Copy Markdown
Contributor

Summary

This replaces the earlier OpenAI-plugin-owned placement with the maintainer-requested shape: openai-compatible embeddings now live in core as a reusable generic embedding provider.

  • Adds a core openai-compatible embedding provider for OpenAI-style /v1/embeddings endpoints.
  • Keeps api.registerEmbeddingProvider(...) / contracts.embeddingProviders as the generic embedding-provider seam for external plugins, while the built-in OpenAI-compatible provider is available from core.
  • Bridges explicit memory configs to generic embedding providers after memory-specific providers, so memorySearch.provider: "openai-compatible" and compatible models.providers.<id> aliases work.
  • Changes legacy memorySearch.provider: "auto" and omitted provider config to OpenAI by default instead of priority-based provider auto-selection.
  • Routes the gateway /v1/embeddings endpoint through the same explicit memory/generic provider resolution path.
  • Removes the OpenAI plugin registration/copy for this provider and updates docs/doctor/CLI output for the new default and provider list.
  • Leaves existing memory-specific providers such as OpenAI, Gemini, Voyage, Mistral, DeepInfra, Bedrock, Ollama, LM Studio, GitHub Copilot, and local embeddings on the compatibility memory-provider seam for now.

Credit: Soham Patankar / @yaanfpv receives explicit credit in this PR body and via a Co-authored-by: Soham Patankar <102520430+yaanfpv@users.noreply.github.com> trailer on the OpenAI-compatible provider implementation commit. Lineage starts with #80476 and #80479, continues through #84930 and #84947, and then through Mariano Belinky PRs #84991 and #84998.

Verification

Rebased onto current origin/main (4dfc2cf14a) and verified head 051732d943:

  • node scripts/run-vitest.mjs src/plugins/embedding-provider-runtime.test.ts src/plugins/embedding-providers.test.ts src/plugins/openai-compatible-embedding-provider.test.ts extensions/memory-core/src/memory/embeddings.test.ts extensions/memory-core/src/memory/generic-embedding-provider.bridge.test.ts extensions/memory-core/src/memory/generic-embedding-provider.integration.test.ts src/gateway/embeddings-http.test.ts src/agents/memory-search.test.ts src/cli/capability-cli.test.ts src/commands/doctor-memory-search.test.ts src/plugins/loader.test.ts -> passed, 12 files / 367 tests.
  • node scripts/run-oxlint.mjs src/plugins/embedding-provider-runtime.ts src/plugins/embedding-provider-runtime.test.ts src/plugins/embedding-providers.ts src/plugins/embedding-providers.test.ts src/plugins/openai-compatible-embedding-provider.ts src/plugins/openai-compatible-embedding-provider.test.ts extensions/memory-core/src/memory/embeddings.ts extensions/memory-core/src/memory/embeddings.test.ts extensions/memory-core/src/memory/generic-embedding-provider.bridge.test.ts extensions/memory-core/src/memory/generic-embedding-provider.integration.test.ts src/gateway/embeddings-http.ts src/gateway/embeddings-http.test.ts src/agents/memory-search.ts src/agents/memory-search.test.ts src/cli/capability-cli.ts src/cli/capability-cli.test.ts src/commands/doctor-memory-search.ts src/commands/doctor-memory-search.test.ts src/config/schema.help.ts src/plugins/loader.test.ts -> passed.
  • node --max-old-space-size=8192 --import tsx scripts/generate-plugin-sdk-api-baseline.ts --check -> passed.
  • node scripts/sync-plugin-sdk-exports.mjs --check -> passed.
  • node scripts/check-plugin-sdk-subpath-exports.mjs -> passed.
  • node scripts/check-docs-mdx.mjs docs/concepts/active-memory.md docs/concepts/memory-builtin.md docs/concepts/memory-search.md docs/concepts/memory.md docs/gateway/doctor.md docs/help/faq.md docs/plugins/adding-capabilities.md docs/plugins/manifest.md docs/plugins/plugin-inventory.md docs/plugins/reference.md docs/plugins/reference/openai.md docs/plugins/sdk-overview.md docs/providers/bedrock.md docs/providers/github-copilot.md docs/reference/memory-config.md -> passed.
  • pnpm format:docs:check docs/concepts/active-memory.md docs/concepts/memory-builtin.md docs/concepts/memory-search.md docs/concepts/memory.md docs/gateway/doctor.md docs/help/faq.md docs/plugins/adding-capabilities.md docs/plugins/manifest.md docs/plugins/plugin-inventory.md docs/plugins/reference.md docs/plugins/reference/openai.md docs/plugins/sdk-overview.md docs/providers/bedrock.md docs/providers/github-copilot.md docs/reference/memory-config.md -> passed.
  • node scripts/run-tsgo.mjs -p tsconfig.core.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/core-openai-compatible-core-embeddings-rebased.tsbuildinfo -> passed.
  • node scripts/run-tsgo.mjs -p test/tsconfig/tsconfig.extensions.test.json --incremental --tsBuildInfoFile .artifacts/tsgo-cache/extensions-test-openai-compatible-core-embeddings-rebased.tsbuildinfo -> passed.
  • pnpm exec oxfmt --check --threads=1 ... on touched TS files -> passed.
  • git diff --check -> passed.
  • pnpm build -> passed.
  • Earlier live local Ollama proof against qwen2.5:3b returned real 2048-dimensional embeddings through the raw endpoint, direct provider adapter, and memory-core bridge.

Real behavior proof

Behavior addressed: OpenClaw now has a core OpenAI-compatible embedding provider for /v1/embeddings endpoints; memory-core and the gateway can use it through explicit provider config without registering it as an OpenAI plugin memory adapter or keeping priority-based memory provider auto-selection.

Real environment tested: local OpenClaw source checkout, plus earlier local Ollama 0.24.0 proof with already-installed model qwen2.5:3b. No model was downloaded for the live proof.

Exact steps or command run after this patch: rebased onto current origin/main; ran the focused Vitest, tsgo, oxlint, SDK, docs, formatting, whitespace, and build commands listed above; earlier called local Ollama POST /v1/embeddings, the OpenAI-compatible provider adapter, and the memory-core bridge against the same local endpoint.

Evidence after fix: focused tests cover core provider registration, configured OpenAI-compatible aliases, SecretRef/env API key handling, gateway /v1/embeddings, memory explicit-provider fallback to generic providers, legacy auto normalization to OpenAI, docs/doctor/CLI behavior, and rejection of unsupported multimodal generic providers. Earlier live proof returned this copied output from local Ollama through the raw endpoint, direct provider adapter, and memory bridge:

{
  "rawOllama": {
    "ok": true,
    "count": 1,
    "dimensions": 2048,
    "model": "qwen2.5:3b",
    "error": null
  },
  "providerProof": {
    "provider": "openai-compatible",
    "model": "qwen2.5:3b",
    "queryDimensions": 2048,
    "batchCount": 2,
    "batchDimensions": [2048, 2048]
  },
  "memoryBridgeProof": {
    "registered": [{ "ownerPluginId": "core", "id": "openai-compatible" }],
    "requestedProvider": "openai-compatible",
    "provider": "openai-compatible",
    "model": "qwen2.5:3b",
    "queryDimensions": 2048,
    "batchCount": 2,
    "batchDimensions": [2048, 2048]
  },
  "gatewayBridgeProof": {
    "explicitProvider": "openai-compatible",
    "aliasProvider": "ollama-local",
    "httpEndpoint": "/v1/embeddings",
    "coveredBy": "src/gateway/embeddings-http.test.ts"
  },
  "singleRegistrationCheck": {
    "newProvider": "openai-compatible",
    "registeredWith": "core embedding provider registry",
    "notRegisteredWith": "api.registerMemoryEmbeddingProvider"
  },
  "aliasPrefixCheck": {
    "providerAlias": "ollama-local",
    "configuredModel": "ollama-local/qwen2.5:3b",
    "requestModel": "qwen2.5:3b"
  },
  "secretRefCheck": {
    "missingEnvTemplate": "throws unresolved SecretRef instead of sending literal template"
  }
}

Observed result after fix: explicit memorySearch.provider: "openai-compatible" resolves to the core generic provider, compatible models.providers.<id> aliases reuse configured base URL/auth/headers/model prefix handling, unresolved env-template secrets fail closed, gateway embeddings use the same explicit provider path, generic text-only providers are rejected for multimodal memory configs, and omitted/legacy auto memory provider config resolves to OpenAI by default.

What was not tested: hosted OpenAI-compatible services beyond local Ollama were not called after this patch.

@openclaw-barnacle openclaw-barnacle Bot added docs Improvements or additions to documentation extensions: memory-core Extension: memory-core extensions: openai size: XL triage: mock-only-proof Candidate: PR proof only shows tests, mocks, snapshots, lint, typecheck, or CI. labels May 22, 2026
@clawsweeper

clawsweeper Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

Codex review: found issues before merge. Reviewed May 27, 2026, 8:32 AM ET / 12:32 UTC.

Summary
The branch adds a core OpenAI-compatible embedding provider, bridges memory and gateway embedding resolution to generic providers, updates provider docs/doctor/CLI output, and changes memory provider defaults from auto-selection to OpenAI.

PR surface: Source +620, Tests +1759, Docs -27. Total +2352 across 37 files.

Reproducibility: yes. Source inspection shows current main keeps auto selection, while the PR changes the resolver, runtime, tests, and docs so omitted and legacy auto provider configs resolve to OpenAI.

Review metrics: 1 noteworthy metric.

  • Memory default behavior: 2 changed default paths. Both omitted provider config and legacy provider: "auto" now resolve to OpenAI, which is upgrade-sensitive before merge.

Merge readiness
Overall: 🧂 unranked krab
Proof: 🦞 diamond lobster
Patch quality: 🧂 unranked krab
Result: blocked by patch quality or review findings.

Overall follows the weaker of proof and patch quality, so missing proof can cap an otherwise strong patch.

Rank-up moves:

  • Preserve omitted and legacy auto provider selection unless maintainers explicitly approve the OpenAI default migration.
  • Remove the release-owned CHANGELOG.md entry from this normal feature PR.
  • Get maintainer direction on whether the baseUrl/OpenAI model API alias policy belongs in core or plugin-owned metadata.

Risk before merge

  • Existing users who rely on implicit local, GitHub Copilot, Gemini, Voyage, Mistral, DeepInfra, or Bedrock auto-selection may be forced to OpenAI or lose semantic memory if OpenAI credentials are absent.
  • The core alias policy for baseUrl/openai-completions/openai-responses model providers is provider-routing policy; maintainers should explicitly accept that ownership or move it behind plugin-owned metadata.
  • The release-owned changelog edit should not land in a normal feature PR.

Maintainer options:

  1. Preserve auto-selection by default (recommended)
    Keep omitted and legacy memorySearch.provider: "auto" on the existing auto-selection path, and make OpenAI-compatible available through explicit provider config only.
  2. Accept an OpenAI-default migration
    Maintainers can intentionally accept the default change, but it should come with explicit upgrade notes, doctor guidance, and proof for users without OpenAI credentials.
  3. Pause for provider-routing direction
    If the core alias policy is not settled, pause this PR until maintainers decide whether baseUrl and OpenAI model API aliases belong in core or plugin metadata.

Next step before merge
Maintainers need to decide or require a repair for the memory default/provider-routing migration before this can be a safe merge candidate.

Security
Cleared: No concrete supply-chain or security-boundary regression was found in the diff; the new configurable network path uses the existing SSRF guard and SecretRef resolution, while provider/default routing remains a compatibility merge risk.

Review findings

  • [P1] Preserve auto provider selection on upgrade — src/agents/memory-search.ts:184-188
  • [P3] Remove the release-owned changelog entry — CHANGELOG.md:5-9
Review details

Best possible solution:

Land the provider and bridge only after preserving existing auto-selection for omitted/legacy provider configs, or after an explicit maintainer-approved migration plan; remove the release-owned changelog edit before merge.

Do we have a high-confidence way to reproduce the issue?

Yes. Source inspection shows current main keeps auto selection, while the PR changes the resolver, runtime, tests, and docs so omitted and legacy auto provider configs resolve to OpenAI.

Is this the best way to solve the issue?

No, not as written. The OpenAI-compatible provider is a useful direction, but it should be added without changing existing default/auto provider selection unless maintainers explicitly approve that migration.

Full review comments:

  • [P1] Preserve auto provider selection on upgrade — src/agents/memory-search.ts:184-188
    This rewrites both omitted memory provider config and legacy provider: "auto" to OpenAI before memory-core can run the current auto-selection path. Existing users who rely on local, GitHub Copilot, Gemini, Voyage, Mistral, DeepInfra, or Bedrock auto-selection can silently switch providers or lose semantic memory when OpenAI credentials are absent; keep the existing auto behavior for upgraded configs and make the new provider explicit.
    Confidence: 0.91
  • [P3] Remove the release-owned changelog entry — CHANGELOG.md:5-9
    Normal PRs should carry release-note context in the PR body or commit message, while CHANGELOG.md is generated during release work. Please drop this Unreleased entry from the branch.
    Confidence: 0.94

Overall correctness: patch is incorrect
Overall confidence: 0.89

AGENTS.md: found and applied where relevant.

Codex review notes: model gpt-5.5, reasoning high; reviewed against 3e351b718e28.

Label changes

Label changes:

  • add proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix live local Ollama terminal output showing raw endpoint, direct provider, memory bridge, and gateway-bridge behavior, with focused tests and build/type/lint checks listed.

Label justifications:

  • P2: This is a useful feature with normal priority, but the remaining blocker is an upgrade-sensitive default/provider-routing change rather than an emergency.
  • merge-risk: 🚨 compatibility: Changing omitted and legacy memory provider configs from auto-selection to OpenAI can break existing upgraded setups.
  • merge-risk: 🚨 auth-provider: The PR changes which provider credentials are used for memory embeddings and maps configured model-provider aliases into the new embedding provider path.
  • merge-risk: 🚨 availability: Users without OpenAI credentials but with another auto-selectable provider can lose semantic memory availability after merge.
  • rating: 🧂 unranked krab: Overall readiness is 🧂 unranked krab; proof is 🦞 diamond lobster and patch quality is 🧂 unranked krab.
  • status: ⏳ waiting on author: ClawSweeper has contributor-facing work open and is waiting for author action. Sufficient (terminal): The PR body includes after-fix live local Ollama terminal output showing raw endpoint, direct provider, memory bridge, and gateway-bridge behavior, with focused tests and build/type/lint checks listed.
  • proof: sufficient: Contributor real behavior proof is sufficient. The PR body includes after-fix live local Ollama terminal output showing raw endpoint, direct provider, memory bridge, and gateway-bridge behavior, with focused tests and build/type/lint checks listed.
Evidence reviewed

PR surface:

Source +620, Tests +1759, Docs -27. Total +2352 across 37 files.

View PR surface stats
Area Files Added Removed Net
Source 12 982 362 +620
Tests 12 1796 37 +1759
Docs 13 120 147 -27
Config 0 0 0 0
Generated 0 0 0 0
Other 0 0 0 0
Total 37 2898 546 +2352

What I checked:

Likely related people:

  • dutifulbob: Merged PR feat(plugins): add embedding provider contract #84947 introduced the generic embedding-provider registry/runtime/docs that this branch extends. (role: introduced current-main embedding provider contract; confidence: high; commits: ae4806ed9adf; files: src/plugins/embedding-providers.ts, src/plugins/embedding-provider-runtime.ts, src/plugin-sdk/embedding-providers.ts)
  • mbelinky: Authored the current branch commits that add the memory bridge, core OpenAI-compatible provider, and latest default-provider test update; also owns related stacked PRs referenced by the discussion. (role: recent area contributor; confidence: high; commits: 7c17ae4d7d27, 2dd3bed8c2c0, 6443dc24377b; files: extensions/memory-core/src/memory/embeddings.ts, src/plugins/openai-compatible-embedding-provider.ts, src/agents/memory-search.ts)
  • yaanfpv: The PR body and comments preserve the lineage from the original OpenAI-compatible embeddings issue and implementation, and the provider implementation commit credits that lineage. (role: originating implementation contributor; confidence: medium; commits: 2dd3bed8c2c0; files: src/plugins/openai-compatible-embedding-provider.ts, src/plugins/openai-compatible-embedding-provider.test.ts)
What the crustacean ranks mean
  • 🦀 challenger crab: rare, exceptional readiness with strong proof, clean implementation, and convincing validation.
  • 🦞 diamond lobster: very strong readiness with only minor maintainer review expected.
  • 🐚 platinum hermit: good normal PR, likely mergeable with ordinary maintainer review.
  • 🦐 gold shrimp: useful signal, but proof or patch confidence is still limited.
  • 🦪 silver shellfish: thin signal; proof, validation, or implementation needs work.
  • 🧂 unranked krab: not merge-ready because proof is missing/unusable or there are serious correctness or safety concerns.
  • 🌊 off-meta tidepool: rating does not apply to this item.

Shiny media proof means a screenshot, video, or linked artifact directly shows the changed behavior. Runtime, network, CSP, and security claims still need visible diagnostics.

How this review workflow works
  • ClawSweeper keeps one durable marker-backed review comment per issue or PR.
  • Re-runs edit this comment so the latest verdict, findings, and automation markers stay together instead of adding duplicate bot comments.
  • A fresh review can be triggered by eligible @clawsweeper re-review comments, exact-item GitHub events, scheduled/background review runs, or manual workflow dispatch.
  • PR/issue authors and users with repository write access can comment @clawsweeper re-review or @clawsweeper re-run on an open PR or issue to request a fresh review only.
  • Maintainers can also comment @clawsweeper review to request a fresh review only.
  • Fresh-review commands do not start repair, autofix, rebase, CI repair, or automerge.
  • Maintainer-only repair and merge flows require explicit commands such as @clawsweeper autofix, @clawsweeper automerge, @clawsweeper fix ci, or @clawsweeper address review.
  • Maintainers can comment @clawsweeper explain to ask for more context, or @clawsweeper stop to stop active automation.

@openclaw-barnacle openclaw-barnacle Bot added proof: supplied External PR includes structured after-fix real behavior proof. and removed triage: mock-only-proof Candidate: PR proof only shows tests, mocks, snapshots, lint, typecheck, or CI. labels May 22, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. P2 Normal backlog priority with limited blast radius. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. merge-risk: 🚨 security-boundary 🚨 May affect sandboxing, authorization, credentials, or sensitive data. labels May 22, 2026
@clawsweeper

clawsweeper Bot commented May 22, 2026

Copy link
Copy Markdown
Contributor

ClawSweeper PR egg

🔥 Warming up: real-behavior proof passed; findings, security review, or rank-up moves are still in progress.

Hatch command

Comment @clawsweeper hatch when this PR is hatchable.

Hatchability rules:

  • Merged PRs are hatchable.
  • Open PRs are hatchable when they are status: 👀 ready for maintainer look, status: 🚀 automerge armed, or labeled clawsweeper:automerge.
  • Closed unmerged PRs are hatchable only when one of those hatchable labels is still present in the durable record.
What is this egg doing here?
  • Eggs appear after the PR passes real-behavior proof. It is here for vibes, not verdicts: it does not change labels, ratings, merge decisions, or automation.
  • The shell reacts to review momentum: open follow-up work warms it up, re-review makes it wobble, and a clean final review lets it hatch.
  • Hatchability usually comes from sufficient real-behavior proof, no blocking P0/P1/P2 findings, no security attention needed, and clean correctness. A merged PR is already final, so merge makes the egg hatchable independently.
  • The hatch is seeded from this repository and PR number, so the same PR keeps the same creature; the reviewed head SHA can only change safe visual details.
  • Rarity is just collectible sparkle: 🥚 common, 🌱 uncommon, 💎 rare, ✨ glimmer, and 🌈 legendary.

@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 22, 2026
@clawsweeper clawsweeper Bot added proof: sufficient ClawSweeper judged the real behavior proof convincing. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action. merge-risk: 🚨 auth-provider 🚨 May break OAuth, tokens, provider routing, model choice, or credentials. and removed rating: 🐚 platinum hermit Good normal PR readiness with ordinary maintainer review expected. status: 👀 ready for maintainer look ClawSweeper has no concrete contributor-facing blocker left for this PR. labels May 22, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 22, 2026
@dutifulbob dutifulbob force-pushed the feat/openai-generic-embedding-provider branch from a697230 to 94187b6 Compare May 22, 2026 08:12
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 22, 2026
@dutifulbob dutifulbob force-pushed the feat/openai-generic-embedding-provider branch from 94187b6 to 5fb22f3 Compare May 22, 2026 08:22
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 22, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 22, 2026
@openclaw-barnacle openclaw-barnacle Bot removed the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 22, 2026
@clawsweeper clawsweeper Bot added the proof: sufficient ClawSweeper judged the real behavior proof convincing. label May 22, 2026
@openclaw-barnacle openclaw-barnacle Bot added the scripts Repository scripts label May 27, 2026
@mbelinky mbelinky force-pushed the feat/openai-generic-embedding-provider branch from 9c7c026 to 03c021c Compare May 27, 2026 09:28
@mbelinky

Copy link
Copy Markdown
Contributor

Merged via squash.

Thanks @dutifulbob!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

agents Agent runtime and tooling cli CLI command changes commands Command implementations docs Improvements or additions to documentation extensions: memory-core Extension: memory-core gateway Gateway runtime merge-risk: 🚨 auth-provider 🚨 May break OAuth, tokens, provider routing, model choice, or credentials. merge-risk: 🚨 availability 🚨 May cause crashes, hangs, restart loops, stalls, or process outages. merge-risk: 🚨 compatibility 🚨 May break existing users, config, migrations, defaults, or upgrade paths. P2 Normal backlog priority with limited blast radius. proof: supplied External PR includes structured after-fix real behavior proof. rating: 🧂 unranked krab Not merge-ready due to missing proof or serious correctness/safety concerns. size: XL status: ⏳ waiting on author ClawSweeper has contributor-facing work open and is waiting for author action.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants