Skip to content

feat(reranker): add OpenAI/Cohere-compatible custom reranker endpoint#5342

Open
sanidhyasin wants to merge 1 commit into
mem0ai:mainfrom
sanidhyasin:feat/openai-compatible-reranker
Open

feat(reranker): add OpenAI/Cohere-compatible custom reranker endpoint#5342
sanidhyasin wants to merge 1 commit into
mem0ai:mainfrom
sanidhyasin:feat/openai-compatible-reranker

Conversation

@sanidhyasin

Copy link
Copy Markdown

Linked Issue

Closes #5149

Description

The reranker layer currently only supports a fixed allow-list of providers (cohere, sentence_transformer, zero_entropy, llm_reranker, huggingface). There is no way to point Mem0 at a self-hosted or third-party reranker that exposes a Cohere/Jina-compatible /rerank HTTP endpoint (e.g. self-deployed bge-reranker, Jina, Voyage, SiliconFlow, Together, vLLM-hosted rerankers, or internal corporate gateways) — even though the equivalent already works on the LLM/embedding side via openai_base_url.

This PR adds a new openai_compatible reranker provider that speaks the standard rerank HTTP contract:

POST {base_url}/rerank
Authorization: Bearer {api_key}
{ "model": "...", "query": "...", "documents": ["...", "..."], "top_n": N }

Changes

  • mem0/configs/rerankers/openai_compatible.pyOpenAICompatibleRerankerConfig extending BaseRerankerConfig with base_url (required), timeout, and headers.
  • mem0/reranker/openai_compatible_reranker.pyOpenAICompatibleReranker that POSTs to {base_url}/rerank using httpx, maps each result's index + relevance_score/score back onto the input documents (same return shape as CohereReranker), applies top_k, and falls back to the original order on failure (consistent with existing rerankers).
  • mem0/utils/factory.py — register "openai_compatible" in RerankerFactory.provider_to_class.
  • Teststests/rerankers/test_openai_compatible_reranker.py mocking the HTTP call (config validation, request shaping, score mapping/sorting, top_k truncation, bare-list responses, env-var base_url, error fallback, and factory wiring).
  • Docs — new docs/components/rerankers/models/openai_compatible.mdx, an entry in docs/components/rerankers/config.mdx, nav entry in docs/docs.json, and docs/llms.txt index line (verified in sync via scripts/check-llms-txt-coverage.py).

Example usage

config = {
    "reranker": {
        "provider": "openai_compatible",
        "config": {
            "base_url": "https://my-reranker.internal/v1",  # or RERANKER_BASE_URL
            "api_key": "sk-...",                              # or RERANKER_API_KEY
            "model": "bge-reranker-v2-m3",
            "top_k": 5
        }
    }
}

Type of Change

  • Bug fix (non-breaking change that fixes an issue)
  • New feature (non-breaking change that adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to change)
  • Refactor (no functional changes)
  • Documentation update

Breaking Changes

N/A — purely additive; a new opt-in provider with no change to existing providers or defaults.

Test Coverage

  • I added/updated unit tests
  • I added/updated integration tests
  • I tested manually (describe below)
  • No tests needed (explain why)

All 43 tests in tests/rerankers/ pass (12 new), and ruff check is clean on the changed files.

Checklist

  • My code follows the project's style guidelines
  • I have performed a self-review of my code
  • I have added tests that prove my fix/feature works
  • New and existing tests pass locally
  • I have updated documentation if needed

@CLAassistant

CLAassistant commented Jun 1, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@sanidhyasin sanidhyasin force-pushed the feat/openai-compatible-reranker branch from 0594c4b to 841c593 Compare June 1, 2026 19:59
Adds an `openai_compatible` reranker provider that speaks the standard
Cohere/Jina-style `/rerank` HTTP contract, so Mem0 can be pointed at any
self-hosted or third-party reranker (bge-reranker, Jina, Voyage,
SiliconFlow, Together, vLLM-hosted rerankers, internal gateways) the same
way `openai_base_url` works on the LLM/embedding side.

- new `OpenAICompatibleRerankerConfig` with `base_url` / `timeout` / `headers`
- new `OpenAICompatibleReranker` that POSTs to `{base_url}/rerank` via httpx,
  maps `index` + `relevance_score`/`score` back onto input docs, applies
  `top_k`, and falls back to original order on failure
- register `openai_compatible` in `RerankerFactory`
- unit tests covering config validation, request shaping, score mapping,
  top_k truncation, bare-list responses, env-var base_url, and error fallback
- docs page, config reference, nav entry, and llms.txt index

Closes mem0ai#5149
@sanidhyasin sanidhyasin force-pushed the feat/openai-compatible-reranker branch from 841c593 to f8c7803 Compare June 2, 2026 10:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feat(reranker): support OpenAI/Cohere-compatible custom reranker endpoint

2 participants