Skip to content

Ollama provider: inf similarity scores + VLM api_base not forwarded to OpenAI client #865

@AmeerJ97

Description

@AmeerJ97

Environment

  • OpenViking: v0.2.9 (pip install)
  • Python: 3.12.3
  • Ollama: local, with nomic-embed-text (768-dim) and llama3.2:3b
  • OS: Ubuntu 24.04

Bug 1: inf similarity scores when using Ollama embedding provider

Reproduction

  1. Configure ov.conf with Ollama dense embeddings (without explicit dimension):
{
  "embedding": {
    "dense": {
      "provider": "ollama",
      "model": "nomic-embed-text",
      "api_base": "http://localhost:11434/v1",
      "api_key": "ollama"
    }
  }
}
  1. Start server, add a resource, wait for indexing
  2. Search: POST /api/v1/search/find{"status":"error","error":{"message":"Out of range float values are not JSON compliant: inf"}}

Root Cause

In openviking_cli/utils/config/embedding_config.py, get_effective_dimension() has no special case for the ollama provider — it falls through to return 2048. But nomic-embed-text returns 768-dim vectors.

This causes:

  1. Collection created with Dim: 2048
  2. Ollama returns 768-dim vectors → dimension mismatch errors during initial indexing
  3. If dimension: 768 is later added to config, new vectors are stored as 768-dim but the collection index still expects 2048
  4. At search time, cosine similarity between mismatched dimensions → float('inf') → JSON serialization error

Suggested Fix

Add Ollama dimension auto-detection in get_effective_dimension():

if provider == "ollama":
    try:
        from openviking.models.embedder.openai_embedders import OpenAIDenseEmbedder
        embedder = OpenAIDenseEmbedder(
            model_name=self.model or "nomic-embed-text",
            api_key=self.api_key or "no-key",
            api_base=self.api_base or "http://localhost:11434/v1",
            dimension=None,  # triggers _detect_dimension()
        )
        return embedder.get_dimension()
    except Exception:
        pass

Alternatively, document that dimension is required when using the Ollama provider.


Bug 2: VLM api_base not forwarded to OpenAI client

Reproduction

  1. Configure VLM with Ollama:
{
  "vlm": {
    "model": "llama3.2:3b",
    "api_base": "http://localhost:11434/v1",
    "api_key": "ollama"
  }
}
  1. Trigger memory extraction (session commit)
  2. Error: 401 - Incorrect API key provided: ollama — the request goes to api.openai.com instead of localhost:11434

Root Cause

In openviking_cli/utils/config/vlm_config.py, _build_vlm_config_dict() only forwards api_key/api_base from a providers dict entry (lines 163-166). But _migrate_legacy_config() only populates the providers dict when both self.api_key and self.provider are set. Since the simple VLM config doesn't include a provider field, get_provider_config() returns (None, None), and api_base is never forwarded to the OpenAI client.

Suggested Fix

Add a fallback in _build_vlm_config_dict():

if config:
    result["api_key"] = config.get("api_key")
    result["api_base"] = config.get("api_base")
    result["extra_headers"] = config.get("extra_headers")
else:
    # Fall back to top-level fields when no provider config found
    if self.api_key:
        result["api_key"] = self.api_key
    if self.api_base:
        result["api_base"] = self.api_base
    if self.extra_headers:
        result["extra_headers"] = self.extra_headers

Workarounds

  1. Embedding dimension: Always set dimension explicitly in ov.conf for the Ollama provider
  2. VLM api_base: Set OPENAI_API_KEY and OPENAI_BASE_URL environment variables as a workaround, or add a provider field to the VLM config

Both bugs were discovered while integrating OpenViking v0.2.9 with a local Ollama instance for an AI agent memory system. Happy to submit PRs for either fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    Status

    Done

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions