Self-hosted RAG pipeline with Ollama and pgvector. No API keys, no telemetry, no SaaS dependencies. Companion code for the Self-Hosted RAG Pipeline with Ollama and pgvector guide on computingforgeeks.com.

Ingests a folder of Markdown or text files into PostgreSQL + pgvector, indexes them with HNSW (cosine) for vector retrieval and GIN (tsvector) for full-text retrieval, then answers questions using a hybrid Reciprocal Rank Fusion query and a local Ollama chat model. Source citations included.

The whole thing is one Python file (~290 lines).

Prerequisites: PostgreSQL 17 with pgvector, Ollama running. See the guide for install instructions on Rocky Linux and Ubuntu.

# Database role + DB
sudo -u postgres psql <<'SQL'
CREATE ROLE rag LOGIN;
CREATE DATABASE rag OWNER rag;
\c rag
CREATE EXTENSION vector;
SQL

# Models
ollama pull nomic-embed-text
ollama pull llama3.1:8b

# Python
python3 -m venv .venv
. .venv/bin/activate
pip install "psycopg[binary]>=3.2" "pgvector>=0.3.6" "ollama>=0.4"

# Initialise schema and run
export RAG_DSN="postgresql://rag@/rag?host=/var/run/postgresql"
python rag.py init
python rag.py ingest path/to/docs/
python rag.py query "What does the documentation say about X?"
python rag.py stats

All via environment variables:

Pure cosine retrieval fails on technical documentation. Ask the demo corpus "How do I install MetalLB on Kubernetes?" using only embedding <=> query and the top 5 are all unrelated install guides — the embedding model matches the "install on platform" pattern more strongly than the rare token "metallb".

The RRF fusion of cosine top-k + tsvector top-k surfaces the right document in 50 ms. See the full explanation in the article.

MIT. See LICENSE.

• Ollama Models Cheat Sheet — picking the right embedding and chat models
• Ollama Commands Cheat Sheet — full CLI and API reference
• Install pgvector on PostgreSQL — database side
• Install Ollama on Rocky Linux or Ubuntu