Developer: Poorna Chandra D | Full Stack Developer & Project Lead

A production-grade, unified REST API middleware designed to sit in front of multiple Large Language Models (LLMs) such as OpenAI, Google Gemini, Anthropic Claude, and more. The gateway provides resilient infrastructure for applications by handling automatic failover, semantic caching via Redis to reduce costs, and granular token usage tracking in PostgreSQL. This project focuses on backend engineering challenges like smart routing and reliability, ensuring seamless model switching without client-side code changes.

• 🔄 Multi-Provider Support - Seamlessly switch between OpenAI, Google Gemini, Anthropic Claude, and more
• 🛡️ Automatic Failover - Intelligent routing with fallback mechanisms for provider outages
• ⚡ Semantic Caching - Redis-based smart caching to reduce API costs and improve response times
• 📊 Token Tracking - Granular request logging and token usage analytics in PostgreSQL
• 🔐 API Key Management - Secure authentication and rate limiting per API key
• 🎯 Smart Routing - Route requests based on prompt complexity, provider health, and cost optimization
• 📈 Observability - Real-time dashboard with health indicators and request monitoring
• 🔌 RESTful API - Standard HTTP interface compatible with LLM client libraries
• 📝 Privacy & Compliance - Built-in privacy policy and terms of service pages

• 🏠 Landing Page - Home and overview
• ⭐ Features - Detailed feature showcase
• 🤖 Providers - Supported LLM providers
• 📚 Resources - Learning materials and documentation
• 📖 API Docs - Complete API reference
• 📊 Status - System health monitoring
• 💬 Contact - Support and contact information
• 🔒 Privacy - Privacy policy
• ⚖️ Terms - Terms of service

Core Technologies:

• Backend: FastAPI (Python 3.11+)
• Database: PostgreSQL 13+
• Cache: Redis 6+
• Deployment: Docker & Docker Compose
• ORM: SQLAlchemy
• Migrations: Alembic
• Frontend: HTML5, CSS3, JavaScript

git clone https://github.com/Poorna-Chandra-D/LLM-Gateway.git
cd LLM-Gateway

docker-compose build
docker-compose up -d

The application will be available at http://localhost:8000

curl -X POST http://localhost:8000/api/chat \
  -H "Content-Type: application/json" \
  -H "X-API-Key: your-api-key" \
  -d '{
    "provider": "openai",
    "model": "gpt-3.5-turbo",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Our implementation follows a modular middleware architecture using an Adapter Pattern for different LLM providers. Requests are authenticated via API keys, checked against rate limits in Redis, and then routed based on prompt complexity or provider health.

• API Layer - FastAPI application with route handlers
• Authentication - API key validation and rate limiting
• Provider Adapters - Abstraction layer for different LLM APIs
• Caching Layer - Redis-based semantic cache for responses
• Data Layer - PostgreSQL for request tracking and analytics
• Monitoring - Real-time health checks and observability

Email: poornacd24@gmail.com
GitHub: @Poorna-Chandra-D
Repository: Poorna-Chandra-D/LLM-Gateway

Quick Links:

• 💬 Discussions
• 🐛 Issues
• 📋 Pull Requests

Focus on robustness, scalability, and API performance. Interested in failover mechanisms, load distribution, and system reliability metrics.

Concerned with cost optimization, team productivity, and deployment efficiency. Values clear documentation and easy integration.

Leveraging LLM APIs for product features. Needs seamless provider switching, caching for cost reduction, and comprehensive API docs.

Last Updated: April 5, 2026
Version: 1.0.0
License: MIT