Build better LLM apps — faster, smarter, production-ready.
A curated, list of 100+ libraries and frameworks for AI engineers building with Large Language Models. This toolkit includes battle-tested tools, frameworks, templates, and reference implementations for developing, deploying, and optimizing LLM-powered systems.
🛠️ Tooling for AI Engineers
Tool
Description
Language
License
Pinecone Managed vector database for production AI applications
API/SDK
Commercial
Weaviate Open-source vector database with GraphQL API
Go
BSD-3
Qdrant Vector similarity search engine with extended filtering
Rust
Apache-2.0
Chroma Open-source embedding database for LLM apps
Python
Apache-2.0
Milvus Cloud-native vector database for scalable similarity search
Go/C++
Apache-2.0
FAISS Library for efficient similarity search and clustering
C++/Python
MIT
Deep Lake AI-native data lake with versioned datasets, optimized for embeddings and multimodal storage
Python
Apache-2.0
Vectara Managed RAG platform with APIs for retrieval and generation
Python/Go
Commercial
Orchestration & Workflows
Tool
Description
Language
License
LangChain Framework for developing LLM applications
Python/JS
MIT
LlamaIndex Data framework for LLM applications
Python
MIT
Haystack End-to-end NLP framework for production
Python
Apache-2.0
DSPy Framework for algorithmically optimizing LM prompts
Python
MIT
Semantic Kernel SDK for integrating AI into conventional programming languages
C#/Python/Java
MIT
Langflow Visual no-code platform for building and deploying LLM workflows
Python/TypeScript
MIT
Flowise Drag-and-drop UI for creating LLM chains and agents
TypeScript
MIT
Promptflow Workflow orchestration for LLM pipelines, evaluation, and deployment
Python
MIT
PDF Extraction Tools
Tool
Description
Language
License
Docling AI-powered toolkit converting PDF, DOCX, PPTX, HTML, images into structured JSON/Markdown with layout, OCR, table, and code recognition
Python
MIT
pdfplumber Drill through PDFs at a character level, extract text & tables, and visually debug extraction
Python
MIT
PyMuPDF (fitz) Lightweight, high-performance PDF parser for text/image extraction and manipulation
Python / C
AGPL-3.0
PDF.js Browser-based PDF renderer with text extraction capabilities
JavaScript
Apache-2.0
Camelot Extracts structured tabular data from PDFs into DataFrames and CSVs
Python
MIT
Unstructured Parse PDFs, DOCX, HTML into structured JSON for LLM workflows
Python
Apache-2.0
pdfminer.six Detailed PDF text extraction and layout analysis
Python
MIT
Llama Parse Structured parsing of PDFs and documents optimized for LLMs
Python
Apache-2.0
MegaParse Universal parser for PDFs, HTML, and semi-structured documents
Python
Apache-2.0
ExtractThinker Intelligent document extraction framework with schema mapping
Python
MIT
PyMuPDF4LLM Wrapper around PyMuPDF for LLM-ready text, tables, and image extraction
Python
Apache-2.0
RAG (Retrieval-Augmented Generation)
Tool
Description
Language
License
RAGFlow Open-source RAG engine based on deep document understanding
Python
Apache-2.0
Verba Retrieval Augmented Generation (RAG) chatbot
Python
BSD-3
PrivateGPT Interact with documents using local LLMs
Python
Apache-2.0
AnythingLLM All-in-one AI application for any LLM
JavaScript
MIT
Quivr Your GenAI second brain
Python/TypeScript
Apache-2.0
Jina Cloud-native neural search framework for multimodal RAG
Python
Apache-2.0
txtai All-in-one embeddings database for semantic search and workflows
Python
Apache-2.0
FastGraph RAG Graph-based RAG framework for structured retrieval
Python
MIT
Chonkie Chunking utility for efficient document processing in RAG
Python
-
FlashRAG Low-latency RAG research toolkit with modular design and benchmarks
Python
-
Llmware Lightweight framework for building RAG-based apps
Python
Apache-2.0
Tool
Description
Language
License
Evals OpenAI's framework for creating and running LLM evaluations
Python
MIT
Ragas Evaluation framework for RAG pipelines
Python
Apache-2.0
Opik DevOps platform for evaluation, monitoring, and observability
Python
Apache-2.0
Phoenix ML observability for LLM, vision, language, and tabular models
Python
Apache-2.0
DeepEval LLM evaluation framework for unit testing LLM outputs
Python
Apache-2.0
TruLens Evaluation and tracking for LLM experiments
Python
MIT
UpTrain Open-source tool to evaluate and improve LLM applications
Python
Apache-2.0
Giskard Testing framework for ML/LLMs with bias and robustness checks
Python
Apache-2.0
Weave Experiment tracking, debugging, and logging for LLM workflows
Python
Apache-2.0
Lighteval Lightweight and fast evaluation framework from Hugging Face
Python
Apache-2.0
Tool
Description
Language
License
Hugging Face Hub Client library for Hugging Face Hub
Python
Apache-2.0
MLflow Platform for ML lifecycle management
Python
Apache-2.0
Weights & Biases Developer tools for ML
Python
MIT
DVC Data version control for ML projects
Python
Apache-2.0
ClearML End-to-end MLOps platform with LLM support
Python
Apache-2.0
Data Collection & Web Scraping
Tool
Description
Language
License
Firecrawl AI-powered web crawler that extracts and structures content for LLM pipelines
TypeScript
MIT
Scrapy Fast, high-level web crawling & scraping framework
Python
BSD-3
Playwright Web automation & scraping with headless browsers
TypeScript/Python/Java/.NET
Apache-2.0
BeautifulSoup Easy HTML/XML parsing for quick scraping tasks
Python
MIT
Selenium Browser automation framework (supports scraping)
Multiple
Apache-2.0
Newspaper3k News & article extraction library
Python
MIT
Crawl4AI Fast, lightweight, and modern web crawling & scraping library for AI data pipelines
Python
Apache-2.0
Colly High-performance scraping framework for Go
Go
BSD-2
Trafilatura Extract clean text from web pages for LLM training corpora
Python
MIT
ScrapeGraphAI Use LLMs to extract structured data from websites and documents
Python
MIT
Crawlee Web scraping and crawling framework for large-scale data collection
TypeScript
Apache-2.0
Framework
Description
Language
License
Google's ADK Flexible and modular framework for developing and deploying AI agents
Python / Java
Apache-2.0
AutoGen Multi-agent conversation framework
Python
CC-BY-4.0
CrewAI Framework for orchestrating role-playing autonomous AI agents
Python
MIT
LangGraph Build resilient language agents as graphs
Python
MIT
AgentOps Python SDK for AI agent monitoring, LLM cost tracking, benchmarking
Python
MIT
Swarm Educational framework for exploring ergonomic, lightweight multi-agent orchestration
Python
MIT
Agency Swarm An open-source agent framework designed to automate your workflows
Python
MIT
Multi-Agent Systems Research into multi-agent systems and applications
Python
MIT
Auto-GPT Autonomous AI agent for task execution using GPT models
Python
MIT
BabyAGI Task-driven autonomous agent inspired by AGI
Python
MIT
SuperAGI Infrastructure for building and managing autonomous agents
Python
MIT
Griptape Framework for building AI agents with structured pipelines and memory
Python
Apache-2.0
Letta (MemGPT) Long-term memory management for LLM agents
Python
MIT
Agno Framework for building AI agents with RAG, workflows, and memory
Python
Apache-2.0
Agents SDK SDK from Vercel for building agentic workflows and applications
TypeScript
Apache-2.0
Smolagents Lightweight agent framework from Hugging Face
Python
Apache-2.0
Pydantic AI Agent framework built on Pydantic for structured reasoning
Python
MIT
CAMEL Multi-agent framework enabling role-play and collaboration
Python
Apache-2.0
Swarms Enterprise agent orchestration framework (“Agency Swarm”)
Python
MIT
Langroid Framework for building multi-agent conversational systems
Python
Apache-2.0
Upsonic Agent framework focused on context management and tool use
Python
Apache-2.0
📦 LLM Development & Optimization LLM Training and Fine-Tuning
Tool
Description
Language
License
PyTorch Lightning High-level PyTorch interface for LLMs
Python
Apache-2.0
unsloth Fine-tune LLMs faster with less memory
Python
Apache-2.0
Axolotl Post-training pipeline for AI models
Python
Apache-2.0
LLaMA-Factory Easy & efficient LLM fine-tuning
Python
Apache-2.0
PEFT Parameter-Efficient Fine-Tuning library
Python
Apache-2.0
DeepSpeed Distributed training & inference optimization
Python
MIT
TRL Train transformer LMs with reinforcement learning
Python
Apache-2.0
Transformers Pretrained models for text, vision, and audio tasks
Python
Apache-2.0
LitGPT Train and fine-tune LLMs lightning fast
Python
Apache-2.0
Ludwig Low-code framework for custom LLMs
Python
Apache-2.0
xTuring Fast fine-tuning of open-source LLMs
Python
Apache-2.0
RL4LMs RL library to fine-tune LMs to human preferences
Python
Apache-2.0
torchtune PyTorch-native library for fine-tuning LLMs
Python
BSD-3
Accelerate Library to easily train on multiple GPUs/TPUs with mixed precision
Python
Apache-2.0
Open Source LLM Inference
Tool
Description
Language
License
LLM Compressor Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment
Python
Apache-2.0
LightLLM Lightweight Python-based LLM inference and serving framework with easy scalability and high performance
Python
Apache-2.0
vLLM High-throughput and memory-efficient inference and serving engine for LLMs
Python
Apache-2.0
TensorRT-LLM NVIDIA library for optimizing LLM inference with TensorRT
C++/Python
Apache-2.0
WebLLM High-performance in-browser LLM inference engine
TypeScript/Python
Apache-2.0
SkyPilot Unified framework to run ML workloads and LLMs on any cloud (AWS, GCP, Azure, Lambda, etc.) with auto-spot, data syncing, and cost optimization.
Python
Apache-2.0
Tool
Description
Language
License
Guardrails Add guardrails to large language models
Python
MIT
NeMo Guardrails Toolkit for adding programmable guardrails to LLM conversational systems
Python
Apache-2.0
Garak LLM vulnerability scanner
Python
MIT
DeepTeam LLM red teaming framework
Python
Apache-2.0
AI App Development Frameworks
Tool
Description
Language
License
Reflex Build full-stack web apps powered by LLMs with Python-only workflows and reactive UIs.
Python
Apache-2.0
Gradio Create quick, interactive UIs for LLM demos and prototypes.
Python
Apache-2.0
Streamlit Build and share AI/ML apps fast with Python scripts and interactive widgets.
Python
Apache-2.0
Taipy End-to-end Python framework for building production-ready AI apps with dashboards and pipelines.
Python
Apache-2.0
AI SDK UI Vercel’s AI SDK for building chat & generative UIs
TypeScript
Apache-2.0
Simpleaichat Minimal Python interface for prototyping conversational LLMs
Python
MIT
Chainlit Framework for building and debugging LLM apps with a rich UI
Python
Apache-2.0
Local Development & Serving
Tool
Description
Language
License
Ollama Get up and running with large language models locally
Go
MIT
LM Studio Desktop app for running local LLMs
-
Commercial
GPT4All Open-source chatbot ecosystem
C++
MIT
LocalAI Self-hosted OpenAI-compatible API
Go
MIT
llama.cpp Lightweight, high-performance inference engine for running LLMs locally across CPU, GPU, and mobile backends
C++
MIT
LiteLLM Lightweight OpenAI-compatible gateway for multiple LLM providers
Python
MIT
AI Gateway Gateway for managing LLM requests, caching, and routing
Python
Apache-2.0
Langcorn Serve LangChain applications via FastAPI with production-ready endpoints
Python
MIT
LitServe High-speed GPU inference server with autoscaling and batch support
Python
Apache-2.0
Platform
Description
Pricing
Features
Clarifai Lightning-fast compute for AI models & agents
Free tier + Pay-as-you-go
Pre-trained models, Deploy your own models on Dedicated compute, Model training, Workflow automation
Modal Serverless platform for AI/ML workloads
Pay-per-use
Serverless GPU, Auto-scaling
Replicate Run open-source models with a cloud API
Pay-per-use
Pre-built models, Custom training
Together AI Cloud platform for open-source models
Various
Open models, Fine-tuning
Anyscale Ray-based platform for AI applications
Enterprise
Distributed training, Serving
Groq Ultra-fast LPU inference engine for running open AI models
Free tier + Pay-per-use
Blazing fast speed, High throughput, Low latency, Open model support
OpenRouter Universal API to find and route to the best LLMs from various providers
Free tier + Pay-per-use
Multi-provider access, Unified API, Model comparison, Caching
RouteLLM Dynamic router for selecting best LLMs based on cost & performance
Open-source
Cost optimization, Multi-LLM routing
We welcome contributions! This toolkit grows stronger with community input.
Fork the repository Create a feature branch (git checkout -b feature/amazing-tool)Add your contribution (new tool, template, or tutorial)Submit a pull request
Quality over quantity - Focus on tools and resources that provide real valueProduction-ready - Include tools that work in real-world scenariosWell-documented - Provide clear descriptions and usage examplesUp-to-date - Ensure tools are actively maintained
Get weekly AI engineering insights, tool reviews, and exclusive demos and AI Projects delivered to your inbox:
📧 Subscribe to AI Engineering Newsletter →
Join 100,000+ engineers building better LLM applications
Built with ❤️ for the AI Engineering community
Star ⭐ this repo if you find it helpful!
