{"id":59765,"date":"2026-03-11T14:33:32","date_gmt":"2026-03-11T18:33:32","guid":{"rendered":"https:\/\/codesamplez.com\/?p=59765"},"modified":"2026-03-11T14:33:34","modified_gmt":"2026-03-11T18:33:34","slug":"local-ai-coding-agent","status":"publish","type":"post","link":"https:\/\/codesamplez.com\/productivity\/local-ai-coding-agent","title":{"rendered":"Local LLM for Coding: Free AI Coding Agent With Ollama + Claude"},"content":{"rendered":"\n

If you’ve been curious about running a local LLM for coding<\/strong> but felt overwhelmed by the sheer number of models, runtimes, and configuration options, this guide is for you. Not the “tab-complete a for loop” kind of local. I’m talking full agentic coding \u2014 an AI agent that reads your repo, plans changes across multiple files, runs shell commands, executes tests, and iterates on errors. All on your machine. Zero API costs. Zero code leaving your network.<\/p>\n\n\n\n

The stack we’re building: Ollama + Claude Code + an open-weight model.<\/strong> It takes about 15 minutes to get running.<\/p>\n\n\n\n

What is a local LLM?<\/strong>

A local LLM for coding is a large language model running entirely on your own hardware \u2014 laptop, workstation, or home server \u2014 that provides AI-powered code generation, refactoring, and agentic multi-file editing without sending a single line of code to the cloud. It matters because it gives you complete privacy, zero recurring costs, and AI coding assistance that works even without internet.<\/p> <\/div> <\/div>\n\n\n\n

\ud83d\udca1Pro Tip: New to agentic coding? Consider going over beginners guide to ai coding assistant setup<\/a> first.<\/p>\n\n\n\n

Why “Local LLM” Changes Everything for Agentic Coding<\/h2>\n\n\n\n

Before we set anything up, let me be specific about what changes when you move agentic coding off the cloud.<\/p>\n\n\n\n

Privacy becomes real, not theoretical.<\/strong> Agentic coding tools read everything<\/em> \u2014 your entire repo structure, environment files, config files, test output, shell history. With a cloud provider, all of that context ships to someone else’s servers. After Samsung’s engineers accidentally uploaded confidential source code to ChatGPT back in 2023, this stopped being a paranoia issue and became a policy issue at many companies. Running locally means your proprietary business logic stays on your machine. Period.<\/p>\n\n\n\n

Cost drops to zero.<\/strong> Claude Code on Anthropic’s API is powerful but expensive \u2014 heavy agentic sessions with Opus can burn through $5-15\/hour easily. Locally? Your electricity bill goes up a few dollars, … for the entire month! The math is absurdly good once you have the hardware.<\/p>\n\n\n\n

Availability becomes unconditional.<\/strong> Flights, coffee shops with garbage Wi-Fi, AWS outages that take down half the internet \u2014 none of it matters. Your AI coding agent is always on.<\/p>\n\n\n\n

The quality tradeoff is shrinking fast.<\/strong> In 2023, local models were a joke for agentic work. In 2026, models like Qwen 3.5<\/a> and GLM-4.7-Flash<\/a> genuinely handle multi-file edits, tool calling, and long-context planning. They won’t match Opus 4.6 on the hardest tasks, but for building features, fixing bugs, and scaffolding projects? They’re shockingly competent.<\/p>\n\n\n\n

Setting Expectations<\/h3>\n\n\n\n

Here’s what nobody warns you about: a local model won’t feel exactly like Claude Opus or GPT-5 on day one.<\/strong> And that’s okay.<\/p>\n\n\n\n

Local models are best at:<\/strong> scaffolding new features, generating boilerplate, writing tests, fixing bugs with clear error messages, and iterating on code with feedback loops. <\/p>\n\n\n\n

They’re weaker at:<\/strong> massive architectural decisions across 20+ files, highly nuanced refactoring of complex legacy code, and tasks requiring enormous context windows. <\/p>\n\n\n\n

The sweet spot for local agentic coding is tasks where you can describe what you want, let the agent take a first pass, review, and iterate. That covers probably 80% of daily development work.<\/p>\n\n\n\n

\n\n\n\n

Choose the Right Model<\/h2>\n\n\n\n

This is where most people get stuck. Hundreds of models on Hugging Face<\/a>, new ones every week, every Reddit thread recommending something different. Let me cut through the noise.<\/p>\n\n\n\n

For agentic coding, your model needs three things:<\/p>\n\n\n\n

long context<\/strong> (~64K tokens minimum),<\/p>\n\n\n\n

tool calling support<\/strong> (so the agent can execute commands, read files, run tests), and <\/p>\n\n\n\n

strong instruction following<\/strong> (so it doesn’t go off the rails mid-task). Not every model delivers on all three.<\/p>\n\n\n\n

The Models Worth Running Right Now<\/h3>\n\n\n\n

Qwen 3.5 35B-A3B<\/a><\/strong> \u2014 My current daily driver. Released February 2026, this is a 35B parameter MoE model that only activates 3B parameters per token. That means it’s fast AND smart. It supports 256K context natively, has strong agentic capabilities, and its tool calling works reliably with Claude Code. Benchmarks back this up \u2014 the 35B-A3B model surpasses much larger predecessors like Qwen3-235B, as well as proprietary models like GPT-5 mini and Claude Sonnet 4.5 in categories including knowledge and visual reasoning. Runs comfortably on 32GB unified memory or a 24GB GPU. Apache 2.0 license, fully open for commercial use.<\/p>\n\n\n\n

GLM-4.7-Flash<\/a><\/strong> \u2014 The best all-around model for 24GB VRAM setups. GLM-4.7-Flash dominates with a 30.1 Intelligence Index and won agentic coding challenges in recent independent testing. It handles planning, multi-step tool use, and code generation across multiple files with real consistency. If Qwen 3.5 35B doesn’t click for you, GLM-4.7-Flash is a rock-solid alternative.<\/p>\n\n\n\n

Qwen3-Coder 30B-A3B<\/a><\/strong> \u2014 Purpose-built for coding agents. It offers 30B total parameters with only 3.3B activated, with exceptional agentic capabilities for real-world software engineering tasks and native support for 256K tokens. Trained specifically on agentic coding workflows through reinforcement learning on SWE-Bench. If your work is purely code (no general reasoning, no docs), this specialist might outperform the generalists above.<\/p>\n\n\n\n

GPT-OSS 20B<\/a><\/strong> \u2014 OpenAI’s open-weight model. Strong reasoning and tool calling capabilities. A solid option at ~13GB, it fits on more modest hardware while still handling agentic workflows.<\/p>\n\n\n\n

Match Your Hardware to a Local LLM Model<\/h3>\n\n\n\n

No CPU-only options here \u2014 agentic coding needs responsive inference, and CPU-only speeds are too slow for the multi-turn, tool-calling loops that Claude Code runs. You need a GPU or Apple Silicon unified memory.<\/p>\n\n\n\n

Here’s the straightforward mapping:<\/p>\n\n\n\n

Your Hardware<\/th>Available Memory<\/th>Model to Run<\/th>Download Size<\/th>What to Expect<\/th><\/tr><\/thead>
Mac M1\/M2\/M3 with 16GB unified<\/td>~14GB usable<\/td>GPT-OSS 20B (Q4)<\/td>~13GB<\/td>Workable but tight. Close other apps.<\/td><\/tr>
Mac M2\/M3\/M4 with 32GB unified<\/td>~28GB usable<\/td>Qwen 3.5 35B-A3B (Q4)<\/td>~22GB<\/td>Smooth daily driver. My recommended starting point.<\/td><\/tr>
Mac M2\/M3\/M4 with 48-64GB unified<\/td>~44-58GB usable<\/td>Qwen 3.5 35B-A3B (Q8) or 122B-A10B (Q4)<\/td>~35GB \/ ~70GB<\/td>Premium experience, larger context windows.<\/td><\/tr>
NVIDIA GPU with 16GB VRAM<\/td>16GB<\/td>GPT-OSS 20B (Q4)<\/td>~13GB<\/td>Solid agentic coding for most tasks.<\/td><\/tr>
NVIDIA GPU with 24GB VRAM<\/td>24GB<\/td>GLM-4.7-Flash (Q4) or Qwen 3.5 35B-A3B (Q4)<\/td>~18-22GB<\/td>Excellent. This is the NVIDIA sweet spot.<\/td><\/tr>
NVIDIA GPU with 48GB+ VRAM<\/td>48GB+<\/td>Qwen 3.5 122B-A10B (Q4)<\/td>~70GB<\/td>Near-cloud quality, serious agentic power.<\/td><\/tr><\/tbody><\/table><\/figure>\n\n\n\n

<\/p>\n\n\n\n

The quick rule of thumb:<\/strong> Take your available memory, subtract 2-4GB for overhead, and that’s your model size budget at Q4 quantization.<\/p>\n\n\n\n

\n

What’s Q4 quantization?<\/strong> Quantization compresses model weights from 16-bit floats to smaller integers, dramatically reducing memory usage. Q4_K_M is the sweet spot \u2014 it cuts memory by ~75% with minimal quality loss. Below Q3, quality degrades noticeably. Ollama handles quantization automatically when you pull a model, so you don’t need to worry about the details.<\/p>\n<\/blockquote>\n\n\n\n

\n\n\n\n

Setup Guide: Ollama + Claude Code<\/h2>\n\n\n\n

One path. No forks. Let’s get a working local AI coding agent in 15 minutes.<\/p>\n\n\n\n

Step 1: Install Ollama<\/h3>\n\n\n\n

Ollama<\/a> handles model downloads, GPU detection, quantization, and serves an API that Claude Code talks to. It’s the foundation.<\/p>\n\n\n\n