Which model for coding? Local LLMs for developers
A local model that writes, explains and refactors your code — with nothing leaving your machine — is one of the most useful things you can self-host. But "which model should I run?" has no single answer: it depends on your hardware and what you want it to do. This guide gives you a durable way to choose, rather than a leaderboard that expires next month.
The four criteria that actually matter
Ignore the hype and judge a code model on these four axes. They change far more slowly than model names do.
1. Size versus your memory
This is the hard constraint. A model has to fit in your VRAM (or system RAM) to run at a usable speed. A code model that spills over into system RAM will crawl.
As a rough guide — the same rule from our complete Ollama guide:
| Your memory | Comfortable code model |
|---|---|
| 8 GB | a 7B model at q4 |
| 16 GB | a 13B–14B model at q4 or 7B at q8 |
| 24 GB+ | 30B+ models or a smaller model with a big context window |
Pair this with GGUF quantization tags to shrink a model into your budget. Getting this wrong is the number-one reason people conclude "local models are too slow" — they were simply running a model too big for their card.
2. Fill-in-the-middle (FIM) support
This is the criterion most beginners miss and it's the difference between a chatbot and a real coding assistant.
Ordinary chat models only continue text at the end. Fill-in-the-middle models are trained to insert code between existing lines — exactly what inline autocomplete in your editor needs. If you want tab-completion inside a function, you need an FIM-capable model. Most dedicated code models support it; general chat models usually don't.
A good setup often uses two models: a larger one for "explain this / write this function" chat and a small fast FIM model for real-time inline completion. They don't have to be the same model.
3. Context window
Code tasks eat context. Understanding a file means reading its imports; understanding a change means reading the surrounding module. A model with a small context window forgets the top of the file by the time it reaches the bottom.
For serious work, favour models that advertise a large context (32K tokens and up). Remember the trade-off from the Ollama guide: a bigger num_ctx costs memory, so there's real tension between criteria 1 and 3 on a small machine.
4. License
If you write code for work, the model's license matters as much as its skill. "Open weight" is not the same as "free to use commercially" — a point we cover in open-source vs open-weight. Check the license before you build a workflow around a model, especially inside a company.
The families worth knowing
Rather than chase version numbers that change monthly, learn the families — each has a consistent character across releases. Some are dedicated code models; others are strong general-purpose families that happen to code well.
Dedicated code models:
- Qwen Coder (Alibaba) — a strong all-rounder family, released in a wide range of sizes, so there's usually one that fits your card. A sensible first choice.
- DeepSeek-Coder (DeepSeek) — well regarded for reasoning through larger, multi-file problems.
- Codestral (Mistral) — built specifically for code with FIM in mind; check its license terms for commercial use.
- Code Llama (Meta) — the older, widely-supported baseline. Not the strongest today, but battle-tested and everywhere.
General-purpose families that code well:
- Gemma (Google) — an open family in small, laptop-friendly sizes. Not a dedicated code model, but a capable generalist for explaining and writing code — a good pick when you want one model for both chat and light coding rather than a code specialist.
- GLM (Zhipu AI) — a Chinese general-purpose family with solid coding ability; larger variants handle multi-step tasks well.
- Yi (01.AI) — another Chinese generalist family, notable for large-context variants that suit reading across big files.
Whichever family you pick, look for the variant whose size matches criterion 1 and — if you want inline completion — confirm FIM support, which the dedicated code models are more likely to offer than the generalists.
Putting it to work
Once you've chosen, pull the model with Ollama and point your editor at it — editor extensions and MCP handle the plumbing between a local model, your IDE and your files. The payoff is the whole reason to go local: a coding assistant that reads your private repository, runs offline on a plane and never sends a line of proprietary code to someone else's server.