Which model for coding? Local LLMs for developers

A local model that writes, explains, and refactors your code — with nothing leaving your machine — is one of the most useful things you can self-host. But "which model should I run?" has no single answer: it depends on your hardware and what you want it to do. This guide gives you a durable way to choose, rather than a leaderboard that expires next month.

The four criteria that actually matter

Ignore the hype and judge a code model on these four axes. They change far more slowly than model names do.

1. Size versus your memory

This is the hard constraint. A model has to fit in your VRAM (or system RAM) to run at a usable speed. A code model that spills over into system RAM will crawl.

As a rough guide — the same rule from our complete Ollama guide:

Your memory	Comfortable code model
8 GB	a 7B model at q4
16 GB	a 13B–14B model at q4, or 7B at q8
24 GB+	30B+ models, or a smaller model with a big context window

Pair this with GGUF quantization tags to shrink a model into your budget. Getting this wrong is the number-one reason people conclude "local models are too slow" — they were simply running a model too big for their card.

2. Fill-in-the-middle (FIM) support

This is the criterion most beginners miss, and it's the difference between a chatbot and a real coding assistant.

Ordinary chat models only continue text at the end. Fill-in-the-middle models are trained to insert code between existing lines — exactly what inline autocomplete in your editor needs. If you want tab-completion inside a function, you need an FIM-capable model. Most dedicated code models support it; general chat models usually don't.

3. Context window

Code tasks eat context. Understanding a file means reading its imports; understanding a change means reading the surrounding module. A model with a small context window forgets the top of the file by the time it reaches the bottom.

For serious work, favour models that advertise a large context (32K tokens and up). Remember the trade-off from the Ollama guide: a bigger num_ctx costs memory, so there's real tension between criteria 1 and 3 on a small machine.

4. License

If you write code for work, the model's license matters as much as its skill. "Open weight" is not the same as "free to use commercially" — a point we cover in open-source vs open-weight. Check the license before you build a workflow around a model, especially inside a company.

The families worth knowing

Rather than chase version numbers that change monthly, learn the families — each has a consistent character across releases:

• Qwen Coder — a strong all-rounder family, released in a wide range of sizes, so there's usually one that fits your card. A sensible first choice.
• DeepSeek-Coder — well regarded for reasoning through larger, multi-file problems.
• Codestral (Mistral) — built specifically for code with FIM in mind; check its license terms for commercial use.
• Code Llama — the older, widely-supported baseline. Not the strongest today, but battle-tested and everywhere.

Whichever family you pick, look for the variant whose size matches criterion 1, and confirm FIM support if you want inline completion.

Putting it to work

Once you've chosen, pull it with Ollama and point your editor at it. The plumbing — connecting a local model to your IDE and giving it access to your files — is exactly what tools like editor extensions and MCP are for.

The payoff is the whole reason to go local: a coding assistant that reads your private repository, runs offline on a plane, and never sends a line of proprietary code to someone else's server. That's the case we make in why we're moving off cloud LLMs — and coding is where it pays off first.

Where to go next

• Not sure a model fits your card? Start with the memory table in the complete Ollama guide.
• Want to give the model tools and file access? See MCP for local AI.
• Working in a company? Read open-source vs open-weight before you commit to a model's license.