Ollama: the complete guide (beyond the 5-minute install)

If you already have Ollama installed and you've chatted with your first model. That's the easy part — and honestly, it's where most tutorials stop.

But running ollama run mistral once is not the same as using Ollama. The tool quietly does a lot more: it manages a local library of models, lets you reshape any model's personality and settings without retraining anything and — the part most people miss — it runs a full HTTP server on your machine that your own scripts and apps can talk to.

This guide picks up where the install leaves off. No cloud, no API keys, no per-token billing. Everything below runs on your hardware and stays there.

Ollama is a server, not just a chat box

The single most useful thing to understand about Ollama: when it's running, there's a background server listening on your machine — by default at http://localhost:11434.

The ollama run command you typed is just a client talking to that server. Your own code can talk to the exact same server. That's the whole reason Ollama is more than a toy: it turns your computer into a private, offline AI backend.

You can start the server explicitly with:

ollama serve

On most installs (the desktop app on macOS/Windows or the systemd service on GNU/Linux) the server is already running in the background, so you rarely call serve by hand. But it's good to know it's there — every command and every API call below depends on it.

Managing your model library

Models are downloaded once and cached locally. Over time you'll accumulate several and you'll want to keep the collection under control. Here are the commands that matter.

Download a model

ollama pull [model]

This fetches the model from the registry and stores it locally. After this, the model works fully offline — no internet required to run it.

See what you have

ollama ls

(ollama list does the same thing.) This shows every model on disk, its size and when you last modified it. Useful when you're wondering where your disk space went — these files are large.

See what's running right now

ollama ps

Models get loaded into RAM/VRAM when you use them and stay resident for a few minutes in case you ask again. ps shows what's currently loaded and how much memory it's eating. If your machine feels sluggish, this tells you whether a model is still parked in memory.

Stop a running model

ollama stop [model]

Unloads the model from memory immediately instead of waiting for it to time out. Handy when you want your RAM/VRAM back now.

Delete a model

ollama rm [model]

Removes it from disk entirely. You'll re-download it if you want it again.

Inspect a model

ollama show [model]

Prints the model's details — its parameters, the template it uses, its license. This is the quickest way to peek under the hood of any model you've pulled.

Customizing a model with a Modelfile

Here's where Ollama gets genuinely powerful — and where most people never venture.

A Modelfile is a small text recipe that takes an existing model and tweaks it: giving it a permanent personality, locking in settings like creativity level or pinning a larger context window. You're not retraining anything. You're wrapping a base model in a configuration that travels with it.

Create a plain text file named Modelfile (no extension):

# Start from an existing model
FROM mistral

# Make it less random and more focused (0 = deterministic, 1 = creative)
PARAMETER temperature 0.3

# Give it a bigger memory of the conversation
PARAMETER num_ctx 8192

# Define its permanent personality and rules
SYSTEM """
You are a concise technical writing assistant.
Answer in plain English. Prefer short sentences.
Never apologize. If you don't know, say so directly.
"""

Then build your custom model from it:

ollama create writer -f Modelfile

Now you have a new model called writer that behaves exactly that way every time:

ollama run writer

The instructions you'll actually use

Parameters worth knowing

One detail that trips people up: the Modelfile is not case sensitive and instructions can appear in any order. The convention is uppercase keywords (FROM, SYSTEM) purely for readability.

Getting the most out of your model

Installing a model gets you a generic setup. Tuning it gets you one that fits your machine and your task. There are three knobs that matter — and the good news is you've already met the controls: they're the same PARAMETER lines from the Modelfile above, plus one decision you make before you even download.

Think of it in this order: pick the right model → make it run smoothly → make it answer well → make it behave the way you want.

Pick the right model first

No amount of tuning fixes a model that's too big for your hardware. Before anything else, match the model to your machine.

The variable that decides everything is how much memory the model needs versus how much you have (VRAM if you have a GPU, otherwise system RAM). A model that fits in your GPU's VRAM runs fast; one that spills over into system RAM runs much slower; one that doesn't fit at all won't load.

Two levers control a model's footprint:

• Parameter count — the 7b, 13b, 70b in a model's name (billions of parameters). Bigger is generally smarter but heavier.
• Quantization — how tightly the model's numbers are compressed, shown as tags like q4_0, q4_K_M, q8_0. Lower numbers = smaller and faster, at a small cost in quality.

A practical rule of thumb:

For most people on a typical laptop or gaming PC, a 7B model at q4 (the default Ollama pulls for many models) is the sweet spot: fast, capable and it fits. Our Hardware guide goes deeper on matching models to your exact setup.

Make it run smoothly (speed & memory)

Once a model fits, these settings squeeze out speed and keep memory in check:

• num_ctx — the biggest memory lever. The context window grows memory use fast. If a model is borderline on your hardware, lowering num_ctx (say from 8192 to 4096) is often what makes it fit and run smoothly. Only raise it when you genuinely need the model to remember long documents.
• num_predict — cap the reply length. Limiting how many tokens the model can generate stops it from rambling and keeps responses snappy.
• OLLAMA_KEEP_ALIVE — avoid reload lag. By default a model unloads from memory after a few minutes idle, so your next prompt pays a reload delay. If you're using a model on and off all day, set this longer (e.g. 30m) to keep it warm. Working with several models on tight memory? Set it short (or 0) so each unloads promptly and frees room for the next.
• Let the GPU do the work. Ollama offloads to your GPU automatically when it can. Run ollama ps while a model is loaded — it tells you whether the model sits in GPU memory (fast) or has spilled to CPU (slow). If it spilled, that's your signal to drop to a smaller model or a lower quantization.

A measurable starting point: run a prompt, watch the tokens-per-second Ollama can report, then change one setting and compare. Tuning blind is guesswork; tuning one knob at a time is engineering.

Make it answer well (quality & consistency)

These settings shape what the model says, not how fast. They're the difference between a model that hallucinates wildly and one you can rely on. Set them as PARAMETER lines in a Modelfile so they stick:

Two combinations cover most real needs:

# A precise, repeatable assistant — coding, analysis, factual Q&A
PARAMETER temperature 0.2
PARAMETER top_p 0.5

# A creative writing partner — varied, surprising output
PARAMETER temperature 0.9
PARAMETER top_p 0.95

And when you're testing — comparing two prompts or two settings — pin the randomness so the only thing changing is what you meant to change:

PARAMETER seed 42
PARAMETER temperature 0

With a fixed seed and temperature 0, the same prompt gives the same answer every time. That's how you tell whether your prompt improved, not just whether you got lucky.

Make it behave the way you want (persona & rules)

Speed and sampling settings shape the form of answers. To shape their content and tone — to make a model consistently act as a code reviewer, a translator, a terse assistant — use the SYSTEM instruction. This is the single highest-leverage tuning you can do and it costs nothing in performance.

A vague system prompt produces a vague assistant. Be specific about role, format and boundaries:

FROM mistral

SYSTEM """
You are a senior Python code reviewer.
For each snippet, reply with exactly three sections:
  1. Bugs — concrete problems or "none".
  2. Risks — things that could break later.
  3. One suggestion — the single highest-impact improvement.
Be blunt. No preamble, no praise. If the code is fine, say so in one line.
"""

You can go a step further and show the model the behavior you want using MESSAGE, which seeds an example exchange into its memory:

MESSAGE user "def add(a,b): return a-b"
MESSAGE assistant "1. Bugs — subtraction where addition is intended. 2. Risks — silent wrong results, no type checks. 3. Suggestion — return a + b and add type hints."

Now the model has a concrete template to imitate. Combined, SYSTEM + a couple of MESSAGE examples turn a generic model into a specialist — no fine-tuning, no GPU hours, just a text file.

Calling Ollama from your own code (the REST API)

This is the feature that turns Ollama from a chat tool into a building block. While it's running, the server exposes an HTTP API at http://localhost:11434. Any language that can make an HTTP request can drive a local LLM through it.

No API key. No rate limit. No data leaving your machine.

Generate a single response — /api/generate

The simplest endpoint. Send a prompt, get a completion.

curl http://localhost:11434/api/generate -d '{
  "model": "mistral",
  "prompt": "Explain what a context window is in one sentence.",
  "stream": false
}'

Setting "stream": false makes Ollama return one complete JSON object. Leave it out (or set true) and the server streams the answer back token by token — which is how chat UIs show text appearing live.

Have a conversation — /api/chat

For multi-turn chat, /api/chat takes a list of messages with roles, just like the major cloud APIs:

curl http://localhost:11434/api/chat -d '{
  "model": "mistral",
  "messages": [
    { "role": "system", "content": "You are a helpful assistant." },
    { "role": "user", "content": "What is local AI?" }
  ],
  "stream": false
}'

Because the message format mirrors the OpenAI-style schema, a lot of existing tools and libraries can be pointed at Ollama with little more than a URL change.

Other endpoints you'll meet

That last one, /api/embed, is quietly important: it's what powers private document search (RAG) without ever sending your files to a cloud service.

Useful environment variables

Ollama is configured through environment variables. A few are worth knowing:

How you set these depends on your OS and whether Ollama runs as an app or a service. Run ollama serve --help to see the variables Ollama currently recognizes on your version.

You now have the full picture: Ollama is a local model manager, a customization tool and a private AI server rolled into one.

Everything you've set up here runs entirely on your hardware. No subscription, no cloud, no one watching. That's the point.