Obsidian + local LLM: your notes as the model's external memory

What is Obsidian?

Obsidian is a note-taking app. Most note apps lock your writing inside a private database; Obsidian stores every note as a plain text file, formatted with Markdown, in a folder you own. You can open those files in any editor, back them up with git and move them freely.

Why pair it with a local model?

Because a language model reads text natively. When your notes are plain files, a local model can search them, summarize them and answer questions about them — all on your own machine, with nothing leaving your computer.

That combination is the subject of this guide. Andrej Karpathy described it in one line:

"Obsidian is the IDE, the LLM is the programmer, the wiki is the codebase."

In plain words: Obsidian is where you write and keep your notes, the model is your assistant and your notes are the material it works from. Here's how far you can take that idea on hardware you own — what works locally today and where a cloud model still wins.

Why Markdown is the format a model digests best

It's a lucky coincidence: the simple format that note apps have used for twenty years is also the one a language model understands best.

• It's plain text. The model reads directly what is written, with no conversion and no special software. A note is just a text file: if you type "buy bread" in Obsidian, the model receives exactly "buy bread".
• Headings show the outline. A ## heading marks the start of a new idea. The model can split the document into sections easily, without having to guess where they begin and end.
• The links are written by you. When you write [[pancake-recipe]] in a note, you are telling the model those two ideas are related. It knows immediately, instead of having to guess.
• Useful info sits at the top of the note. The date, tag or project are written at the start of the file. Filtering notes becomes simple: just look at the top of the file, no separate index needed.
• You can see every change. If the model rewrites a note, you can compare before and after to check exactly what changed. In a classic database, that's impossible.

The 3 progressive steps

Karpathy's idea describes an agent that maintains your wiki on its own. That's the end goal, not the starting point. You can begin with a simpler step that already works very well locally.

Step 1 — Finding a note in your own words

The goal: ask a question the way you'd ask a friend and get the right note back, even if you forgot its title or the words you used.

Example: you ask "when was my last dentist visit?" without remembering you have a note about it. Semantic search still finds the right note.

This runs entirely on your machine and it's fast. Measured on a real vault of 383 notes / 2.4 MB of Markdown, with nomic-embed-text on Ollama:

Eighteen seconds to make three years of notes searchable. After that, every edit is re-indexed automatically: you only pay that cost once.

But it's not magic. In the same test, asking "which local model should I pick for my RAM?" returned a note about the RAM budget of an embedded firmware — the subject was close, the answer useless. Search matches subjects, not truths. When your vault doesn't contain the answer, it still hands you its three closest guesses without telling you it failed. Treat the results as leads to read, not as answers.

Step 2 — Reading, summarising and drafting

Once you've found the right note, hand it to a local model to work on: summarise this cluster of notes, draft a synthesis page, look for contradictions between two documents.

Example: you give it ten meeting notes and it produces a "decisions made in 2026" page you just need to review.

Model choice matters here. Match it to the job, not to the benchmark leaderboard:

• 8–16 GB RAM — a 7–8B model (Qwen 3 8B, Gemma 12B quantised). Good at summarising a passage you hand it. Don't expect it to hold your whole vault's structure in mind.
• 24–32 GB — a 24–32B model. The sweet spot for drafting and reorganising notes.
• 48 GB+ — a 70B-class model or the larger Qwen/Gemma variants. Now the summaries stop feeling like summaries.

For note work specifically, prioritise context length over model size. A model that reads nine notes at once is more useful than a bigger model that reads only one. Check that setting in Ollama before blaming the model's intelligence.

Step 3 — An agent that maintains the wiki on its own

This is the full step: you give a source to the model, it decides which existing notes it affects, rewrites each of them, creates the missing pages and fixes the links. A single request can touch a dozen files.

This is the layer where local models still fall short. The problem isn't writing quality, it's the number of steps: each step depends on the last and no one is there to catch a wrong turn along the way. A model that's 95% reliable per step is 54% reliable over twelve steps.

To pull this off today, you need a cloud model with access to your files. The trade-off is real: an agent that rewrites your vault has read everything in it. That's exactly the reasoning in local first, cloud as a fallback — the answer isn't "never use the cloud", it's knowing which notes you'd rather it never saw.

A workable middle path: let a local model handle the high-volume work (ingestion and first-pass summaries) and keep the cloud model for restructuring passes you actually review.

Configuring Obsidian with Ollama

1. The two models you need

You need to download two models, because they do two different jobs: one finds your notes, the other writes. Here are the two commands:

# The search model — small and fast, it finds the relevant notes
ollama pull nomic-embed-text

# The language model — pick per the RAM table above
ollama pull qwen3:8b

Two distinct models, two distinct jobs. The search model turns text into vectors to compare notes against each other; it is never asked to write anything. It's 270 MB and runs on a CPU. Don't try to do the search with the language model — most refuse outright and the ones that comply are worse at it.

Verify that search works before going further:

curl -s http://localhost:11434/api/embed \
  -d '{"model":"nomic-embed-text","input":"hello"}' | head -c 80

You want a JSON array of floats back. If you get This server does not support embeddings, your Ollama server was started with embeddings disabled — restart it plainly with ollama serve.

2. Wire it into Obsidian

Two community plugins do the job and they answer different needs:

• Copilot for Obsidian — a chat panel beside your notes, with the vault as context. Point it at Ollama: set the provider to a custom OpenAI-compatible endpoint at http://localhost:11434/v1, then name your language model and nomic-embed-text as the search model. This is the one to install first.
• Smart Connections — no chat, just a "related notes" panel that surfaces links you never made. Lower effort and honestly the one that changes daily habits more.

Both index locally and both talk to Ollama without an API key. Install from Settings → Community plugins and expect the first full index to take a minute or two on a large vault.

3. Give the vault a shape the model can navigate

This matters more than the plugin choice. A model reading a flat pile of 400 notes performs about as well as you would.

The structure that holds up in practice has three layers, mirroring the one Karpathy describes:

sources/     raw captured material — never edited by hand
notes/       your own writing, one idea per file
index.md     the entry point: what lives where and why

Then two rules that do the real work:

Write the index by hand. A model can read 400 files, but it costs it (and you) far more than reading one file that says where to look. This is the single highest-return thing in the whole setup — an entry-point note listing what each area contains, maintained by you.

Write down the why, never the what. If a fact is recoverable from the source material, don't copy it into a note; the source is the truth and it will drift. Constraints, rationale, what you rejected and why, the thing that bit you last March — none of that is derivable from anywhere and it's what future-you will actually search for.

The counterintuitive part: less context, better answers

Here's the finding that surprised me most, from running this setup daily.

The instinct with a knowledge base is to load it: inject a summary of recent activity at every session start, keep a running recap, front-load everything the model might need. It feels like helping.

It isn't. On the vault measured above, an automatic chronological recap was being injected into every single session — a few thousand tokens of "here's what happened lately," every time. Removing it entirely made the setup better and not by a small margin. Two effects compounded:

1. The obvious one. Thousands of tokens per session, freed for actual work.
2. The one I didn't predict. With the automatic recap gone, the reflex changed — to reading the index and the relevant note, on demand. The passive recap had been a crutch that answered "where were we?" adequately enough that nobody reached for the source built to answer it properly.

Deleting a mediocre solution beat adding a good one. You don't have to train the better habit if you stop bypassing it.

The general rule this points at: retrieved-on-demand beats always-injected. Context spent before you know what the question is, is context spent on a guess. A vault with a good index is a system for spending it after.

What this actually buys you

Strip away the tooling and the change is this: your notes stop being a place you file things and become a place you ask things.

Not because the model got smart about your life — it didn't and the retrieval will hand you a firmware note when you ask about RAM. But because finding stopped depending on remembering. A note you wrote in 2024 and forgot entirely surfaces because it's about what you're asking, not because you recalled its title.

The parts that run locally — search, summarising, drafting — run on hardware you own, over files you own, with no subscription and nothing leaving the machine. The part that doesn't yet run locally is the fully autonomous librarian and that gap will close. Meanwhile the vault you build for a cloud model today is the same vault a local model will read tomorrow, because it's just Markdown in a folder. That's the quiet advantage of a boring format: it outlives the tool you chose this year.