What is local AI? Discover the different ways to run AI models
Local AI is simple in principle: you run the model on your own machine, at home, rather than shipping your data to a cloud server controlled by someone else. No API to pay per token, no leaked prompts, no silent model updates overnight. You download an open-weight model, fire it up with Ollama or LM Studio and talk to it like any other locally installed tool.
The question everyone asks next is: "but is my machine powerful enough?" Let's be honest: a standard PC, without plenty of RAM, a recent CPU and a dedicated GPU, will struggle to run even the smallest AI models — and your hardware still needs to be compatible with local-AI toolchains (GPU drivers, ROCm, CUDA…). And if your current machine isn't enough, there are now four paths to AI sovereignty — from modest to ambitious.
No matter which path you pick, you keep 100% control over the tools, the models and the data. No API keys, no opaque terms of service, no forced updates. It's the exact opposite of the "we lend you an AI" model run by cloud giants.
Option 1: run it on your existing laptop or PC
This is the on-ramp. If you have a recent laptop (Intel Core Ultra, AMD Ryzen 7000/9000 series) with at least 16 GB of RAM and ideally a discrete GPU, you can already run models locally.
A few realistic ballpark figures for 2026:
| Your machine | Models it runs comfortably |
|---|---|
| Ultrabook with Ryzen 7 / Core Ultra 7 (16 GB, no dGPU) | Llama 3.2 3B, Phi-4, Qwen 3 4B — fast chat, summaries, light RAG |
| Gaming laptop with RTX 4070 (12 GB VRAM) | Llama 3.1 8B, Mistral Nemo, Qwen 3 8B — general use, embeddings |
| Desktop PC with RTX 5070 Ti (16 GB VRAM) | Qwen 3 14B, Gemma 4 e4B, DeepSeek-R1-Distill 14B — code, analysis |
| Workstation with RTX 5090 (32 GB VRAM) | Qwen 3 32B, Llama 4 Scout, DeepSeek-V3 (quantized) — frontier models |
On these PCs, llama.cpp and Ollama are the everyday toolchains. They lean on CUDA (NVIDIA), ROCm (AMD), and Vulkan for acceleration, with mature support for running large models even when they don't fit entirely in GPU VRAM.
The downside? Your laptop becomes your infrastructure. You can't easily turn it off, update it or use it elsewhere without interrupting your AI services.
The external GPU enclosure trick (USB4 / Thunderbolt)
If your laptop has a USB4 or Thunderbolt 4/5 port, you can plug in an external GPU enclosure containing a desktop graphics card (RTX 4090, RTX 5090, etc.).
It's a stopgap option but with three serious limitations worth knowing:
- Reduced performance. USB4 bandwidth (≈40 Gbps) is far below a PCIe x16 slot (≈128 Gbps). You typically lose 20 to 40% of performance compared to a GPU plugged directly into the motherboard.
- Separate power supply. The external GPU enclosure needs its own PSU (often 650W to 1000W). Forget the corner café to work from.
- It's heavy and bulky. An external GPU enclosure weighs between 4 and 8 kg plus the card. It's not really a "laptop" anymore at this point — it's a semi-fixed setup.
Bottom line: it's a good choice when you keep your laptop parked on your desk, have a power outlet nearby, and want a more affordable solution.
Option 2: buy a tower or specialized PC with a good GPU
When the laptop isn't enough, the natural next step is investing in a proper desktop machine that you'll leave running at home or in the office. This is where you explicitly pick your components.
GPUs worth considering for local AI in 2026
Here are the options that offer the best VRAM-per-dollar ratio, by category:
| GPU | VRAM | Best for | Price range |
|---|---|---|---|
| RTX 5070 Ti | 16 GB | All-rounder, 14B-30B quantized models | $750-950 |
| RTX 5080 | 16 GB | Similar to the 5070 Ti with more system RAM | $950-1,200 |
| RTX 5090 | 32 GB | Comfortable 30B+ models, Stable Diffusion XL | $2,000-2,500 |
| RX 9070 XT (AMD) | 16 GB | GNU/Linux alternative with ROCm 6.5+ | $650-850 |
| RX 7900 XTX (AMD) | 24 GB | Excellent VRAM/price ratio on GNU/Linux | $800-1,100 |
If you want a single "VRAM-rich" workstation without juggling two cards, the Framework Desktop with AMD Ryzen AI Max 395 supports up to 128 GB of unified memory (more on that in Option 3), and dual-GPU workstations can pool 48 to 96 GB of VRAM on a single motherboard. There are also purpose-built NVIDIA RTX PRO / Ada Generation cards with 48 GB each for those with a deeper budget.
On AMD/nVIDIA under GNU/Linux: ROCm (the AMD stack) and CUDA (NVIDIA) are both mature, but NVIDIA remains the reference for third-party tools. ROCm on RDNA 4 (RX 9070) has nevertheless become solid in 2026 for Ollama, ComfyUI and llama.cpp.
RAM and storage not to be overlooked
A good GPU isn't enough. To avoid bottlenecks:
- 64 to 128 GB of system RAM minimum (DDR5), for CPU offload of larger models.
- 2 TB of NVMe at minimum — a Q4-quantized 70B model is about 40 GB and you'll want to store several.
- 1000W+ PSU if you grab an RTX 5090.
The total cost of a well-equipped tower runs between $2,000 and $4,000 depending on your choices.
Option 3: a dedicated machine built for local AI
Rather than building your own tower, some manufacturers now offer machines specifically designed for running AI models at home. The most polished example in 2026 is the Framework Desktop, a modular mini-PC configurable with up to 128 GB of RAM (Ryzen AI Max 395, unified memory).
Why this kind of machine is interesting:
- Compact form factor (4 liters, roughly the size of a thick book), quiet, sleek.
- Unified CPU/GPU RAM: you dynamically allocate the amount of VRAM you need.
- Built to run 24/7: efficient PSU, optimized cooling, low noise.
- 100% at home, no cloud component, no subscription.
It's a great option if you want an AI node that stays on around the clock without burning through your electricity bill or hearing a server fan all night. Typical consumption: 60-150W under load or roughly $30 to $60 per year in electricity if you leave it on 24/7.
Other alternatives in the same niche: Lenovo ThinkStation workstations, Beelink / MinisForum mini-PCs (SER7, UM890 Pro), or modded Synology/QNAP NAS units.
On a traditional PC, VRAM and system RAM are separate: if your model exceeds VRAM, it's "offloaded" to RAM with a big speed hit. With unified memory (Ryzen AI Max on the Framework Desktop or the AMD Strix Halo platform), **it's the same physical memory** shared between CPU and GPU. Result: a Q4 70B model (≈40 GB) running nearly as fast on 64 GB of unified RAM as on 48 GB of dedicated VRAM, in a ridiculously small footprint.
Option 4: a VPS or private network to access it anywhere
Once you have your machine at home (option 2 or 3), the logical question is: "how do I reach it from my phone, my office or on the go?"
Self-hosted behind your box, reachable via VPN
Tools like ZeroTier, Tailscale, or Netbird let you create a mesh VPN that connects all your devices (laptop, phone, home server, work machine) as if they were on the same LAN. A few lines of config, no ports open on your router, and your Ollama becomes reachable from anywhere.
Benefits:
- No intermediary server: traffic flows directly between your devices.
- End-to-end encryption by default.
- Free for personal use (ZeroTier, Tailscale freemium, Netbird open source).
- Your local IPs are preserved: your Ollama server, your RAG instance, your Nextcloud — all reachable on demand.
Rent a VPS from a sovereign provider
If you don't have the space or the fiber connection to host at home, you can rent a machine from a provider based in a country that respects your regulations. In Europe, this means providers that comply with GDPR and store data in EU data centers.
A few criteria to pick well:
| Criterion | Why it matters |
|---|---|
| Datacenter location | Spain, France, Germany, the Netherlands — avoid the US for sensitive data |
| Independent provider | No Big Tech: players like Hetzner (DE/FI), OVH (FR), Scaleway (FR), Ikoula (FR) |
| Encryption policy | Who has access to the disks? Encryption at rest? |
| IPs and network | Clean IPs, not blacklisted, acceptable latency from your home |
| Legal terms | GDPR enforcement, no US PATRIOT Act applicable |
Once the VPS is rented, you install your Ollama, your models, your tools exactly as if it were at home. You control the OS (Debian, Fedora, NixOS, whatever you want), you pick the models, you stay in charge of updates. It's your machine — just not in your living room.
A VPS at Hetzner (Germany) or Scaleway (France) isn't more sovereign than a mini-PC in your living room. Sovereignty is about who controls the data and the tools, not where the physical hardware sits. Self-hosting at home is the maximum ideal; a VPS at a trusted provider is a very acceptable approximation.
The common thread: control
Whichever option you choose, the result is the same: you keep end-to-end control.
- Your data never leaves a machine you control.
- Your models are downloaded by hash, verifiable, open-weight.
- Your tools (Ollama, Open WebUI, ComfyUI, your MCP agents) run without depending on a cloud platform.
- Your API keys, system prompts and business contexts stay confidential.
The cloud is convenient for fast experimentation. Local sovereignty is convenient for real — for privacy, predictable costs, and no longer depending on a product roadmap you don't control.
Pick your path. Hardware is no longer the bottleneck it was in 2023: in 2026, running local AI is a choice, not a sacrifice.