Local first, cloud as a fallback: a pragmatic hybrid setup
We make no secret of where we stand: local first, always. But being honest about local AI means admitting there are moments a 7B model on your laptop can't handle a job a 400B cloud model does effortlessly. The mistake is to conclude from those moments that you should live in the cloud. You shouldn't. You should make local the default and treat the cloud as a fallback you reach for deliberately — not a habit you drift into.
This is the hybrid setup we actually recommend: local for everything, cloud for the rare exception.
The wrong kind of hybrid
Most "hybrid" advice online means cloud first, local for toys. Every prompt goes to a provider and the local model is a novelty you run on weekends. That is not hybrid — it's just cloud with extra steps. Your data still leaves, your bill still grows, your workflow still breaks when the connection does.
A sovereign hybrid inverts the default. Local handles the work. The cloud is the escape hatch.
When local should stay local
Keep it on your machine — no exceptions — whenever any of these is true:
- The data is sensitive. Client files, medical records, proprietary code, anything under NDA or GDPR. If it shouldn't leave the building, it shouldn't leave the machine. This is the whole point of privacy-first RAG.
- The task is routine. Summarising, drafting, reformatting, everyday code help — a well-chosen local model does this all day for free. Paying a cloud provider per token for work your laptop does fine is just waste, as the numbers in our cost comparison show.
- Free tiers are paid for in ads. Cloud providers increasingly fund free AI plans by injecting ads into answers and profiling every prompt. Your local model shows no ads and builds no profile on you.
- You need it offline or reliable. On a plane, on bad wifi, or when a provider has an outage, the local model just works.
For the vast majority of what people actually do with an LLM, that's the end of the story. No cloud required.
When a cloud fallback is defensible
There's a narrow band of tasks where a frontier cloud model genuinely earns its place:
- A one-off reasoning problem far beyond your local model's size — a gnarly architecture question, a hard proof, a large-scale refactor plan.
- Work on non-sensitive, public data where the extra capability is worth it and privacy isn't at stake.
- A capability your hardware simply can't host yet — a giant context window, a specialised modality.
The key word is fallback. You escalate to the cloud consciously, for a specific task, having decided the data is safe to send — not because it was the path of least resistance.
Never let a fallback be automatic for sensitive data. An "if local struggles, retry on the cloud" script will happily ship your confidential files to a third party the moment your local model hesitates. The escalation decision must be yours, made per task — never a silent default.
A simple decision rule
Before any prompt, ask two questions in order:
- Is the data sensitive? If yes → local, full stop. No fallback, ever.
- Can my local model do it well enough? If yes → local. Only if the answer is no and question 1 was also no, does the cloud become an option.
That's it. Two questions keep you sovereign by default and pragmatic at the edges. The cloud stops being where your AI lives and becomes a tool you occasionally borrow — with your eyes open.