Olala mini: OVH's 7B model and how to use it today
On 17 November 2026, OVHcloud announced Olala, its first family of language models and the first member of that family: Olala mini 7A1B. The official launch takes place at the OVHcloud AI Summit on 19 November, the weights are already on Hugging Face under the Apache 2.0 licence and OVH says the model is small enough to run on a laptop.
That last point is the interesting one. Olala mini is not a fine-tune of an American or Chinese model. It is a new architecture, trained from scratch in Europe on public supercomputers and it is fast enough on modest hardware to be a realistic local assistant. Qwant already runs it in production for its flash answers, with responses 20% faster and running costs divided by three compared with the previous model.
What OVH actually released
The published family is larger than a single model. OVHai LLM released the pre-trained base model, the supervised fine-tuned checkpoint, the final post-trained model and two specialised variants for summarisation and translation. Everything is under Apache 2.0, with the technical report and the AI Act training-content summary published alongside the weights.
The headline specs:
| Olala mini 7A1B | |
|---|---|
| Parameters | 6.8B total, about 1B active per token |
| Architecture | 36 layers: 29 Mamba-3 and 7 attention layers, each followed by a mixture-of-experts block |
| Experts | 256 routed experts per layer, 6 active, plus 1 shared expert |
| Context | 32,768 tokens (input and output) |
| Licence | Apache 2.0 |
| Formats | Safetensors (BF16) |
| Training | 12.8T tokens, on JUPITER, Jean Zay and MeluXina |
The architecture explains the speed. Only about a billion parameters are used for each token and only 7 of the 36 layers keep a KV cache, so memory use stays low and the cache grows far more slowly with context length than in a dense transformer. OVH claims 160 tokens per second on an RTX A4500 and 50 tokens per second on a plain CPU. One caveat: in BF16 the weights alone weigh 13.6 GB, so the 8 GB GPU figure in the press release assumes a quantized version, not the full BF16 weights.
Can I use it with Ollama?
Short answer: not today and a Modelfile workaround will not save you.
Ollama embeds llama.cpp and ships its own runtime, which only loads architectures it knows. olala is a brand-new architecture — hybrid Mamba-3 and differential attention, with 256 experts per layer — and it is not in upstream llama.cpp yet. The official llama.cpp, vLLM and mlx-lm runtimes cannot load it either. Importing the weights with ollama create fails when the runner tries to load the model, because the architecture is unknown.
Support will arrive the day the architecture lands in upstream llama.cpp and Ollama rebases on it. Nobody has announced a date. Until then, there are two ways to use the model: a hosted API or the Transformers library.
Use it through a hosted API
OVHcloud serves Olala mini on AI Endpoints, an OpenAI-compatible API hosted in Europe. A few requests per minute work without a key; beyond that, create an AI Endpoints key. The model name is Olala-v1.0-mini-7A1B and the base URL is https://oai.endpoints.kepler.ai.cloud.ovh.net/v1.
curl -s https://oai.endpoints.kepler.ai.cloud.ovh.net/v1/chat/completions \
-H "Authorization: Bearer $OVH_AI_ENDPOINTS_KEY" \
-H 'Content-Type: application/json' -d '{
"model": "Olala-v1.0-mini-7A1B",
"messages": [{"role": "user", "content": "Explain how a heat pump works."}],
"temperature": 0.8, "top_p": 0.9
}'
Two settings you should not improvise:
- Sampling:
temperature=0.8,top_p=0.9,top_k=40. Other values send the model into repetition loops. - Reasoning effort:
disabled,low,medium(default) orhigh. It is a chat-template argument, so pass it inchat_template_kwargsif your client exposes them.
Because the API is OpenAI-compatible, Open WebUI, Continue, OpenCode and most agent frameworks can talk to it with the base URL and the model name above. The reply carries the reasoning in reasoning_content, the answer in content and tool calls in tool_calls.
Run it with Transformers
The weights ship with their own model code, so the standard Transformers library can load the model without any additional runtime:
pip install torch "transformers==5.8.*" accelerate einops
from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "OVHaiLLM/Olala-v1.0-mini-7A1B"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
repo, trust_remote_code=True, dtype="bfloat16", device_map="cuda",
)
inputs = tok.apply_chat_template(
[{"role": "user", "content": "Explain how a heat pump works."}],
add_generation_prompt=True, reasoning_effort="medium",
return_tensors="pt", return_dict=True,
).to(model.device)
out = model.generate(
**inputs, max_new_tokens=4096, use_cache=True, do_sample=True,
temperature=0.8, top_p=0.9, top_k=40,
)
print(tok.decode(out[0, inputs["input_ids"].shape[1]:], skip_special_tokens=False))
This is research code, not a serving stack. The model code runs in plain PyTorch, so a standard install is enough, but it is slow without optimised kernels: on a 64-core CPU it generates about 6 tokens per second. Use a GPU with enough memory for the 13.6 GB of BF16 weights if you want usable speed. Keep use_cache=True, because the configuration disables it by default. Send one prompt at a time.
The output contains the reasoning, the answer and any tool calls in separate channels. Use tokenizer.apply_chat_template() with add_generation_prompt=True to format messages automatically and set reasoning_effort to disabled, low, medium or high depending on how long you want the model to think.
What it is good at and what it is not
Olala mini averages 68.0 across the twelve benchmarks published in the technical report, 15.2 points above its supervised fine-tuned checkpoint. Its strongest area is instruction following, which is exactly what you want from a small local agent model; its weakest is competition mathematics. Tool calling is part of its training, even if the published table leaves out the BFCL benchmark.
The limitations are the usual ones for a model with a billion active parameters, plus two specific to this release:
- It stores less factual knowledge than a larger model and OVH designed it to read data through tools rather than answer from memory. Give it sources or tools when accuracy matters.
- It handles short, single-purpose requests better than long autonomous tasks. Chain short calls in your code and check each step.
- The context window is 32K, input and output combined.
- Long reasoning can fall into a repetition loop. Keep the recommended sampling and set a token limit.
- It has no clock and its knowledge stops in early 2026, so give it the date and any recent facts.
- It can reproduce biases and produce harmful content despite its safety training: in OVH's tests, 6.3% of attack prompts still succeeded and the model refuses about one benign request in ten. Add filters and human review suited to your application.
If you fine-tune it or ship a product under your own name, you may become a provider under the EU AI Act, with the obligations that come with it. That is true of every open-weight model, but OVH publishes the compliance documentation you need to start.
What comes next
Version 1.1 is expected by the end of the year, with a 128,000-token context and better mathematical reasoning. Version 2.0 will exceed 100 billion parameters and target three areas: agentic AI for businesses, cloud DevOps and cybersecurity. OVH also says Olala will eventually power OVHai Workspace, whose encrypted version needs models that run locally on the workstation — the same need this 7B model already addresses.
The bottom line
Olala mini is the first European model that makes the local-first argument concrete rather than ideological: a novel architecture, trained on public supercomputers, released under Apache 2.0, fast on a laptop and already in production at Qwant. It is not a ChatGPT replacement and it is not an Ollama model yet.
Today, the easiest way to use it is OVHcloud AI Endpoints, which gives you an OpenAI-compatible API without installing anything. For offline experiments, Transformers can load the weights, but expect research-grade speed until the architecture reaches the mainstream runtimes. And if you were waiting for ollama run olala, keep an eye on upstream llama.cpp — that is where the architecture has to land first.
This article will be updated as soon as Olala mini is available on Ollama.