Model Showdown Round 3: Ditching Ollama in Favor of llama.cpp

In , we added Gemma 4 and Kimi K2, fixed our scoring methodology, and watched Gemma climb to the top.

But something kept nagging at us.

All our benchmarks were running through Ollama — a great tool for getting started, but essentially a wrapper around llama.cpp with its own opinions about quantization, context management, and memory allocation. We were benchmarking Ollama's choices as much as the models themselves.

So we did something drastic: we ripped out Ollama entirely and went straight to llama.cpp. Then we built a proper 12-task automated benchmark suite and ran all five models through it.

The results changed everything. Spoiler: Qwen 3.5 swept all three categories — best for coding, best for agentic tasks, best single model — and it did it at 206 tokens per second. Read on to find out how.

Why llama.cpp Over Ollama?

Ollama is fantastic for ollama pull model && ollama run model. It's genuinely the best way to get started with local models. But when you're running them as infrastructure — serving through an OpenAI-compatible API to

Three models downloading sequentially. The Samsung 9100 Pro writes at 250+ MB/s — all five models landed in under 10 minutes.

The DNS Incident

Halfway through downloading, our DNS resolution failed. Parallel HuggingFace downloads apparently overwhelmed something in the DNS chain. The fix was unglamorous:

echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf

The benchmark suite ripping through Devstral's tasks. Consistent ~77 tok/s throughput — the dense models don't waver.

What's Next

• 
  Round 4: Max Aggression — Each model with its native chat template, optimized temperature per task type, and fine-tuned reasoning budgets. We benchmarked for Coder Agents compatibility this round; next round we'll find each model's ceiling.


• 
  Retesting Qwen 3.5 against the Cloud King, Claude - We'll test Opus 4.6 and 4.7 with the goal of figuring out our perfect hybrid setup.


• 
  Dailying Qwen 3.5 is now the default model on our homelab. llm-switch.sh qwen made it so.

By the Numbers

• 
  5 models benchmarked


• 
  12 tasks across 5 categories


• 
  ~25 minutes total benchmark runtime on the RTX 5090


• 
  206.7 tok/s — Qwen 3.5's peak throughput (fastest local model we've tested)


• 
  100.0 — Qwen's todo app score (first perfect score in three rounds)


• 
  44 GB reclaimed by removing Ollama


• 
  3 seconds — model swap time with llm-switch.sh
  


• 
  3 scoring bugs found and fixed before we trusted the results


• 
  85.3 — Qwen 3.5's weighted overall score, 7.2 points clear of #2