🔧 16 GB VRAM LLM benchmarks with llama.cpp (speed and context)

Here I am comparing speed of several LLMs running on GPU with 16GB of VRAM, and choosing the best one for self-hosting.

I have run these LLMs on llama.cpp with 19K, 32K, and 64K tokens context windows.

For the broader performance picture (throughput versus latency, VRAM limits, parallel requests, and how benchmarks fit together across hardware...