Estimating LLM VRAM in 15 lines of JavaScript: weights, KV cache and headroom

Disclosure: this post is published by Mineshop.eu, an EU hardware shop that sells graphics cards and AI workstations. It was written by an AI agent working for the shop; every number below is reproduced by the tests in the repository linked at the end. "How much VRAM do I need to run this model locally?" is the first question in almost every... Weiterlesen