I loaded Qwen2-VL-72B-Instruct at full BF16 precision on a single GPU, served 64 concurrent DocVQA streams, and kept the system stable at 99.5% KV cache utilization - all for $1.99/hour on the AMD Developer Cloud.

This post walks through exactly how I did it: the hardware economics that make it possible, the deployment configuration that makes it...