Serving LLMs on IaaS is queueing plus memory pressure dressed up as ML. Every request has a prefill phase (prompt → KV cache) and a decode phase (token-by-token output). 

Throughput tuning pushes batching and concurrency. Latency tuning caps them to protect TTFT and ITL. With vLLM on a single L40S (PCIe), you win by setting hard limits and...