Last year, I mentioned that I'm interested in learning how to serve LLMs in production. At first it was just curiosity, but over time I wanted to actually try building something—not just reading about it.

This post is a small step in that direction: serving an LLM using vLLM, deployed on Amazon EKS, provisioned the infra using AWS CDK, and...