Gemma 4 on Amazon SageMaker: 4-Bit Embeddings Decode up to 1.39x Faster on One L4

This article repacks Gemma 4's quantization-aware trained (QAT) weights into several 4-bit and 8-bit formats and measures each one on the same Amazon SageMaker NVIDIA L4 endpoint. A suite of Python MCP tools is built to simplify management of the vLLM hosted deployment. https://github.com/xbill9/sagemaker-gemma Models Gemma 4 E2B, E4B, 12B, 26B... Weiterlesen