Google Cloud Tech: Model types and performance bottlenecks

Author: Google Cloud Tech - Bewertung: 0x - Views:7

Learn why your powerful new AI model might be running slowly during inference. This video dives into the landscape of modern AI models, including Large Language Models (LLMs), Diffusion Models, Visual Language Models (VLMs), and Mixture of Experts (MoE). We uncover the four common performance bottlenecks—compute, memory capacity, memory bandwidth, and networking—and provide practical strategies for engineers to identify and address these issues, helping you achieve optimal performance for your AI applications.



Chapters:

0:00 - Introduction: Why is my AI model slow?

0:47 - The 4 types of modern AI models

2:03 - The four common bottlenecks

4:12 - Practical strategies for LLMs (Quantization)

4:52 - Practical strategies for Diffusion Models

5:24 - Practical strategies for Mixture of Experts (MoE)

5:53 - Conclusion: A playbook for performance



Resources:

AI Hypercomputer overview → https://goo.gle/3JMXNb2

Introduction to Cloud TPU → https://goo.gle/4nMA0WE





Subscribe to Google Cloud Tech → https://goo.gle/GoogleCloudTech



#GoogleCloud #LLM #VLM #AIModel



Speakers: Duncan Campbell

Products Mentioned: AI Infrastructure