Cloud Hardware Diagnostics for AI Workloads

With the recent boom in AI, the footprint of AI workloads and AI-supported hardware servers deployed in cloud data centers has grown exponentially. This growth is spread across multiple regions worldwide over various data centers. To support this growth and to ensure leadership over various cloud competitors (like Azure, AWS, and GCP), they have started building a fleet of specialized high-performance computing servers. The AI workloads that perform a huge amount of data processing, training, and inference of data models require a special kind of hardware, unlike traditional general-purpose compute servers. 

Hence, all cloud service providers are investing heavily in GPU, TPU, and NPU-based servers that are effective in hosting AI workloads. The majority of these servers are of the Buy Model type, and cloud service providers are dependent on the ‘Other Equipment Manufacturer’ (OEM) for diagnostics and maintenance of the hardware. This dependency has caused a lot of pain for cloud service providers as the repair SLAs are uncertain and expensive, impacting the fleet's availability.