AMD wants to make enterprise inference cheaper and faster with chips from Taalas

As enterprises look for ways to cut the cost of running AI models in production, AMD is betting that not every AI workload will be best served by a power-hungry general-purpose GPU.

AMD has agreed to buy Taalas, the Canadian designer of chips that permanently embed a trained AI model’s weights into custom silicon, instead of repeatedly loading them from memory during inference as conventional GPUs do.

Taalas says its approach reduces the time and power required to move model weights between memory and compute units, making things run faster and cheaper.

The result is a highly specialized inference processor optimized for one model, trading the flexibility of programmable hardware for substantially higher throughput and energy efficiency.

Operational tradeoffs

While AMD is planning to integrate the chips into its Instinct GPU roadmap, targeting system-level AI inference solutions in data centers, analysts remain skeptical that enterprises will readily embrace hardware tied to a specific AI model.

Enterprises would, effectively, be buying a chip and a model together because unlike GPUs, which can be repurposed to run different AI models through software updates, Taalas’ chips are tied to a specific trained model, meaning they would need different hardware to support different inference tasks, said Amit Kumar Jena, AI development manager at IT Consulting firm Kanerika.

Or as Forrester Principal Analyst Charlie Dai put it, “The biggest risk is inflexibility.”

The requirement to swap hardware in order to swap tasks would, Dai said, introduce new challenges with costs, governance, capacity planning, lifecycle management, and supplier dependency, especially for enterprises managing multiple AI workloads.

Manoj Chandra Jha, principal analyst at Nord-IQ Research, said the risk of fusing chip and model into one component is larger than one might think, as “early model obsolescence strands both together, so this should be modeled as one shorter-lived asset rather than two independently amortized ones.”

Taalas says it can update a model by modifying only two metal layers of the chip rather than redesigning it from scratch, but that will only apply to chips that haven’t yet left its factory, not those already in use.

That means enterprises will still need to plan for hardware refresh cycles measured in weeks or months and retain programmable GPUs for workloads that evolve frequently, said Pareekh Jain, principal analyst at Pareekh Consulting.

It also means, said Jha, that what is typically a software decision becomes one about capital expenditure for Taalas customers, as replacing or switching workloads or models could require investing in new hardware rather than simply updating software.

Where model-specific silicon fits

Those tradeoffs significantly narrow the range of enterprise workloads where model-specific silicon is likely to make economic sense.

Dai sees the technology as best suited for mature, predictable inference workloads that run at massive scale and rely on relatively stable AI models, such as customer service automation, fraud detection, industrial computer vision, network operations, edge AI, and embedded copilots.

For CIOs, that effectively limits model-specific silicon to a small subset of enterprise AI deployments, rather than a wholesale replacement for GPU infrastructure, he said. “GPUs will remain the preferred enterprise platform because most enterprises value flexibility, multi-tenancy, and rapid model evolution over maximum efficiency.”

This article first appeared on Network World.