AMD ATOM + ATOMesh: Prefill/decode Disaggregation on ROCm

What: AMD shipped ATOM + ATOMesh, a ROCm-native LLM serving stack whose headline trick is prefill/decode disaggregation — splitting the two phases of inference onto separate pools of GPUs instead of crowding them onto one.

Why: Prefill and decode have opposite bottlenecks — prefill is compute-bound, decode is memory-bandwidth-bound — so running them on the same worker wastes hardware and lets one long prompt stall everyone else's token stream.

vs prior: A co-located server (vanilla single-pool vLLM) interleaves prefill and decode on the same GPUs; disaggregation runs each on its own pool tuned for its bottleneck, paying for it by shipping the KV cache across the interconnect between them.

Think of it as

A restaurant kitchen that splits the prep station from the plating line.

   ORDER (the prompt)

        │

        ▼

  ┌──────────────┐    KV cart      ┌──────────────┐

  │ PREP STATION │   down the      │ PLATING LINE │

  │  (prefill)   │═══ hallway ════▶│   (decode)   │──▶ tokens

  │ compute-heavy│  (KV transfer)  │ memory-bound │

  └──────────────┘                 └──────────────┘

   chops a whole                    plates dishes

   order at once                    one at a time

• prefill = the prep cook chopping a whole order's ingredients in one compute-heavy burst


• decode = the plating cook building dishes one at a time, back to the fridge each plate


• KV cache = the fridge of prepped ingredients every plate reaches into


• disaggregation = giving prep and plating their own stations and staff, each tuned to its job


• KV-cache transfer = wheeling the prep cart down the hallway from prep to the plating line


• KV-aware scheduling = sending each order to the line whose fridge already holds its prep

Quick glossary

Prefill — The first phase of inference: the model reads your entire prompt in parallel in one pass, building the KV cache. It does a lot of math per byte of memory it touches, so it is compute-bound.

Decode — The second phase: the model generates one output token at a time, and each step must read the whole KV cache plus all the weights to produce that single token. It moves a lot of memory for little math, so it is memory-bandwidth-bound.

KV cache — The stored keys and values for every token already processed, so the model never recomputes them. It is the dominant memory cost of inference — and, in a disaggregated stack, the thing that has to travel from the prefill pool to the decode pool.

Compute-bound vs memory-bound — The roofline distinction: a job is compute-bound when the GPU's math units are the limit, and memory-bound when memory bandwidth is. Prefill and decode sit on opposite sides of that line, which is the whole reason to split them.

Disaggregation — Running prefill and decode on separate pools of workers instead of one shared pool, so each pool can be sized and scheduled for its own bottleneck.

KV-aware scheduling — A scheduler that routes a request with knowledge of where its KV-cache blocks already live — so it can reuse a cached prefix (prefix caching) or steer a request to the worker that avoids a transfer.

ROCm / AITER / MORI / Instinct — ROCm is AMD's CUDA-equivalent software stack and Instinct its datacenter GPU line. AITER supplies the optimized ROCm kernels (the analogue of CUDA kernels), while MORI handles the distributed, RDMA-style communication for tensor/expert parallelism (AMD's own collective library, RCCL, is the closer NCCL analogue).

The news. On June 16, 2026, AMD published ATOM + ATOMesh, a paired ROCm-native LLM serving stack for Instinct GPUs, shipped as an early (alpha) preview. ATOM is an AITER-optimized inference engine (kernel acceleration via AITER, distributed communication via MORI); ATOMesh is the orchestration layer on top — it exposes an OpenAI-compatible API, manages multiple engine backends, and applies prefill/decode disaggregation and KV-aware scheduling, evaluated serving DeepSeek-V4-Pro on Instinct hardware. In AMD's framing it deliberately mirrors the vLLM/SGLang design — the same serving primitives, now on AMD silicon. — SGLang is one half of the vLLM/SGLang design ATOMesh mirrors; this is the engine-level optimization that lives inside a pool like ATOM's.

— shrinks the KV cache itself, which is exactly the payload a disaggregated stack has to transfer between pools.

.

Vollständiger Original-Bericht

Ausführliche Details, Code-Beispiele & Hersteller-Stellungnahme auf dev.to.

↗ Original-Artikel auf dev.to lesen