Tencent Introduced "CALM" a New Language Models That Makes "LLM" Look Like a JOKE!

Author: Evolving AI - Bewertung: 2x - Views:0

For years, modern AI has relied on one core assumption: that language must be generated one token at a time. No matter how large or advanced today’s models appear, underneath, they all follow the same slow, step-by-step process. In this video, we break down why that assumption may no longer hold — and how a new approach from Tencent and Tsinghua University quietly challenges the foundation of modern language models.

Tencent’s new research introduces CALM (Continuous Autoregressive Language Models), a radically different way to generate language. Instead of predicting the next token, CALM predicts the next vector of meaning, a continuous representation that can encode multiple tokens at once. This allows the model to move through language in larger semantic steps, dramatically reducing sequence length, attention costs, and overall compute requirements. We explain how CALM works under the hood, including its use of autoencoders to compress multiple tokens into a single vector, why continuous representations scale semantic bandwidth better than discrete vocabularies, and how this approach avoids the inefficiencies of diffusion-style generation. We also explore the technical challenges this creates, from probability estimation in continuous space to new evaluation metrics like BrierLM, and how Tencent’s team solved them. The implications go far beyond one paper. As power grids saturate, data centers stall, and GPUs become increasingly expensive, the AI industry is running into hard physical limits. CALM shows that efficiency gains don’t have to come from bigger models or more hardware; they can come from raising the level of abstraction itself. Instead of predicting the next word fragment, models may soon predict the next idea. This video explores why CALM represents a potential post-token future for language models, how it compares to traditional Transformers, and whether this marks the beginning of the end for next-token prediction or just an early glimpse of what comes next.

So what do you think: is this the next major shift in AI architecture, or a research direction that won’t scale? Drop your thoughts in the comments, and if you want the real story behind the world’s fastest-moving AI breakthroughs, make sure to like and subscribe to Evolving AI for daily coverage.