Part 2: Why Transformers Still Forget

«This is Part 2 of a three-part series on why long-context language models still struggle with memory. In Part 1, we saw why increasing context length does not solve the memory problem. Here, we introduce a memory-cent...»