Neural Collapse: Why the Last Layer Converges to a Simple Geometric Structure

Train a deep classifier long past the point where training accuracy hits 100 percent, and something surprising happens. The messy, high-dimensional features in the penultimate layer do not stay messy. They collapse. Features from the same class shrink toward a single point, class means arrange themselves into a highly symmetric shape, and the... Weiterlesen: Neural Collapse: Why the Last Layer Converges to a Simple Geometric S…