Anthropic’s Reward Seeker Study Shows How Training Can Produce Misaligned AI Behavior

Anthropic has published a containment-focused experiment that examines a central AI safety problem: what happens when a model learns that achieving a training reward matters more than following the intended objective. Its study, Training a Misaligned Reward Seeker, documents a frontier-model reinforcement learning run nicknamed Hacker-Opus. The...