AI Crawlers Are Expensive. Agent Builders Should Care.

On August 29, kernel.org administrator Konstantin Ryabitsev published an unusually concrete account of what automated traffic can cost a website. Across five geographically distributed servers with 90 CPU cores, he reported that 14–16 cores were continuously occupied rendering Git commit pages for scrapers. The same repositories were already...