The 2 AM failover we couldn't fix without a deploy


Last March our Singapore origin started serving stale trending lists to the Japanese and Korean edges. The fix was trivial — reroute jp and kr traffic to our Tokyo pull node and drop the weight on Singapore to zero. The problem was how. Our region routing table lived in a PHP config array baked...