Never debug in production." It's one of the first rules every developer learns, right up there with "always backup your code" and "test everything twice." But after 15 years of building software, I've learned that sometimes—just sometimes—debugging in production isn't just acceptable, it's necessary.

Last month, I found myself SSH'd into a...