Why I did this to myself and why distributed systems are hard


So here's the thing. You can read the Google MapReduce paper in an afternoon. It's 13 pages. I've read it maybe four times now. And every time I thought I understood it. Then I tried to actually build the thing.

The scenario that got me started: you have a 1TB log file. You need word...