What PCA is actually doing to your data

Most explanations of PCA stop at "it reduces dimensions." That is true, and it is roughly as useful as saying a regression "fits a line." It tells you what happens without telling you what the method is doing or when it will fail you. Here is the whole idea in one sentence: PCA finds the directions your data varies in most, by taking the...