📏 Play with the LayerNorm vs BatchNorm visualizer: https://dev48v.infy.uk/dl/day27-layer-norm.html

Normalization inside a neural net is almost embarrassingly simple: take some numbers, subtract their mean, divide by their standard deviation, then rescale with two learnable knobs. Every normalization layer you have ever used does exactly that. The...