This is deeply inspired by Andrej Karpathy's series about makemore, actually this post is just the notes of Andrej's first video, together with some insights that I came up with.




An Introduction To N-gram Language Model


N-gram language model is perhaps the simplest form of language model that any human could think of. Imagine the...