I have been trying to understand how transformers work lately, and whenever we read or hear about transformers, there is one word which comes up more than any other: ‘Attention’ , although it’s not something that first appeared with transformers, but it has become the centerpiece of the whole architecture.

That said, attention by itself doesn’t...