When a Zero-Parameter Cache Overtakes a Transformer

The question this series exists to answer, put in a form that can be measured: a transformer sees a fixed 64-token window, a count table over the current document sees the whole document, and as documents get longer, how much of the trained model does the free mechanism replace? An earlier experiment had established that the document cache is... Weiterlesen