Attention and a 1982 Hopfield memory compute identical outputs, but accuracy degrades with noise

A developer showed that a modern Hopfield network and a single attention head produce bit-identical outputs when their inputs are set equivalently, since both are the same softmax-weighted retrieval computation. Testing the retrieval with random patterns, accuracy declined gradually rather than collapsing as the number of stored patterns grew, reaching about 59% at 200 patterns. Noise in the cue was far more damaging, dropping success from 100% to roughly 3% as noise rose. Sharpening the softmax with a higher inverse temperature helped, though with diminishing returns. The findings were reported from the author's own numpy experiments.