Fixed-size wave memory can copy strings via a Fourier-based read operation

Researchers present Echo, a read operation for a sequence model whose only memory is a fixed set of complex waves, enabling it to copy strings from input to output. Echo treats the wave state as a spectrum, decomposing it over distances to select and retrieve what was written at each offset, all without extra state or attention layers. On a random-letter copy task, adding Echo sharply cut training loss compared with the same model without it, and a layout whose full turn is a power of two lets the decomposition use a fast Fourier transform. Models with Echo also performed well on a two-model cooperation task, but the method failed on a warehouse control task, and the reason is unclear. The authors note that a two-layer transformer solves the copy task with fewer parameters, while the fixed-size memory can only copy up to its capacity, a tradeoff that suits devices with limited RAM.