Princeton Researcher Proposes RLT Architecture to Give Transformers Recurrent Memory

Princeton researcher Yifan Zhang published a technical report on September 12, 2026, introducing the Recurrent Looped Transformer (RLT), a new neural network architecture designed to address a core limitation of standard decoder-only language models. Unlike conventional Transformers that process every token through the same fixed number of layers regardless of context length, RLT carries the decoder's final hidden state and attention cache forward across each new token without resetting. The architecture pairs a causal encoder, which produces immutable key-value memory from input tokens, with a recurrent decoder that maintains a sliding-window attention cache updated at every step. Importantly, Zhang's report is a theoretical specification only — no training experiments, efficiency benchmarks, or reasoning quality results have been reported. The proposal raises open questions about whether the structurally deeper recurrent computation path would translate into practical gains once trained at scale.