Opinion: RNNs Are Superior to Transformers — A Technical and Environmental Case

A developer essay published on DEV Community argues that RNN and LSTM architectures are fundamentally more capable and sustainable than Transformer-based large language models. The author cites complexity theory research, including a 2022 TACL paper and a 2023 DeepMind study, claiming Transformers are bounded by constant-depth circuits and fail on tasks that RNNs handle natively. The piece contends that techniques like chain-of-thought prompting and long-context windows are workarounds that expose Transformers' lack of true stateful memory, rather than genuine architectural strengths. Energy consumption is raised as a major concern, referencing a 2025 IEA report projecting AI-driven data centre electricity use to reach roughly 945 TWh by 2030, contrasted with RNNs' constant per-token compute cost. The author also points to a 2024 Nature study on model collapse and Ilya Sutskever's NeurIPS remarks about the limits of pre-training data to argue that the Transformer paradigm faces an imminent ceiling.