Analog In-Memory Computing Versus Digital NPUs: Why Energy per MAC Is Only Part of the Story

Digital accelerators are limited mainly by the cost of moving data from memory to arithmetic units, not by the arithmetic itself. Analog in-memory computing sidesteps this by storing weights as conductances in a crossbar, where Kirchhoff's laws perform multiplication and accumulation in place, achieving femtojoule-scale energy per operation. However, converting the resulting analog current back into digital numbers consumes energy that can erase the advantage unless arrays are large, conversion costs are amortised, and low precision is acceptable. These conditions limit the technology's practical niches to always-on sensing and tiny, fixed-weight models. Large language models are a poor fit because they demand consistent precision, and device drift and calibration burdens have led several programmes to switch to digital designs.