On-Device AI Hardware Faces Physical Limits: Heat, Power Droop, and Bandwidth

A PCB manufacturer warns that moving large language and vision models onto portable devices exposes physical constraints that software teams often overlook. Chips sustaining high compute loads can produce intense heat, and without adequate thermal copper, vias, or heatsinks, they throttle performance or shut down. Sudden surges in current demand can also cause voltage droop that crashes systems, a failure that code changes cannot fix. The article adds that memory bandwidth becomes a critical bottleneck on edge hardware, which is more constrained than in PCs or servers. The author argues that hardware design must be planned alongside software to make on-device AI reliable.