Guide to Edge AI Benchmarking on Android: Beyond Simple Execution Time Metrics

Deploying AI models on Android devices presents a 'Performance Paradox' where models that run smoothly on development workstations struggle on mid-range consumer devices due to thermal throttling, memory management, and heterogeneous hardware. Unlike cloud environments with predictable hardware, Android devices distribute AI workloads across NPUs, GPUs, and DSPs, each with distinct performance profiles and energy costs. NPUs are optimized for deterministic latency and power efficiency, GPUs offer flexibility and high throughput for complex model architectures, while DSPs handle low-power always-on tasks like voice detection. Effective Edge AI benchmarking requires measuring not just execution speed but also thermal behavior, energy efficiency, and hardware-specific performance across this fragmented ecosystem. Google's shift toward system-level AI services like AICore further changes how large models such as Gemini Nano are deployed and updated on Android devices.