Developer builds on-device Flutter app that counts workout reps using pose estimation

A developer building TrainWiz, a Flutter fitness app that gamifies exercise through a virtual pet, faced significant technical challenges in counting workout reps using only on-device camera processing. The app uses tflite_flutter and a local pose estimation model to track squats and push-ups without sending any video footage to external servers, prioritizing privacy and low latency. Initial approaches relying on joint angle calculations failed in real-world conditions because 2D camera perspective distorts angle readings depending on phone placement. The developer found that tracking torso axis orientation — the vector from hips to shoulders — proved far more reliable than multi-joint geometry for detecting exercise type and counting reps. The key takeaway was treating rep counting as a signals problem rather than a geometry problem, and using live debug switches to test candidate signals on real bodies in real environments.