Developer finds 45M-parameter on-device LLM fabricates numbers with full confidence

A developer built react-native-needle, described as the first React Native binding for Cactus Compute's Needle 2, a 45-million-parameter language model that runs entirely on a mobile device without any server or network connection. During testing on an arm64 emulator, the model performed reliably on simple, single-intent commands such as controlling lights or setting timers, with response times as low as 136 milliseconds. However, the developer discovered a consistent flaw: when processing numbers embedded in complex strings, the model fabricated incorrect values while simultaneously reporting a confidence score of 1.0 and even flagging the exact fabricated field in its own validation output. Additional limitations included the model failing to parse multi-intent inputs and string fields occasionally capturing unrelated adjacent text. The package is restricted to 64-bit devices due to an ABI mismatch in the prebuilt binary, and adds roughly 28 MB to an APK rather than the advertised 14 MB.