Gemini 3.1 Flash-Lite Beats Incumbent and Open-Weights Rival in Reading App Benchmark

Developers at IO reader ran a structured benchmark to find a replacement for gemini-2.5-flash, which is set to retire on October 16, testing it against gemini-3.1-flash-lite and gemma-4-26b across 12 production prompts from their iOS reading app. All three models were evaluated on literary analysis tasks using a passage from Melville's Moby-Dick, with results judged by both a human reader and Claude as an LLM judge. Gemini 3.1 Flash-Lite emerged as the top performer, delivering sharper and more precise responses while costing 40% less on output tokens than the incumbent. Gemma 4 26B, despite having the lowest list price, turned out to be the most expensive per reply at roughly four times the cost of Flash-Lite, largely due to billed reasoning tokens. The benchmark concluded that well-defined prompt contracts held consistently across all three models, suggesting production engines can be swapped without altering the product's behavior.