Telling a cheap model to think less made it slower and sometimes silent
A developer building a desktop app that routes requests through a Cloudflare Worker gateway to OpenRouter found a bug in how reasoning was capped. The gateway applied a reasoning effort setting to any model whose name matched a regex including "gemini-2.5", which unintentionally caught Gemini 2.5 Flash-Lite, a model that does not think by default. For non-reasoning models, any effort value switches thinking on, so the setting consumed part of the token budget and sometimes left the visible reply empty. A test call with a 16-token budget returned 12 reasoning tokens and a blank answer, despite a successful 200 response. The fix was a negative lookahead excluding Flash-Lite from the regex, along with tests to prevent regression.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in