SShortSingh.
Back to feed

Telling a cheap model to think less made it slower and sometimes silent

0
·6 views

A developer building a desktop app that routes requests through a Cloudflare Worker gateway to OpenRouter found a bug in how reasoning was capped. The gateway applied a reasoning effort setting to any model whose name matched a regex including "gemini-2.5", which unintentionally caught Gemini 2.5 Flash-Lite, a model that does not think by default. For non-reasoning models, any effort value switches thinking on, so the setting consumed part of the token budget and sometimes left the visible reply empty. A test call with a 16-token budget returned 12 reasoning tokens and a blank answer, despite a successful 200 response. The fix was a negative lookahead excluding Flash-Lite from the regex, along with tests to prevent regression.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Running Webhook Gateways at the Edge with Cloudflare Workers and Wasm

Webhooks drive event-driven workflows, but routing every incoming request to a central origin wastes capacity on invalid traffic and exposes shared resources to floods. The article proposes moving signature checks, payload validation, and tenant routing onto Cloudflare Workers, optionally with WebAssembly, so bad requests are dropped at the edge. Valid events can then be acknowledged quickly and passed to a durable queue for asynchronous processing by the origin. The piece also covers the architecture and code involved, along with platform limits and provider-specific signature schemes such as Stripe's and GitHub's HMAC-based approaches.

0
ProgrammingDEV Community ·

Apple Launches $299 HomeView Smart Home Display With A18 Chip and Face Recognition

Apple has unveiled HomeView, a $299 smart home command center combining its A18 silicon with Apple Intelligence. The device features a 7-inch square Liquid Retina display and is offered in both a tabletop countertop model and a flush wall-mounted version. Its 12MP camera supports facial recognition for up to six household profiles, which personalizes the on-screen widgets for each user. Pre-orders are open ahead of a November 13 global shipping date, positioning the device as a direct challenge to Amazon Echo Show and Google Nest Hub.

0
ProgrammingDEV Community ·

Vercel apps can now upload to S3 without long-lived AWS keys

Vercel's Amazon S3 Marketplace integration lets an app store uploaded files in S3 without embedding a permanent AWS access key in code or environment variables. The integration links an S3 resource to a Vercel project, provides connection details as environment variables, and authenticates using short-lived OIDC credentials. Developers still have to handle authorization, object key naming, file validation, and read permissions themselves. The integration is best suited to teams already using AWS, and it only works with buckets created through Vercel rather than arbitrary existing buckets. The article also stresses that the IAM role's permissions still govern what the credentials can do, so access should be limited to the required bucket and key prefix.

0
ProgrammingDEV Community ·

Developer Builds PubMed Memory Connector for AI Assistant on KRAS G12C Drug Resistance

A developer built a PubMed connector for cognee, an open-source memory layer for AI agents, to track research on how KRAS G12C tumours resist sotorasib and adagrasib. The connector takes a standard PubMed query and ingests matching records into a store the agent can query over time. In a test run against live NCBI data, 155 records from 2024 onward were stored in about seven seconds, and a repeat sync made no downloads because nothing had changed. The article highlights a 2026 case report describing a patient whose tumour developed Y96 mutations after treatment, with combination therapy failing to halt progression. The author argues that such persistent memory lets an assistant reflect recent findings that a frozen training dataset would miss.