Client-Side Watchdog Fixes Silent AI Stream Drops Caused by Proxy Timeouts

·5 views

AI chat streams can cut off silently at exactly 30 seconds due to a proxy read timeout, not a model failure — the reverse proxy closes idle upstream connections after 30 seconds of no data. A developer identified the issue while testing a chat widget, finding that tokens flowed normally until the model paused, at which point the gateway dropped the socket without sending any error or close event. The bug was confirmed by reproducing the cutoff via curl and a local test server that simulated a 35-second pause between streamed responses. The root cause is the classic proxy_read_timeout behavior in reverse proxies like Nginx, which measures silence between reads rather than total connection time. The recommended fix is a client-side watchdog that monitors chunk delivery intervals and transitions the UI to an honest error state when the stream goes quiet beyond a set threshold.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer lets Claude Code AI agent autonomously attempt to build a business from scratch

A developer gave Claude Code, an AI coding agent, a single open-ended instruction: start with no money and figure out how to generate revenue. Before writing any code, the agent independently conducted market research by analyzing freelance platform trends and exploring browser extension opportunities. During the process, it unexpectedly discovered real leaked database credentials and authentication tokens in a local project on the developer's own machine. The agent also attempted autonomous distribution by trying to register on Hacker News to promote the resulting tool, revealing an unanticipated failure mode around AI-driven self-promotion. The experiment produced a free, MIT-licensed security scanning tool called agent-audit, now publicly available on GitHub, along with reusable automation infrastructure the developer plans to apply to future projects.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Student Builds C++ and Qt Banking System with Client-Server Architecture

A student developer created a banking system project using C++ and the Qt framework, split into a separate client GUI and server application. The client was built with QML, handling user inputs as signals sent to the server, while the server ran as a command-line interface. A JSON-based database was used to store data, with Qt's built-in JSON library managing file operations. The project featured two access levels — a standard user and an admin — with the admin class inheriting from the user class and receiving additional permissions. TCP socket communication between the two applications proved the most challenging part, with the developer relying on Claude AI to help implement the networking logic.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer cuts AI agent telemetry by 81% by eliminating redundant file echoes

A developer auditing their AI coding agent's telemetry discovered that 81% of stored data was redundant, with two-thirds of the largest rows being full copies of files already stored in version control. The agent's Edit and Write tools were logging entire edited files as tool responses, consuming 35.1 MB per week while carrying no unique information. After measuring actual data consumers and splitting telemetry by tool type, the developer devised a rule to retain only responses that contain information not available elsewhere. Implementing this change reduced per-row storage from 4,119 bytes to 2,326 bytes, with Edit tool response data dropping 92% per row. The exercise highlighted that aggregate metrics only signal a problem exists, while breaking data down by category reveals what to actually fix.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Fable 5.1 cuts cache costs 75% but thinking-heavy tasks cost 20% more

AI model Fable 5.1 launched with a significant reduction in cache read pricing, dropping from $1.00 to $0.25 per million tokens, while input and output prices remained unchanged at $10 and $50 per million tokens respectively. The developer behind the analysis found that the cost impact depends heavily on workload type: agentic sessions with repeated context re-reads can see bills drop by up to 45%, while high-effort single-shot tasks cost significantly more. Independent benchmarking firm Artificial Analysis recorded a 20% cost increase per task over Fable 5, measuring at max effort settings where the cache savings are less impactful. A separate test by researcher Simon Willison showed that running the same prompt at maximum versus low effort could cost 33 times more, highlighting how the model's effort dial drives costs far more than token pricing. Subscription users on the Pro plan also faced surprise charges, as Fable 5.1 falls outside standard usage limits and runs on pay-as-you-go credits with no introductory credit offered at launch.

0 comments Read more at DEV Community