Next.js AI streams keep billing after users leave, unless the upstream request is aborted

·1 views

Developers building AI chat endpoints in Next.js often find token costs exceed what users actually read. The cause is that when a browser disconnects mid-response, the upstream model request continues generating, incurring charges for output nobody sees. The article recommends building the SSE stream with a ReadableStream and calling the upstream abort in its cancel() callback, which fires when the client connection drops. It also advises setting Cache-Control with no-transform so proxies don't buffer the stream, raising the route's max duration for long generations, and metering input, output, and cache tokens per request. The author promotes a starter kit that packages these patterns.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Disabling SSH Password Login May Not Work: Cloud-init Drop-in Overrides Config

A recent OpenSSH security advisory from Alpine Linux, which reached the top of Hacker News, prompted many administrators to question whether password authentication is really disabled on their servers. The author audited seven VPS instances and found that three still accepted password logins, despite having edited sshd_config long before. The cause was a drop-in file created by cloud-init, which sets PasswordAuthentication to yes and is read before any directives added later in the main file. Because sshd uses the first value it reads, the author's later 'no' setting was silently ignored. The article recommends checking the effective configuration with sshd -T and verifying from outside the server by probing which authentication methods it advertises, and it includes a Python script for auditing multiple hosts without sending any credentials.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

SimpleSAMLphp Metadata Signing Requires Managed Certificate Lifecycles

SimpleSAMLphp can sign the metadata it publishes and verify signed metadata from other entities, which makes certificates central to federation trust. A valid signature only shows that a known key signed the document, so operators must still decide which keys to trust and how metadata is refreshed and certificates rotated. The article advises using separate keys per entity and environment, keeping private keys restricted, and replacing any sample credentials before production. For rollovers, it recommends publishing the new verification certificate alongside the old one, waiting for consumers to refresh, and then removing the old certificate, while checking how cached metadata behaves when refreshes fail. Testing should cover malformed signatures, expired certificates, and unknown issuers, with monitoring for expiry, refresh failures, and validation errors.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Fujitsu Unit 1Finity Launches AI-Native VRAN to Run AI on Phone Tower Gear

1Finity, a telecom equipment and software unit of Fujitsu, has announced its full-stack AI-Native VRAN system is now available. The platform is built on NVIDIA Aerial, which virtualizes and accelerates radio access network functions. It aims to let carriers run mobile service and local AI workloads, such as factory or campus AI, on the same tower or edge hardware. The company is pitching the approach as a way for operators to earn additional revenue from existing infrastructure. However, 1Finity did not name any customers or publish performance figures in its announcement.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Builds Role-Based React Task Dashboard Using Context API and localStorage

A developer built EMS, an admin and employee task management app, to practice React state management without a backend. Persistence is handled entirely by localStorage, with no Express server or database involved. A single AuthProvider manages login state, user role, and task data, and admin and employee dashboards are rendered as separate component trees. The author isolated storage behind a utility module so the backend could be swapped later without changing the dashboards, and noted the approach isn't suited to multi-user or multi-device use. They invited feedback on whether the Context setup would hold up if a third role were added.

0 comments Read more at DEV Community