Developer proposes method to prevent AI API costs from duplicate token generation

·1 views

A developer outlines a potential operational habit for managing AI model API streams that fail mid-generation. The proposal aims to avoid paying twice for the same output by resuming from a stored prefix rather than restarting the entire prompt. A sample Python implementation tracks a job's progress with a hash of the prompt and a cursor to mark generated tokens. The method suggests using API continuation features to complete a response without regenerating already-received text, thereby controlling costs.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Passkeys Criticized as a Potential Security Trap in Hacker News Discussion

An article published on Substack argues that passkeys are a security trap and advises against their use. The piece was shared on Hacker News, generating community discussion. The post garnered 23 points and one comment on the platform. The criticism centers on potential flaws or risks in the passkey authentication system. The debate highlights ongoing security concerns within the tech community regarding new authentication methods.

0 comments Read more at Hacker News

ProgrammingDEV Community ·

Developer Creates Free AI-Powered Accessibility Scanner for Small Agencies

Sheree from Curbcut developed a free, white-label web accessibility scanner using a team of AI agents in approximately one week. The tool is designed to help small agencies and freelancers provide clients with a preliminary, plain-English accessibility report without the need for a full specialist audit. Users can paste a URL, including password-protected staging site links, to receive a scored report identifying issues, relevant WCAG rules, and suggested fixes within about a minute. The scanner runs a fixed set of 14 concrete checks on the main page and up to four linked pages. The initiative aims to address common client questions about website accessibility that small shops often lack the resources to answer comprehensively.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Secure AI Prompts by Auditing Each Data Access Point

A DEV Community article advises treating each stage of an AI prompt's journey as a separate trust boundary. These boundaries include the AI model service, the hosting server, and the log pipeline. The article warns that sending sensitive data like credentials or private system details across these boundaries poses security risks. It recommends developers carefully audit what data each 'room' will access before sending a prompt.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

New app helps residents give feedback on urban redesigns after neighborhood walks

Sidewalk Senate is a new prototype app designed for residents who walk in their neighborhoods but do not engage with formal public meetings. The app prompts users to briefly note a planned outing and later write a reflection on their experience. If the visited location matches a curated public design proposal, a local AI model suggests connections between the user's reflection and the official document. The app is currently piloted for Long Island City, Queens, and does not track users or send any data.

0 comments Read more at DEV Community