Developer proposes unified AI gateway to simplify access to 250+ LLM models

·1 views

A developer is validating an idea for a Unified LLM API Gateway to address the difficulty of managing multiple AI provider integrations. The proposed proxy would sit between an application and over 40 AI providers offering more than 250 models. It would be compatible with the OpenAI SDK, requiring only a change to the base URL and API key, and would automatically route requests to backup models during outages or rate limits. A single prepaid balance would also cover charges across all models. The developer is seeking community feedback before committing to building a minimum viable product.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

No Firewall Exists for Prompt Injection in AI Agents, Engineer Argues

An applied AI engineer at LuumenAI, a platform for SAP and ERP observability, argues that language models cannot reliably separate instructions from untrusted input, so conventional security controls fall short. The piece notes that the model treats operator prompts, user requests, and pasted content as one stream, which means text an attacker places in front of it carries real authority. It cites the EchoLeak vulnerability in Microsoft 365 Copilot, disclosed by Aim Labs in June 2025 and rated CVSS 9.3, in which a crafted email caused the assistant to exfiltrate sensitive data without any user action. The author frames the fix as controlling what the model can read and what it is permitted to do, and says a follow-up paper will address containment after the model acts.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Slothshot Uses a Local Gemma Vision Model to Rename Screenshots by Content

A developer built slothshot, a Go command-line tool that renames screenshots after what they show so they can be found through search. The author had 39 screenshots across three default naming schemes from macOS, GNOME and Flameshot, none of which described their contents. The tool runs Gemma 4 E4B, a 4-bit open-weight vision model served locally through llama.cpp, so no images leave the machine. It offers a dry-run default, an apply mode, an undo option that restores old names from a log, and a watch mode that names new screenshots as they appear. The author acknowledges errors, such as misspelling a brand name and omitting a key error message, but argues that imperfect names are still easier to search than default timestamps.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Blocking Embedding Calls Caused a 40% Request Drop in a vLLM Semantic Router Under Load

A FastAPI gateway wrapping the vLLM semantic router began silently discarding requests during a 2 AM traffic spike, with queue latency rising from about 80ms to 4.8 seconds. The root cause was a synchronous embedding computation inside the router call, which blocked the asyncio event loop and prevented it from accepting new requests. Requests that exceeded the internal five-second timeout were discarded without any error response. The fix moved the router call into a thread pool using asyncio.to_thread so the event loop stays free, and the author also benchmarks routing latency against a multi-provider relay.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Solo developer's one-bullet puzzle game Bloop changed after player feedback

Bloop is a mobile puzzle game built entirely by one developer, in which each level gives the player a single bullet to bounce off walls and clear every target. The game is now live on Google Play and the App Store with 300 levels, though the developer describes it as still in progress. Early feedback led to a switch from touch-based aiming to a slingshot-style pull-back control, and animated, reactive enemies that proved the most popular change. Some levels were found to be unbeatable, so the developer built an automated checker that tests every level with the game's physics and requires a realistic margin for aiming or timing. All 300 levels passed the checker on October 15, and future plans include new enemy types and additional galaxies.

0 comments Read more at DEV Community