SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

Developer builds LLM gateway to track AI token costs per product after billing blind spots

0
·1 views

A developer running multiple AI-powered products — including a thesis search engine, an image processor, and a video tool — found it impossible to tell which product was driving up shared LLM bills. To solve this, they built a gateway proxy that sits between all products and OpenRouter, an OpenAI-compatible model aggregator, logging token usage and cost per tenant with each request. The gateway also adds semantic caching to avoid redundant model calls and enforces per-product consumption quotas. While preparing to publish the project, the developer discovered the README falsely claimed an automatic failover feature that had never been implemented, and corrected this by moving it to an honest roadmap section. The gateway is now live in production, with the thesis search tool as its first connected product, and the full code including architectural decision records is available on GitHub.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

How Tech Leads Can Safely Adopt AI Coding Agents and Agentic System Design

The term 'agentic' currently describes two distinct but related concepts: AI tools that autonomously read, write, run, and verify code in a loop, and software architectures where LLM-based agents make decisions and coordinate to complete business tasks. Both approaches share a common principle — an agent is only reliable when it operates within a clearly defined scope and explicit contract. As AI coding agents take over mechanical coding tasks, the tech lead's role shifts toward writing precise specifications and conducting more rigorous code reviews, with final production sign-off remaining a human responsibility. Poorly scoped tasks fed to fast-moving agents risk accelerating technical debt rather than reducing it. Guardrails such as explicit permissions, sandboxed environments, reversible actions, and mandatory human review on sensitive changes are considered essential, not optional, when deploying these methods.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Developer Builds Transcript-to-Study-Notes CLI Tool Using Only Python Standard Library

A developer created StudySift, a command-line tool that converts lecture transcripts into structured study notes, as part of a Zero Dependency Hackathon. The tool extracts keywords, definitions, examples, and important points from raw transcript text without installing any third-party packages. StudySift relies entirely on Python's built-in standard library modules, including argparse, re, collections.Counter, and pathlib, to handle all processing tasks. The tool processes transcripts through multiple stages, from sentence splitting and word frequency counting to definition detection and sentence scoring. The project demonstrates that many common development needs can be met with Python's standard library alone, reducing reliance on external dependencies.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

AI Agent Pinpointed Zero-Ad-Campaign Bug in Minutes, Saving Hours of Troubleshooting

Ad-tech startup Affset, which builds a white-label ad server and CPA network tracker, experienced a sudden flatline in campaign performance with thousands of clicks recorded but zero conversions. The team's first instinct was to blame traffic quality or targeting issues, but an AI agent with live access to both analytics data and a browser tool quickly investigated the problem. The agent identified that the CDN hosting their landing pages had suspended the domain due to an unpaid invoice, effectively taking the pages offline. Once the invoice was settled, the team verified the fix using the same AI-assisted method, confirming the tracking pixel was firing correctly before restoring traffic. The incident prompted Affset to implement uptime monitoring to prevent a recurrence, and the team credited the AI's combined view of data and live page state for the fast diagnosis.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Frontier LLM API prices shifted three times in August after five months of no change

After five months of complete price stability across ten flagship large language model APIs, August 2025 saw three pricing changes in under three weeks. DeepSeek raised its V4 Pro rates by roughly 264%, tripling the blended price and introducing a peak/off-peak schedule without notifying customers by email. Alibaba's Qwen model slot saw a modest price reduction, while OpenAI cut GPT-5.6 Sol prices by 29%, though the cut was labelled promotional and guaranteed only until November 2026. Despite the DeepSeek spike, the equal-weight index of ten flagship models ended August 5.7% lower, largely due to the OpenAI cut. The changes signal a shift from the previously rigid pricing pattern, with two of ten flagship models now publishing list prices that represent the top of a variable range rather than a fixed rate.

0 comments Read more at DEV Community
Developer builds LLM gateway to track AI token costs per product after billing blind spots · ShortSingh