Token Metering at an LLM Gateway Shows Which Teams Drive AI Costs

·1 views

A single LLM bill and one shared endpoint leave companies unable to tell which teams or use cases drive spending. The author's team built token metering into its LLM gateway, since request counts are a poor cost proxy and token usage appears only in each response body. Because every call passes through the gateway, inbound and outbound policies tag requests with subscription metadata and extract token counts to send to an analytics pipeline. Metering also had to include failed and rate-limited requests, which still consume input tokens, and once teams could see that spend, they fixed their clients.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

GitHub Rebuilds Git Storage as AI Agents Drive Surging Commit and Push Volumes

GitHub has published a blog post describing a rebuild of the storage layer beneath all its Git repositories, driven by the growth of AI agents. In September, developers and agents made 7.38 billion commits, more than five times the previous year, with pushes reaching 3.35 billion a month. The current Spokes system keeps a full copy of each repository on five file servers and requires a majority to agree on every ref update, so adding read capacity by adding copies slows down writes. GitHub plans to separate data safety from traffic serving, running heavy push steps in parallel so only the small ref-update step is serialized. This matters most for the main branch, where merges compete to update the same ref.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

PayGuard: Open-Source SDK Enforces Spend Limits and Risk Checks for AI Agent Payments

PayGuard is an open-source security SDK developed during a Razorpay hackathon to add financial safeguards to transactions made by AI agents. It acts as an automated guardrail that enforces spending limits, verifies idempotency, and runs risk evaluations before any purchase goes through. Users can define financial policies once and let the SDK apply spend caps and checks without human intervention. The tool is designed to run locally or on self-hosted infrastructure so sensitive transaction data stays off third-party servers, and it works with open-weight local models to avoid per-request costs. Because the code is open source, developers can adjust risk models, swap inference backends, or modify rule sets.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

GoldLift Offers a Free Alternative to Fitbod's Paid Workout Planning

GoldLift is presented as a free alternative to Fitbod, which charges about $15.99 per month or $95.99 per year after a 7-day trial and has no permanent free tier. Both apps generate workouts, but GoldLift builds a structured multi-week programme and adjusts per-set loads based on your results, while Fitbod creates each session from muscle freshness and available equipment. The developer, who is affiliated with GoldLift, adds that GoldLift includes an in-workout chat coach, nutrition logging, gamification and social features that Fitbod lacks. The comparison also notes Fitbod's advantages, including its mature recovery modelling, exercise videos and existing Android app, whereas GoldLift currently runs only on iOS. Prices were checked on 23 October 2026 in the US App Store and may differ by region or legacy plan.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Designing a Progress-Aware Controller to Decide When AI Agents Should Stop

A developer argues that deciding when an AI agent should stop is a system-level design problem rather than a matter of prompting or a fixed step cap. Fixed limits like max_steps are useful as safety boundaries but cannot tell whether a task is actually complete, since simple tasks and complex ones need very different numbers of turns. The proposed architecture separates the agent from a controller that checks actions against a budget gate, evaluates progress after each tool call, and tracks completion, progress, and remaining budget. The controller would return one of four outcomes: finish, continue, replan, or stop as incomplete, keeping budget exhaustion distinct from genuine success. The author favors a transparent, rule-based policy for an MVP, durable state storage, and head-to-head testing of fixed-limit, stagnation-aware, and progress-aware policies on success rate, cost, and premature stops.

0 comments Read more at DEV Community