Developer Cuts AI Agent Token Costs by 60% With Four Infrastructure Changes

·3 views

A software developer reduced token spending on multi-step AI agent workflows by 60% without degrading output quality by overhauling the underlying infrastructure rather than the agents themselves. Instrumentation revealed that 40% of tokens were wasted on repeated context, 15% on unnecessarily powerful models, and 10% on unused prompt content. A three-tier caching system combining exact-match, semantic near-match, and in-process LRU caching achieved a 30–35% call deflection rate, meaning nearly a third of model calls were eliminated entirely. Scoping context windows so sub-agents receive only the information they need — rather than full parent context — alone accounted for a 12% reduction in total spend. The developer emphasizes that granular per-call instrumentation, not aggregate token counts, is essential to identifying and fixing these inefficiencies.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Study Compares Input Latency on Linux Across X11, Wayland, VRR, and DXVK

A technical blog post by Marco Nett examines and measures input latency on Linux systems under different configurations. The study compares the legacy X11 display server against the newer Wayland compositor in terms of responsiveness. It also factors in Variable Refresh Rate (VRR) technology and DXVK, a Vulkan-based translation layer used for running Windows games on Linux. The findings aim to help Linux users and gamers understand how their display stack choices affect real-world input lag. The post attracted discussion on Hacker News, where it received points and community comments.

0 comments Read more at Hacker News

ProgrammingDEV Community ·

Engineer tests six LLM eval frameworks in CI merge queue; only two proved reliable

A software engineer ran six open-source LLM evaluation frameworks inside a real GitHub Actions merge queue over eight months, testing them against production pull requests. The core criterion was determinism: a gate had to return the same pass or fail verdict on unchanged inputs every single time. Frameworks relying heavily on LLM-as-judge scoring failed this test, as judge-assigned scores drifted between runs without any code changes, causing false blocks and eroding team trust in the gate. Promptfoo and DeepEval were the only two frameworks that consistently met the bar, largely because they kept deterministic checks in the blocking path and treated judge scores as non-blocking signals. The author concludes that a flaky CI gate is worse than no gate, since teams learn to bypass it, rendering it meaningless.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

DeFi Bot Triggers Rebalance Alert on $20 Position, But Gas Costs Kill the Trade

A DeFi investor monitoring a $20 USDC position on Base blockchain watched Morpho bbqUSDC's APY climb to 7.30% against Moonwell mUSDC's 4.77% over twelve days, eventually crossing a 2% rebalance threshold on July 14, 2026. Although the automated system logged a rebalance action, it was a known bug — the position remained untouched after the investor manually reviewed the economics. The 30-day incremental yield from switching protocols amounted to roughly $0.42, while estimated gas fees for the two required transactions totalled $0.45–$0.50, meaning transaction costs would exceed 100% of the expected gain. The investor's secondary guardrail requires gas to represent less than 10% of the 30-day incremental yield, a bar the trade failed decisively. The case highlights how small DeFi positions are structurally disadvantaged by fixed gas costs, with the same trade becoming justifiable only at around $500 in principal under current conditions.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

RepCN: Open-Source Tool Lets You Deploy React Code With a Shareable URL

A developer behind the content platform iHateReading has released RepCN, an open-source tool designed to deploy single-file React components to a shareable URL or custom domain. The tool addresses a gap in AI coding assistants like Claude, which can render HTML in-browser but cannot preview React apps built with frameworks like Next.js, Vite, or TanStack. Currently, React code generated by Claude can only be shared through its Artefacts feature, with no option to produce a standalone link or subdomain. RepCN aims to fill this need by providing a lightweight deployment pipeline for React code snippets. The project is part of a broader series of internal open-source tools the developer has built to streamline content creation and reduce operational overhead.

0 comments Read more at DEV Community