A lot of good points here https://antirez.com/news/169
0
·3 viewsantirez.com
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Related stories
0
ProgrammingDEV Community ·
How One Engineering Team Tripled Its AI Coding Bill and Clawed Costs Back
A small software company's AI coding expenses tripled in a matter of months after engineers fully adopted tools like Claude Code and Codex, with no visibility into where the money was going. The engineering lead identified three core blind spots: no per-user or per-task usage breakdown, no controls to stop runaway processes, and no governance over which AI model was selected for each task. A single developer was consuming more tokens than five colleagues combined, simply because their workflow defaulted to the most expensive model for routine tasks like documentation and commit messages. Enabling prompt caching proved the highest-impact fix, since agentic tools repeatedly send the same context and can read cached tokens at a fraction of the full input price. The team also found that cheaper, smaller models handled many routine tasks at identical quality, highlighting that model choice — often made automatically by the tool — has a major cost impact teams rarely scrutinize.
0
ProgrammingDEV Community ·
Developer flags harness bug after local 35B model passes task but fails eval on stop token
A developer running agentic evaluations on a local 35-billion-parameter quantized model (ornith-1.0-35b-Q8_0) via llama.cpp found a mismatch between actual model behavior and harness-reported results. In one test involving API key rotation, the model chose correct actions for all service classes, avoided trap tools, and produced an honest summary, yet the harness marked the run as failed because the model ended with prose rather than a recognized stop token. A separate database migration test revealed genuine model defects, including repeated discovery-action loops, a double-applied migration, and proceeding with a staging deployment despite a 503 error on the backup check. The developer is now seeking community input on reliable agent-loop termination strategies for llama.cpp, such as grammar-enforced finish tokens or treating a tool-call-free assistant turn as terminal. The core concern is distinguishing true model failures from instrumentation gaps in the evaluation harness itself.
0
ProgrammingDEV Community ·
Claude Code API Costs Explained: What a Typical Coding Session Actually Runs You
A developer who tracked Claude Code usage over 95 days has broken down the real API costs of running the agentic coding tool. Unlike standard chat interfaces, Claude Code can trigger 10 to 40 model calls per user request, and every call re-sends the entire conversation history as input tokens, making input volume the primary cost driver. A typical mid-size session of 40 model calls across a medium-sized repo generates roughly 3.6 million input tokens, costing around $11.40 on Sonnet or $19 on Opus at Anthropic's list prices before caching discounts. Prompt caching and the /compact command can meaningfully reduce costs, but the core billing mechanic remains: every token in the active window is charged on every subsequent call. To manage spending, the author recommends keeping CLAUDE.md files lean, scoping file reads tightly, and starting fresh sessions per task rather than letting context windows grow unchecked.
0
ProgrammingDEV Community ·
38% of AI Teams Cite Evaluation Debt as Top Blocker for Agent Deployments
Software engineer Paul Twist, writing in July 2026, argues that most AI teams unknowingly accumulate 'evaluation debt' — a gap between offline test suites and real-world production behavior. According to Voker's State of YC AI Agents 2026 survey, 38% of AI teams identify keeping evaluations current as their primary development blocker. Offline eval frameworks such as LangSmith, Braintrust, and DeepEval score agents against static datasets that quickly become outdated as production traffic shifts. The problem intensifies with multi-agent systems, where compounding execution paths and emergent behaviors make it nearly impossible to anticipate failures through pre-deployment testing alone. Twist concludes that the most critical failure signals exist in live production traffic, yet offline tools can only capture them weeks later after manual labeling and dataset updates.
Discussion (0)
Log in to join the discussion and vote.
Log in