SShortSingh.
Back to feed

Why AI agents burn through tokens: resent context costs add up each turn

0
·16 views

The author argues that an AI agent's output quality often depends less on the model or prompt than on the economics of its context window. Because agents resend their entire working context on every turn, sloppy context is paid for repeatedly until a run ends. Common culprits include tool schemas that ship with every request even when unused, large tool outputs that linger in history, files re-read unnecessarily, stale logs, and verbose reasoning traces. The writer suggests a rough model in which cost scales with context size multiplied by how many turns it survives, which makes small inefficiencies dominant in long agent loops. Bloated context also dilutes the model's attention, so trimming waste can improve both cost and performance. The article recommends profiling the actual context sent to the model as a first step.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

ReA-Bench Introduced as Benchmark for Auditing Responsible AI Agents

A DEV Community post presents ReA-Bench, a benchmark for evaluating how responsibly AI agents can be audited, as reported by arXiv AI. The piece is a systems-focused write-up from ShVaSa Lab that discusses the work through the lens of infrastructure and hardware efficiency. It includes a sample Python pipeline runner and a flowchart outlining input preprocessing, a compute kernel, and verified output with telemetry. The authors report performance figures comparing an unoptimized baseline with their optimized architecture, citing reduced memory footprint, lower p99 latency, and improved symbolic invariant compliance. They also recommend that engineers weigh memory sizing, quantization trade-offs, and latency bounds when deploying such workloads.

0
ProgrammingDEV Community ·

Staged Retrieval and Deterministic Keys Prevent Duplicate Candidates in Search

The article argues that deduplication should be built into retrieval design for recruiting candidate search rather than handled as a cleanup step after a model responds. It recommends staged retrieval with explicit collections, bounded queries, and a stable source key derived from tenant, document ID, and content version. A retrieval result should carry identifiers, access-control metadata, source references, chunk text, and a score so reviewers can trace ranking changes. The author compares Elasticsearch, Pinecone, Weaviate, and a thin vector API called Infrai, noting each option's operational trade-offs. Because queues are at-least-once, the piece stresses consumer idempotency so that retries after worker timeouts update existing chunks instead of creating duplicate candidate hits.

0
ProgrammingDEV Community ·

Coding agents leave git worktrees piling up and consuming gigabytes of disk space

Tools like Claude Code, Cursor, and Codex run tasks in separate git worktrees so several agents can work simultaneously. Each worktree is a full checkout with its own dependencies and build output, so these copies accumulate unnoticed. One Claude Code user reportedly found 27 forgotten worktrees occupying 80 GB, and a Codex user with a Rails app had 61 worktrees taking roughly 100 GB. The article explains where each tool stores its worktrees and offers commands to list their sizes, check for uncommitted or ignored files, and confirm whether work has been merged before removal. It also recommends using git's standard removal without the force flag, and lowering each tool's retention settings to limit future buildup.

0
ProgrammingDEV Community ·

Auditing SPF and DMARC across many domains: why a 'pass' can still mislead

Checking email authentication for hundreds of domains is often scripted, but many scripts report misleading results. The guidance covers reading SPF and DMARC TXT records correctly, including joining records split into 255-character chunks. It flags domains publishing multiple SPF records as broken and explains that a +all mechanism effectively means no SPF protection. It also notes that DMARC p=none only collects reports while enforcement requires p=quarantine or p=reject, and that a low pct value weakens enforcement. The article concludes with a safe record set for parked domains that send no mail and a Node.js approach for sorting domains into enforcement buckets.