LangGraph's default state model silently inflates token costs, audit finds

·1 views

A cost audit of LangGraph, the official migration path from LangChain's deprecated ConversationBufferMemory, has found that its default state model carries the same unbounded memory-growth problem as the tool it replaced. LangGraph's built-in MessagesState uses an append-only reducer called add_messages, which accumulates every message in state without trimming or summarising. In a modelled 20-turn conversation, this default behaviour consumed roughly 77,500 input tokens — about seven times more than a naive flat estimate of 11,000 tokens. For a chatbot handling 500 conversations per day on Claude Haiku, the gap translates to approximately $798 per month in excess input-token costs. The audit notes that opt-outs exist within LangGraph, but teams must actively configure them, as the framework does not apply any windowing or trimming by default.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

KV Cache Quantization Expands Qwen 35B Context 8x on a 12GB GPU

A developer running Qwen 35B on an RTX 4070 with 12GB VRAM faced a critical bottleneck: with 95% of VRAM already in use, expanding the context window from 4,096 to 32,768 tokens was impossible without running out of memory. The key insight was that the KV cache — which stores all previously processed tokens — grows linearly with context length, making it the primary target for optimization. By switching the Key and Value cache data types from the default 16-bit (f16) to 8-bit quantization (q8_0) using llama.cpp flags, the cache's VRAM footprint was cut by roughly 50%. Benchmarks showed the 8x context expansion came with negligible quality loss and no measurable speed reduction, with community data suggesting under 0.1% perplexity degradation for q8_0. The developer recommends this approach specifically for heavy agent workloads, where tool definitions and system prompts can easily consume 19,000 tokens in a single turn.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Seven lessons from building real-world MCP servers beyond the demo stage

A developer who built multiple Model Context Protocol (MCP) servers — spanning personal, product, and enterprise use cases — has shared recurring design principles that separate production-ready servers from fragile demos. Key findings include keeping tool counts low to conserve context window space, and using MCP's native resource primitives for reading data instead of burning tool definitions. The developer also emphasizes that authorization should fail closed while rate limiters should fail open, and that tool annotations like readOnlyHint and destructiveHint help clients distinguish safe from destructive operations. Optimistic concurrency control, using content hashes to prevent silent data overwrites, proved essential when multiple agents or users write simultaneously. For enterprise deployments, the developer stresses defaulting to read-only access, logging every tool call, and masking PII in responses to satisfy security audits.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer builds shared design system that works natively in React and Angular

A developer created bpdm/ui after struggling with visual inconsistencies between a React app and an Angular admin panel at the same company, where even minor details like button border radius and focus ring colors kept drifting apart. The library solves the problem by storing all design tokens — colors, spacing, radius, and transitions — as plain CSS variables in a separate package that both framework implementations read from. It ships with 38 components across React and Angular, covering dashboard-focused elements like money inputs, KPI stat cards, and tree-selects, with accessibility handled by Radix on the React side and Angular CDK on the Angular side. The project requires Tailwind v4 and is MIT-licensed, though the author acknowledges it is early-stage and the Angular side has seen less real-world use than the React version. Source code is available on GitHub, with packages published to npm as @bpdm/ui and @bpdm/ng.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Temporal vs. Diagrid Catalyst: Choosing the Right Durable Execution Tool for AI Agents

Building reliable long-running AI agents requires durable execution platforms that handle failures, prevent duplicate tool calls, and maintain state across crashes. Temporal uses a workflow-and-activity model with broad language support, making it well-suited for teams comfortable writing application logic around its abstractions and replay semantics. Diagrid Catalyst, built on Dapr Workflows, sits beneath existing agent frameworks like LangGraph and CrewAI, offering a governance and execution layer without forcing teams to abandon their current tools. The choice largely depends on team size and structure: single-project teams may prefer Temporal's mature SDK, while platform teams managing multiple agent frameworks benefit from Catalyst's framework-agnostic policy enforcement. Regardless of platform, developers must still design idempotent side effects and define human-in-the-loop checkpoints, as durable execution addresses recovery but does not inherently make tool calls safe.

0 comments Read more at DEV Community