Article URL: https://var0.xyz/posts/what-even-are-microservices.html Comments URL: https://news.ycombinator.com/item?id=49081594 Points: 9 # Comments: 9
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
A developer running Qwen 35B on an RTX 4070 with 12GB VRAM faced a critical bottleneck: with 95% of VRAM already in use, expanding the context window from 4,096 to 32,768 tokens was impossible without running out of memory. The key insight was that the KV cache — which stores all previously processed tokens — grows linearly with context length, making it the primary target for optimization. By switching the Key and Value cache data types from the default 16-bit (f16) to 8-bit quantization (q8_0) using llama.cpp flags, the cache's VRAM footprint was cut by roughly 50%. Benchmarks showed the 8x context expansion came with negligible quality loss and no measurable speed reduction, with community data suggesting under 0.1% perplexity degradation for q8_0. The developer recommends this approach specifically for heavy agent workloads, where tool definitions and system prompts can easily consume 19,000 tokens in a single turn.
A developer who built multiple Model Context Protocol (MCP) servers — spanning personal, product, and enterprise use cases — has shared recurring design principles that separate production-ready servers from fragile demos. Key findings include keeping tool counts low to conserve context window space, and using MCP's native resource primitives for reading data instead of burning tool definitions. The developer also emphasizes that authorization should fail closed while rate limiters should fail open, and that tool annotations like readOnlyHint and destructiveHint help clients distinguish safe from destructive operations. Optimistic concurrency control, using content hashes to prevent silent data overwrites, proved essential when multiple agents or users write simultaneously. For enterprise deployments, the developer stresses defaulting to read-only access, logging every tool call, and masking PII in responses to satisfy security audits.
A developer created bpdm/ui after struggling with visual inconsistencies between a React app and an Angular admin panel at the same company, where even minor details like button border radius and focus ring colors kept drifting apart. The library solves the problem by storing all design tokens — colors, spacing, radius, and transitions — as plain CSS variables in a separate package that both framework implementations read from. It ships with 38 components across React and Angular, covering dashboard-focused elements like money inputs, KPI stat cards, and tree-selects, with accessibility handled by Radix on the React side and Angular CDK on the Angular side. The project requires Tailwind v4 and is MIT-licensed, though the author acknowledges it is early-stage and the Angular side has seen less real-world use than the React version. Source code is available on GitHub, with packages published to npm as @bpdm/ui and @bpdm/ng.
Building reliable long-running AI agents requires durable execution platforms that handle failures, prevent duplicate tool calls, and maintain state across crashes. Temporal uses a workflow-and-activity model with broad language support, making it well-suited for teams comfortable writing application logic around its abstractions and replay semantics. Diagrid Catalyst, built on Dapr Workflows, sits beneath existing agent frameworks like LangGraph and CrewAI, offering a governance and execution layer without forcing teams to abandon their current tools. The choice largely depends on team size and structure: single-project teams may prefer Temporal's mature SDK, while platform teams managing multiple agent frameworks benefit from Catalyst's framework-agnostic policy enforcement. Regardless of platform, developers must still design idempotent side effects and define human-in-the-loop checkpoints, as durable execution addresses recovery but does not inherently make tool calls safe.
Discussion (0)
Log in to join the discussion and vote.
Log in