How to architect a backend system capable of handling 100,000 requests per second

·1 views

Scaling a backend to 100,000 requests per second requires a layered architecture rather than a single powerful database. The approach involves global traffic routing, edge caching, load balancing, and stateless autoscaled microservices working in concert. Multi-layer caching with dynamic TTLs and cache pre-warming before peak traffic periods helps maintain high hit ratios without serving stale data. Write scalability is addressed through database sharding, read/write splitting, and asynchronous processing via event queues like Kafka. Resilience is reinforced through cell-based isolation, load shedding during stress, and progressive rollout of features from single-region setups to full multi-region deployments.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Why Unit-Testing LLMs Fails and How a Structured Eval Harness Solves It

A developer building an internal LLM-assisted tool found that standard unit tests are ineffective for non-deterministic AI outputs, where results vary with every run. To address this, they adopted a two-part strategy: minimizing the LLM's role to a single drafting step while keeping routing, tracking, and safety checks as plain deterministic code. The LLM sits behind a provider interface, allowing the entire deterministic layer to be tested with ordinary unit tests using no API calls. For the irreducible non-deterministic portion, a dedicated eval harness was built to honestly assess what each testing layer can and cannot catch. The approach shifts the goal from chasing exact outputs to verifying system behavior within clearly defined, testable boundaries.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Why Batch Pipelines Fail Corporate Knowledge Graphs and How to Fix Them

A developer building 2asy.ai, a Graph-RAG system for East Asian corporate intelligence, discovered that standard batch pipelines break down when applied to incrementally updated corporate knowledge graphs. The core problem is that batch-based entity resolution relies on blocking keys across an entire document corpus, meaning new incoming data cannot be matched against existing graph nodes, leading to duplicate disconnected entries. A second issue arises when company details change over time, as naive append logic stores both old and new values simultaneously, causing conflicting facts that confuse downstream language models. The proposed fixes include a live-graph entity resolution pattern that checks incoming entities against existing nodes before insertion, and a time-bounded fact assertion model that supersedes outdated property values rather than overwriting or duplicating them. Both solutions are demonstrated with Python and Neo4j code examples designed for production incremental update pipelines.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer finds JSON-locked AI design kills creativity, shifts to token-only approach

A developer managing over 600 web tools across five locales experimented with ways to prevent AI-generated UI from drifting in design between generations. An initial approach of defining each tool's layout as JSON eliminated inconsistency but produced rigid, spreadsheet-like screens with no visual expressiveness. The developer concluded that locking layout structure through JSON caps design quality at whatever the renderer can implement, effectively trading drift for lifelessness. A revised strategy locked only design tokens — CSS variables for color, typography, spacing, and radii — while allowing the AI to compose layouts freely using a shared UI kit. This shift preserved brand consistency while restoring visual flexibility, aided by improving AI design capabilities in models available through early 2026.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Self-Taught Developer Builds Edge-Computing Air Quality Monitor Using AI and Modular Hardware

A developer with no formal electronics or C++ background built a fully functional smart air quality monitor using an ESP32 microcontroller, off-the-shelf sensors, and AI-assisted coding. The device autonomously reads gas, temperature, and humidity levels, categorizes air quality and room comfort in plain language, and triggers a fan via relay when air quality drops. The creator followed a six-step framework centered on modular open-source hardware, pre-written libraries, and iterative AI-guided learning rather than traditional study. Troubleshooting revealed real-world pitfalls, including an OLED display using an unexpected driver chip and an ESP32 board that was not fully seated in its adapter. The project was shared on DEV Community as a practical guide to encourage others to build hardware projects without prior technical expertise.

0 comments Read more at DEV Community