How Token-Bucket Rate Limiting Contained a 40x API Traffic Spike Gracefully

·1 views

A firmware update pushed to an integration partner's device fleet triggered a simultaneous sync, causing API traffic from that partner to surge from 40 to over 1,600 requests per second within 90 seconds. The engineering team discovered their existing fixed-window rate limiter handled the incident poorly, alternating between allowing all requests and rejecting everything with hard 429 errors, worsening the retry storm. Switching to a token-bucket approach allowed traffic to degrade smoothly into a steady trickle rather than cutting off abruptly at window boundaries. The team also implemented per-API-key buckets instead of shared endpoint-level buckets, successfully isolating the incident so no other partners experienced elevated error rates. A further lesson involved tuning Retry-After headers to accurately reflect real bucket refill times, which prevented well-behaved clients from inadvertently creating secondary retry surges.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Developer Builds Enterprise-Style DevOps Lab Around Fictional Fintech Company

A software developer has documented the creation of a self-built DevOps homelab modeled on a fictional fintech company called TNP Technologies, rather than running tools in isolation as most tutorials do. The lab simulates a 150-employee digital banking firm with hybrid infrastructure spanning on-premises servers in Vietnam and AWS, and runs entirely on a single 64GB Windows machine. A fictional product called TNP Pay — comprising a frontend, backend, and PostgreSQL database — serves as the real workload moving through the pipeline, giving every DevOps tool a practical purpose. The developer introduced a fictional org chart with six distinct roles to enforce realistic access control and security constraints, including audit trails, secrets management, and environment isolation. The project is planned as a 10-part series progressing from infrastructure setup through to chaos engineering, with each installment building on the last.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Code, LLM Calls, or AI Agents: A Developer Breaks Down When to Use Each

A developer set out to demystify what an AI agent actually is after finding job postings full of the term but lacking clear explanation. Through a hands-on Go project, they built four endpoints of increasing complexity — from a stateless LLM call to a multi-tool agent loop — without using any framework. The core finding: an agent is simply a loop where the model can request tool executions, with the developer's code running those tools and feeding results back until a final answer is reached. Most agent challenges, such as runaway loops, partial failures, and context overflow, map directly to well-known engineering problems like circuit breakers, sagas, and bounded caches. The project also raised a pointed question: if plain code can reliably solve a problem, there may be little reason to route it through a language model at all.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AI Agent Skills Pose Serious Security Risks as Adoption Surges on GitHub

AI 'Agent Skills' — small instruction folders that extend the capabilities of AI coding assistants like GitHub Copilot — have surged in popularity, with five related GitHub projects gaining over 6,600 stars in a single day on July 24. GitHub recently added a native install command, making it as easy to add a Skill as running a standard package manager command. However, GitHub itself warns in its documentation that Skills are unverified and may contain prompt injections, hidden instructions, or malicious scripts. A security researcher demonstrated the risk by building a deceptive Skill disguised as a harmless CSV formatter, exposing how bundled files can remain unreviewed by humans yet still be accessed and acted upon by the AI. Unlike browser extensions, Agent Skills currently have no review process or centralized vetting, leaving users largely responsible for their own security.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

The Odin Project Post Explores Why Learning to Code Feels So Difficult

A widely shared article on DEV Community titled 'Why Learning to Code is So Damn Hard' was written by Rachel Moser for The Odin Project and published on March 16, 2025. The post, tagged under web development and programming, accumulated 2,642 reactions and is an estimated 16-minute read. It addresses the challenges faced by people learning to code, resonating with both beginners and experienced IT professionals. The piece is associated with The Odin Project, a well-known open-source coding curriculum. Its broad appeal suggests it touches on common frustrations and milestones in the coding learning journey.

0 comments Read more at DEV Community