SShortSingh.
TopTechnologyAI & MLStartupsProgrammingScienceBusinessFinanceCrypto & Web3World
Back to feed
Programming

How Apache Kafka Rescued a Hyperlocal Delivery Startup From System Collapse

0
·1 views

SwiftKart, a fast-growing hyperlocal delivery startup, saw daily orders surge from 4,000 to 40,000 in just two months, overwhelming its Redis-based queue architecture. Backend engineer Arjun and DevOps engineer Meera faced cascading failures including delayed notifications, overflowing queues, duplicate messages, and crashing workers. The core problem was that Redis lacked durability, multi-consumer support, and the ability to replay events, making it unsuitable for high-volume real-time streaming. The company's CTO decided to migrate to Apache Kafka, a distributed, durable, and replayable event log designed to handle millions of events per second. The article serves as both a narrative and a technical walkthrough of why Kafka was chosen, covering its architecture, key concepts, trade-offs, and scenarios where it may not be the right fit.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Meteor 3.5 Launches accounts-express to Unify Meteor and Express Authentication

Meteor 3.5 has introduced accounts-express, a new package that connects Express routes to Meteor's existing authentication system. Previously, developers needed custom middleware or separate session logic to use Meteor user credentials within Express endpoints. The package works by reading a Meteor login token from incoming requests and attaching the resolved user ID to req.userId, making it available throughout the Express handler chain. It also ships an authentication-aware fetch client, allowing browser code to automatically include login tokens when calling protected endpoints. This integration is designed to extend Meteor Accounts to HTTP routes without replacing or duplicating the app's existing authentication setup.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Keycloak CVE-2026-18963: Critical Flaw Allows Full Account Takeover Without Authentication

Red Hat and the Keycloak project patched CVE-2026-18963 on August 19, 2026, a critical vulnerability rated 9.1 on the CVSS scale affecting the password reset flow. The flaw allows an unauthenticated remote attacker to bypass the email token verification step and directly reset the password of any account, including administrator accounts. Any organization running Keycloak with the Forgot Password feature enabled on an internet-facing login page is at risk of complete identity compromise. Fixed versions include upstream Keycloak 26.7.2 and Red Hat builds 26.4.15 and 26.6.6. Organizations unable to patch immediately are advised to disable the Forgot Password option in every realm's login settings as a temporary mitigation.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Why mixing cryptographic proof with heuristics in one 'verified' field is risky

A developer has raised a concern about the widespread misuse of the 'verified' field in codebases, arguing it dangerously conflates two distinct concepts. The first is cryptographic proof, a mathematically certain outcome, while the second is an inferred judgment based on heuristics or scoring models. When both are stored in the same boolean field, downstream code treats probabilistic guesses as hard facts, which can lead to real users being incorrectly blocked or trusted actors being misjudged. The author recommends reserving a single boolean strictly for cryptographically settled outcomes and storing all inferred signals separately as scores or opinion sets. They also flag similar risks with fields like is_bot, is_fraud, and trusted, and are seeking community input on naming conventions that clearly distinguish proven facts from estimates.

0 comments Read more at DEV Community
0
ProgrammingDEV Community ·

Tiny 11.9M-Parameter Transformer Learns to Generate Web Apps After Simple Data Fixes

A researcher built a small 11.9-million-parameter transformer, trained on 4,176 generated programs, to convert plain-text feature specs into working Python web applications. Despite correct architecture and sufficient training, the model scored zero compilable outputs across five consecutive experiments, which the researcher initially misattributed to insufficient model capacity. The actual cause turned out to be two simple data pipeline flaws: docstrings were tokenised as single atomic tokens bloating the vocabulary to nearly 6,000 entries, and training samples lacked end tokens so the model never learned to stop generating. Fixing both issues — splitting docstrings into words and adding end tokens — reduced vocabulary size from 5,953 to 1,817 entries and immediately lifted compile success from 0 out of 6 to 6 out of 6. The researcher notes that the production tool still uses rule-based compositional synthesis rather than the model, and this post serves as a public retraction of an earlier claim that turned out to be premature.

0 comments Read more at DEV Community