WKafka Library Brings Request-Response Pattern to Apache Kafka for Python Devs

·27 views

WKafka, an open-source Python library, addresses the lack of a native request-response pattern in Apache Kafka by enabling decoupled, asynchronous task processing. The library allows worker nodes to consume tasks from one Kafka topic and publish results to another, using message keys as correlation IDs to match requests with responses. This approach eliminates common pain points such as HTTP 504 timeouts, tight coupling between APIs and background workers, and lost task notifications during worker crashes. WKafka supports Python versions 3.9 through 3.14 with strict typing and has been tested against real Apache Kafka broker clusters. The project is available on both GitHub and PyPI as part of an ongoing open-source engineering series.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Handling Queue Timeouts in Delayed Game Webhook Delivery

A worker delivering delayed game-event webhooks should acknowledge a queue message only after the delivery attempt reaches a durable outcome. The article argues that a timeout must lead to a retryable state rather than a lost cleanup job or an unbounded retry loop. It recommends separating the queue lease from the HTTP request timeout, recording an attempt before sending the request, and making the receiving service idempotent. Because a timeout only shows that the worker stopped waiting, it cannot confirm whether the remote side applied the change. The proposed design classifies each result as delivered, retryable, or terminal, and acknowledges the message only once that state is stored.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Databricks Unveils Autonomous Genie That Runs Scheduled and Triggered Enterprise Jobs

On 11 November 2026, Databricks announced a new version of its Genie AI, which can now carry out delegated jobs on its own rather than only answering questions. Users can assign work through Genie One, email, Slack or Microsoft Teams, with jobs running on a schedule or in response to events such as a metric drop. The release also adds Genie Ontology, Genie Search, Genie Agents and Unity Gateway for governance, along with connections to tools such as ChatGPT, Claude and Copilot. Databricks says some components are generally available while others remain in beta or coming soon, and that availability for autonomous jobs was not specified in its main post. The company also reported a 91% correct-answer rate in a benchmark, noting that the results reflect an established deployment and should be treated as a ceiling.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Next.js page cut from 854 KB to 459 KB by trimming duplicated RSC payload data

A developer's /projects page built with Next.js 16 shipped 854 KB of HTML for 49 projects, largely because identical strings were repeated. In the App Router, such repeats are sent twice, once in the HTML and again in the inline React Server Components payload. The author reduced the page to 459 KB through three changes: an SVG sprite for icons, JSON-LD linking to detail pages instead of copying them, and named Tailwind utilities for row classes. A fourth change stopped the page from prefetching 36 unneeded routes. The author notes that compression hides this duplication in transfer size, though devices still must decompress and parse the full payload.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

MetarAndTaf.com adds VFR and IFR flight planning to its pilot weather briefing site

A developer has added flight planning features to metarandtaf.com, a weather briefing site for pilots that already offers METAR/TAF decoding and 3D weather views. Pilots can enter two airports or draw a route on the map to generate a VFR or IFR plan that avoids prohibited and restricted areas. VFR planning includes a wind-corrected nav log, an EASA-style fuel plan, and NOTAMs along the route, while IFR planning adds airway routing, an ICAO route string, and an ICAO flight plan. The route engine runs in the browser, with weather, NOTAM, airspace, and navigation data drawn from sources including AWC, Open-Meteo, the FAA, and openAIP. Access is sold as subscriptions priced at EUR 25 per year for VFR, EUR 30 per year for IFR, or EUR 35 per year for both, and the tool is explicitly a planning aid rather than an official briefing or flight plan.

0 comments Read more at DEV Community