kafka-harbor 1.0 adds retries, dead-letter topics and safe shutdown to Node.js Kafka

·6 views

The author has repeatedly rebuilt the same error-handling layer on top of Node.js Kafka clients, covering retries, dead-letter topics, offset commits and graceful shutdown. Existing clients such as kafkajs and @platformatic/kafka handle the wire protocol but leave these reliability concerns to each service. kafka-harbor, now at version 1.0.0, provides this layer on top of a client through a small adapter interface, with adapters for Confluent's client and @platformatic/kafka. Failed messages are re-produced to tiered retry topics and then to a dead-letter topic, while the consumer parks messages without being evicted from its group. Its stated principle is that a message is never silently lost, since every failure ends in a retry topic, the DLQ, or an explicit consumer stop.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Ivanti Connect Secure Flaws Chained to Gain Root Access, Patches Issued

Attackers chained an authentication bypass, a file-write flaw and a privilege escalation to reach root on Ivanti Connect Secure, a VPN appliance at network edges. The exploitation reportedly began on 3 October 2026 against a build described as fully patched at the time. Ivanti published fixes and Volexity released its incident report on 13 October, ten days after exploitation started. The initial two flaws share a single CVE rated 8.6, while the final privilege-escalation step had no patch yet and a fix was promised in a future release. The compromised appliance can hold directory service credentials, local administrator passwords and BitLocker recovery keys, raising the stakes of the breach.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Dagster vs Airflow: choosing the right orchestrator for a small data team

A solo data engineer compares Dagster and Airflow, arguing the core difference is what each tool schedules: Airflow runs tasks, while Dagster manages the data assets those tasks produce. The author runs a production Dagster pipeline with 112 assets, 93 asset checks and 57 schedules, and credits the asset model with tracking partitions and keeping checks beside the data they validate. The piece notes that partition granularity is a deliberate trade-off, citing a case where one unpartitioned asset rebuilt 1,882 dates in about 45 seconds. The author has not run Airflow in production, so the Airflow comparison draws on its documentation. They conclude that for a large team already on Airflow migration rarely pays off, while a small team starting fresh may benefit from Dagster's asset-centric approach.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Adding AI Content Detection to Apps via REST API: Node.js and Python Guide

A tutorial explains how developers can integrate AI-generated content detection into applications that accept user text such as essays, applications, reviews, or support tickets. It uses the ProofAI API as its example, with Node.js and Python implementations covering input length validation, error handling, and secure API key management. The guide recommends placing the detection call behind a server-side endpoint so secret keys are never exposed in browser or mobile code. It also walks through mapping HTTP response codes to appropriate app behaviors and suggests queuing bulk requests with concurrency limits. The author notes that the patterns generally apply to any detection provider, and stresses that how results are presented to users is a product decision.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Changing an Image's DPI Requires Editing Metadata, Not Pixels

Print shops and government forms often require images at 300 DPI, but many people accidentally resample their photos when trying to meet this demand. DPI is actually a metadata value in the file header that tells a printer how many pixels fit in an inch, so the pixel data itself can stay untouched. In PNG files, this value lives in the pHYs chunk, while JPEGs may store it in a JFIF segment, an EXIF segment, or both, which means every existing location should be updated consistently. A developer found that a test suite passed only because the writer and reader shared the same assumptions, and an independent library revealed that PNGs with an eXIf chunk kept a stale value. The key lessons are to update all existing density fields and to verify results with a reader that was not written by the same developer.

0 comments Read more at DEV Community