SShortSingh.
Back to feed

Developer builds lightweight Kafka-free CDC pipeline in Go to stream Postgres changes

0
·1 views

A developer has released Rift, an open-source Go binary that reads PostgreSQL Write-Ahead Log (WAL) changes and forwards them to destinations such as webhooks, Redis, or another Postgres database without requiring Kafka, Zookeeper, or Debezium. The tool uses logical replication via pglogrepl and pgx, supports optional JavaScript-based transformations through goja, and queues events to disk using BoltDB when a destination is unavailable. Rift also tracks schema changes by installing an event trigger in the source database that logs DDL operations like ALTER TABLE. During development, the author encountered a bug where inserts into the internal rift_ddl_log table were themselves treated as replication events, causing repeated sync errors against destination databases that lacked the table. The fix involved a helper function that filters out internal tables by checking both exact and suffix matches to handle inconsistent schema-qualified table names from the WAL decoder.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

Log in to join the discussion and vote.

Log in

Related stories

0
ProgrammingDEV Community ·

Developer builds free self-destructing web tools suite on Cloudflare's free tier

A solo developer has launched TempTools, a free, no-signup collection of web utilities designed to automatically delete themselves after use. The suite includes a mock API endpoint, expiring QR codes and short links, a file-sharing tool supporting up to 100MB, and single-view self-destructing notes and passwords. Built entirely on Cloudflare's free tier using Next.js, Cloudflare Workers, D1 SQLite, and R2 storage, the project avoids the need for user accounts or manual cleanup. Expiry is enforced at the moment of access rather than relying solely on scheduled cron jobs, ensuring secrets cannot be read after their expiration time. The developer also documented technical hurdles encountered, including a Next.js 16 middleware compatibility issue with Cloudflare Workers that required a configuration-level workaround.

0
ProgrammingDEV Community ·

Software Testing Intern Shares QA Lessons from Real-World Project Experience

A software testing intern at Scode Software Solutions documented key lessons gained from hands-on work in quality assurance across web and mobile applications. The intern's responsibilities spanned writing test cases, identifying bugs, reporting defects, and validating features before deployment. Exposure to multiple testing techniques — including functional, regression, smoke, compatibility, UI, and basic security testing — provided a broad understanding of QA practices. One of the most valuable skills developed was writing clear, actionable bug reports that help developers reproduce and resolve issues efficiently. The experience reinforced that software quality is a shared team responsibility built throughout the development process, not just at the end.

0
ProgrammingDEV Community ·

Dev Team Ditches AWS SES After Hidden Visibility Gaps Trigger Customer Alert Failure

A development team discovered a critical gap in AWS SES after a Friday incident where 200 of 14,000 emails silently bounced, leaving an enterprise client without system alerts. The root cause was an SPF record misconfiguration following a domain migration, but SES provided no proactive alerts or easy way to investigate individual delivery failures. The team found that while SES costs roughly $1 per 10,000 emails — potentially saving hundreds monthly over rivals like SendGrid — it lacks built-in delivery logs, searchable bounce reasons, and actionable event data without custom AWS infrastructure. This experience led them to build Reloop, an open-source, self-hostable email infrastructure tool designed to combine SES-level cost efficiency with the delivery visibility of managed providers. The episode highlights a broader trade-off: low-cost email sending engines require significant engineering investment to match the operational transparency that managed platforms offer out of the box.

0
ProgrammingDEV Community ·

Advanced Server-Side Caching Strategies to Boost Next.js App Performance

A technical deep-dive published on DEV Community explores advanced server-side caching patterns for Next.js applications, going beyond the framework's built-in revalidation options. The guide highlights that while Next.js offers Incremental Static Regeneration via the revalidate option, many real-world apps require finer control over cached data, especially for dynamic or authenticated content. Server-side caching is presented as essential for reducing database load, improving response times, and enhancing scalability under high traffic. The article covers in-memory caching using libraries like node-cache for single-instance deployments, with code examples showing how to wrap expensive API calls in a reusable cache utility. It also notes that horizontally scaled deployments — such as those on Vercel or Kubernetes — require external caching layers, as each server instance otherwise maintains its own isolated cache.