How One Developer Uses Postgres and ClickHouse Together for Uptime Monitoring

·8 views

A developer built an uptime monitoring platform that splits data across two databases based on a single rule: whether a row will ever be changed after it is written. Mutable data such as monitor configurations, incidents, and team settings live in Postgres, which handles transactions, constraints, and concurrent edits. Immutable check results — generated every 20 seconds per monitor per region — are stored in ClickHouse, an append-only column store suited for billions of rows. ClickHouse performance is further improved through careful choices like Enum8 types, LowCardinality strings, and DoubleDelta compression for predictable timestamp intervals. The author argues the choice is not about which database is faster, but about matching each store to the write pattern of the data it holds.

Read the full story at DEV Community

This is an AI-generated summary. ShortSingh links to the original source for the complete article.

Discussion (0)

What Really Happens When Cloudflare D1 Free Tier Hits Its 500 MB Storage Limit

A developer stress-tested Cloudflare D1's free-tier database by filling it to its 500 MB storage cap via the REST API in July 2026 to document exactly what breaks at the limit. The database rejected new writes with an HTTP 400 error code 7500 at precisely 499,994,624 bytes, a threshold enforced at SQLite's 4 KB page granularity, while read operations continued unaffected. Contrary to an initial assumption, small writes do not bypass the wall — they simply consumed the last remaining headroom before the limit was enforced. Recovery is possible immediately by deleting rows, which drops storage usage at once, but the deletion itself counts against the free tier's daily write quota of 100,000 rows. The author also flagged that building an index on a blob column can roughly double storage consumption, potentially pushing a database into the limit on its own.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

kafka-lag-exporter Archived in 2024: Why Teams Are Switching to klag for Kafka Monitoring

Apache Kafka's long-standing consumer lag monitoring tool, kafka-lag-exporter, was archived in 2024, leaving production teams without an actively maintained Prometheus exporter. Consumer lag — the gap between a partition's latest offset and a consumer's last committed offset — is a critical metric affecting data freshness, SLAs, and downstream systems. A newer tool called klag has emerged as a replacement, offering native GraalVM executable packaging with roughly 50 MB memory usage, Prometheus and OpenTelemetry integration, and time-based lag estimation. Unlike its predecessor, klag also provides hot partition detection, consumer imbalance indicators, and a read-only MCP server that enables AI-assisted operational troubleshooting. Platform teams managing large Kafka or Redpanda clusters are advised to evaluate actively maintained alternatives that reduce broker overhead and go beyond raw offset metrics.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

Developer Tests 10 Free AI Coding Tools, Finds Only 3 Worth Using

A developer with eight years of backend experience spent three days testing ten free AI coding tools across multiple languages and IDEs, including VS Code and IntelliJ IDEA. The evaluation covered code completion quality, free usage limits, domestic accessibility in China, and response speed. Cursor scored highest at 9.5/10, praised for its daily 500-completion free tier and built-in Claude and GPT-4o support, though it requires a proxy in China. GitHub Copilot earned 9/10, offering 2,000 free monthly completions with strong multi-language support, but also needs a proxy and has higher latency within China. Tongyi Lingma, Alibaba Cloud's free coding assistant, rated 8.5/10 as the top pick for China-based developers due to its unlimited free usage, fast local response times, and no proxy requirement.

0 comments Read more at DEV Community

ProgrammingDEV Community ·

AWS Lab: SQL Injection in Lambda Function Exploited to Escalate IAM Privileges

A security lab writeup details how a SQL injection flaw in an AWS Lambda function was exploited to escalate IAM privileges and retrieve a secret from AWS Secrets Manager. Using the Pacu framework, researchers enumerated IAM roles and identified a key invoker role they were permitted to assume via sts:AssumeRole. After assuming the role and enumerating Lambda functions in us-east-1, they downloaded the function's source code and discovered it concatenated user input directly into a SQLite query without sanitization. By injecting a payload that bypassed a public policy visibility check, the researchers tricked the function into attaching the privileged SecretAccessPolicy to their user. This granted them direct access to the target secret, demonstrating how unsanitized inputs in cloud functions can lead to full privilege escalation.

0 comments Read more at DEV Community