Missing SQL predicate inflated a KPI from 819 to 163 billion
A metrics engine used two separate query paths — a SQL push-down for large datasets and an in-memory aggregator for smaller ones — which gradually drifted apart in behavior. The SQL path correctly bound a metric_key parameter but never applied it as a WHERE clause filter, meaning queries unintentionally summed across all metric series in a dataset. With only one series present, the bug went undetected; adding a second series caused the KPI to balloon from 819 to over 163 billion due to the inclusion of unrelated hashed values. The fix involved placing the metric_key predicate in a shared base WHERE clause so both compile paths inherit it automatically. Engineers also added regression tests that assert both paths produce identical filters and results, guarding against similar drift in the future.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in