Why Your Apache Iceberg Tables Are Slow and How to Fix Them

Apache Iceberg has become the default table format across major platforms like Snowflake, Databricks, and AWS, but query performance often degrades over time in production environments. The root cause is physical table deterioration: small files from streaming writes, fragmented manifests, scattered data layouts, and accumulating delete files from CDC pipelines. These issues force query engines to scan five to ten times more data than necessary, driving up S3 API costs and slowing query planning from milliseconds to seconds. When a query runs against an Iceberg table, the engine performs a multi-level scan planning process — pruning manifests, skipping files via column statistics, and filtering Parquet row groups — whose effectiveness depends entirely on the table's physical state. Engineers can address this degradation either through continuous automated optimization via a control plane or through manual SQL-based maintenance routines, both targeting the same underlying physical layout levers.