Why Two Queries on the Same Table Can Return Different Row Counts
Two queries run against the same Iceberg table can return different row counts without either failing — one querying the catalog and another scanning storage directly returned results differing by over 211,000 rows. The discrepancy arises because Delta and Iceberg tables separate physical storage from logical table state, meaning a folder scan sees all Parquet files while a catalog query reads only those tied to a specific snapshot or version. Additional objects in storage may belong to older table versions, abandoned writes, or files superseded by delete metadata, none of which affect a committed snapshot's row count. To diagnose such differences, analysts should first resolve the exact Delta version or Iceberg snapshot used, follow the metadata chain to identify active data and delete files, and only then compare row counts. A query returning results successfully does not guarantee it read the intended table state or that all referenced files were accessible.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.
Discussion (0)
Log in to join the discussion and vote.
Log in