Apache Iceberg v4 Spec Drops Manifest Lists for a Root Manifest Design
A DEV Community author, an AWS Community Builder, reviewed the Apache Iceberg v4 specification and proposals and concluded that v4 focuses more on formalizing work currently handled through operations than on adding new features. Under the draft design, the per-snapshot manifest list is removed and replaced by a single root manifest, so small changes can be recorded directly without rewriting large metadata files. Data files and their deletion vectors are stored in the same entry, which the spec says removes the need for two-phase planning. The spec also bans writing new equality deletes and stores manifests in Parquet rather than Avro. The author notes that v4 is still under development and not formally adopted, with only some items such as relative paths and content stats already adopted, and the final spec may change.
This is an AI-generated summary. ShortSingh links to the original source for the complete article.


Discussion (0)
Log in to join the discussion and vote.
Log in