Iceberg versus Hudi versus Delta: the full comparison?

Summary

• Apache Iceberg, Apache Hudi and Delta Lake are open table formats that add transactions, snapshots and row-level changes to Parquet data in object storage. All three integrate with Apache Spark and Apache Flink.
• Their cores differ: Iceberg keeps table state in a tree of metadata files and manifests, Hudi records every action on a timeline, and Delta Lake commits to a transaction log at the table's location.
• Hudi organizes updates around record keys, indexes and two table types, Copy-on-Write and Merge-on-Read. Iceberg picks copy-on-write or merge-on-read per operation, and Delta Lake can mark deleted rows with deletion vectors.
• Each project ships its own incremental reads and maintenance: Iceberg procedures, Hudi table services, and Delta Lake's Change Data Feed, OPTIMIZE and VACUUM. Compare capability by capability against your workloads.

Iceberg versus Hudi versus Delta: the full comparison?

All three formats solve one problem, reliable tables on top of files, and they overlap on the basics: atomic commits, time travel, schema evolution and upserts. They differ in how table state is recorded, how updates are written and merged, how much indexing and maintenance each project provides, and how each reaches engines and catalogs. The projects' documentation describes trade-offs, not a ranking, so the full comparison goes capability by capability.

Three designs for the same problem

Apache Iceberg tracks individual data files instead of directories. A table metadata file holds the schema, partitioning and snapshots; each snapshot points to a manifest list, and manifests record data files with their partition data and column metrics. Every change writes a new metadata file and replaces the old one with an atomic swap, and a catalog tells engines which metadata file is current.
Apache Hudi records each change to table state, whether a write, a table service or a schema change, as an action on its timeline at a given instant. Data sits in file groups made of base files and, for Merge-on-Read tables, log files, and an index maps each record key to a file group. An internal metadata table, itself a Merge-on-Read Hudi table, serves file listings, column statistics and indexes.
Delta Lake writes each commit to a transaction log stored with the table, and by default the filesystem alone handles discovery and commit atomicity. Catalog-managed tables, available from Delta Lake 4.0.1, hand commit coordination to a catalog instead. Minimum and maximum values collected per column on write let readers skip files.

Capability by capability

Capability	Apache Iceberg	Apache Hudi	Delta Lake
Update model	Copy-on-write or merge-on-read per operation; delete files (v2) or deletion vectors (v3)	Copy-on-Write or Merge-on-Read table type; MoR merges log files at query time and during compaction	Rewrites affected files; deletion vectors, documented as experimental, can mark rows instead
Locating records	Partition and column-level stats in table metadata prune files	Indexes map record keys to file groups	Per-column minimum and maximum values drive data skipping
Time travel	VERSION AS OF and TIMESTAMP AS OF; branches and tags	TIMESTAMP AS OF time travel queries; savepoints	VERSION AS OF and TIMESTAMP AS OF
Incremental reads	create_changelog_view procedure; streaming reads of append snapshots	Incremental queries for latest state or CDC, for example through hudi_table_changes	Change Data Feed, in batch or streaming
Concurrency	Optimistic concurrency; writers retry on conflict	OCC between writers, MVCC between writers and table services, and non-blocking concurrency control	Optimistic concurrency control
Schema evolution	Add, drop, update or rename with no side effects	Backward-compatible changes on write; evolution on read is experimental	Enforcement on write; column mapping enables rename and drop
Data layout	Hidden partitioning and partition evolution	Optional partition path plus a clustering table service	Partitions or liquid clustering
Maintenance	expire_snapshots, rewrite_data_files and remove_orphan_files procedures	Compaction, clustering and cleaning table services	OPTIMIZE for compaction and VACUUM for unreferenced files
Streaming	Structured Streaming reads and writes	Structured Streaming writes; Hudi Streamer for ingestion	Streaming source and sink
Catalogs	Hive, REST and directory-based catalogs; the REST Catalog API	Table metadata can sync to a Hive Metastore	Path-based by default; catalog-managed tables

Where the formats meet

Delta Lake's UniForm documentation notes that the three formats "all consist of Parquet data files and a metadata layer." UniForm uses that to generate Iceberg metadata, and Hudi metadata in preview, so Iceberg or Hudi clients read the same files. Conversions exist: Delta Lake converts Parquet-based Iceberg tables in place with CONVERT TO DELTA (not those with partition evolution), and Iceberg's snapshot and migrate procedures bring existing Hive or Spark tables into Iceberg. On an existing Delta table (Delta Lake 3.3 and above, column mapping on; deletion-vector tables need REORG), enabling UniForm Iceberg is a property change:

ALTER TABLE sales SET TBLPROPERTIES (
  'delta.enableIcebergCompatV2' = 'true',
  'delta.universalFormat.enabledFormats' = 'iceberg');

How Databricks approaches open table formats

Databricks states its position in the managed tables documentation: Unity Catalog managed tables are the default and recommended table type for Delta Lake and Apache Iceberg, with interoperability for Delta Lake and Apache Iceberg clients. External Iceberg engines can read and write Unity Catalog's Iceberg tables through the Iceberg REST Catalog API, and Iceberg reads (UniForm) let Iceberg clients read Delta tables without rewriting files.

FAQs

Which format handles upserts best?

The documentation describes different mechanics, not a ranking: Hudi's record-key indexes and table types, Iceberg's per-operation copy-on-write or merge-on-read, and Delta Lake's MERGE with optional deletion vectors. Test with your own update pattern.

Can one copy of the data serve all three formats?

Through Delta Lake's UniForm, yes: a Delta table can carry Iceberg metadata and, in preview, Hudi metadata over a single set of Parquet files.

Are all three open source?

Yes. Apache Iceberg and Apache Hudi are Apache Software Foundation projects under the Apache License 2.0, and Delta Lake is an open source project.

Do all three work with Apache Spark and Apache Flink?

Yes. Each project documents both integrations.