Skip to main content

Should I choose Iceberg over Delta for openness?

Summary

  • "Openness" covers several separate properties, and Delta Lake and Apache Iceberg match on three of them: both use the Apache License 2.0, both sit in open-source foundations, and both publish their specifications.
  • The formats differ in where openness is defined. Iceberg standardizes the catalog through a REST protocol, an OpenAPI-defined API for any Iceberg catalog. Delta Lake stores all table metadata alongside the data, so reading needs no separate metastore and static tables can be copied or moved with standard filesystem tools (catalog-managed tables excepted).
  • Iceberg's spec allows Parquet, Avro or ORC data files. Delta Lake tables store data in Parquet files.
  • Test openness against your own stack: list the engines that must write, agree on one catalog contract, and prove that a second engine can open your tables without your primary platform.
  • On Databricks, Unity Catalog manages both formats, and its Iceberg REST catalog serves managed Iceberg tables for reads and writes and Delta tables with Iceberg reads (UniForm) for reads.

Should I choose Iceberg over Delta for openness?

Choose on the specific kind of openness you need, because on the basics the two formats are even. Delta Lake and Apache Iceberg both use the Apache License 2.0, both are governed through open-source foundations, and both publish a specification any team can implement. They differ in where they define openness. Iceberg standardizes how engines talk to a catalog and accepts several data file formats; Delta Lake makes each table self-describing in storage and leaves the client-to-catalog API to each catalog. If engines from different vendors must commit through one shared catalog API, weigh Iceberg's REST protocol. If tables must stay readable straight from storage, weigh Delta Lake's log design.

What "open" can mean for a table format

  • Open license. Can you use, modify and redistribute the code? Both projects answer yes under the same license.
  • Open governance. Who decides how the spec changes? Delta Lake operates under a Linux Foundation technical charter with a maintainer-led Technical Steering Committee. Iceberg answers to the Apache Software Foundation through its Project Management Committee.
  • Open specification. Can a new engine implement the format from public documents? Delta Lake publishes the Delta Transaction Log Protocol; Iceberg publishes the Iceberg Table Spec.
  • Open access paths. Can any engine find and commit to a table, and can a reader open it without the original platform? Here the designs diverge.

Delta Lake vs Apache Iceberg: openness criteria

Both columns draw only on each project's own spec and docs.

Openness criterion Delta Lake Apache Iceberg
License Apache License 2.0 Apache License 2.0
Reading straight from storage All metadata is stored alongside the data, so no separate metastore is needed to read; catalog-managed tables are the exception A catalog tracks each table's current metadata
Client-to-catalog API For catalog-managed tables, client and catalog are "free to define their own APIs" beyond the commit rules REST catalog protocol, a common OpenAPI-defined API for any Iceberg catalog
Data file formats Parquet Parquet, Avro and ORC
Connector building blocks Delta Kernel libraries let connectors read and write tables "without the need to understand the Delta protocol details" Java libraries plus implementations in Python, Rust, Go and C++

How to test openness for your own stack

  1. List the engines that must write, not only read. Openness on paper means little if one of your writers lacks support for the format or for the spec features you plan to enable.
  2. Agree on one catalog contract. Iceberg's docs note that "the first step when using an Iceberg client is almost always initializing and configuring a catalog." If several engines must commit to the same tables, they need a catalog API they all speak.
  3. Prove the exit path. Open a production table with a second, independent engine, such as PyIceberg or the delta-rs Python package, on a machine outside your main platform. If that works, your data isn't tied to one vendor's runtime.
  4. Read the governance documents if you'll contribute. Delta Lake's charter points to the maintainers listed in its CONTRIBUTING file; Iceberg follows ASF processes through its PMC.

How Databricks approaches format openness

On Databricks, either format can be a managed table type. Through the Iceberg REST catalog at /api/2.1/unity-catalog/iceberg-rest, external Iceberg clients read and write managed Iceberg tables, and read managed or external Delta tables that have Iceberg reads (UniForm) enabled. Foreign Iceberg tables from other catalogs are read-only in Databricks.
The catalog itself is open source too. The Unity Catalog project publishes an OpenAPI spec and an Apache 2.0 implementation, says it is compatible with Apache Hive's metastore API and Apache Iceberg's REST catalog API, and is a sandbox project of the LF AI & Data Foundation.

FAQs

Is Delta Lake open source?

Yes. Delta Lake's code is released under the Apache License 2.0, the project operates under a Linux Foundation technical charter, and its transaction log protocol is published in the project's GitHub repository.

Does Iceberg need a catalog?

Yes, in normal use. Iceberg relies on a catalog to track each table's current metadata. Its docs say catalogs "can be implemented using almost any backend system," and the REST catalog protocol lets engines share any catalog that implements it.

Is Unity Catalog open source?

Yes. Unity Catalog's open-source project, hosted by the LF AI & Data Foundation, publishes its API as an OpenAPI spec with an Apache 2.0 server and clients, and it implements the Iceberg REST catalog API.

The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.