platforms supporting Apache Iceberg

Summary

• Apache Iceberg is an open-source table format designed for analytics that works across Apache Spark, Apache Flink, and other query engines.
• Databricks actively advances Iceberg in the Apache community, driving innovations like v3 deletion vectors and Iceberg v4 metadata improvements.
• Databricks Lakehouse offers managed Iceberg tables via Unity Catalog with built-in governance, plus support for reading foreign Iceberg tables from external catalogs.
• Through the Iceberg REST Catalog API, external engines can access Databricks-managed Iceberg tables across different languages and platforms.
• Delta Lake tables can automatically generate Iceberg metadata via UniForm, allowing a single copy of data to serve both Delta and Iceberg clients.

platforms supporting Apache Iceberg
Apache Iceberg is an open-source table format built for analytics workloads. It provides isolation, versioning, and time-travel capabilities across multiple compute engines, making it a portable choice for teams using different tools. Today, Iceberg runs on Apache Spark, Apache Flink, and other query engines, each with varying levels of read and write support.

What is Apache Iceberg?

Apache Iceberg separates table metadata from data files. This design lets you do fast metadata operations like schema changes and snapshots without rewriting data. Iceberg tables track full history, supporting time-travel queries and rollback. The format works with Apache Parquet files and supports Iceberg spec versions 1 through 3. Because Iceberg is engine-agnostic and open-source, data teams can choose compute engines without vendor lock-in.

Key capabilities to evaluate

Capability	Why it matters
Multi-engine support	Run the same Iceberg tables on Apache Spark, Apache Flink, and other query engines; choose engines per workload rather than per table.
Time-travel and versioning	Query historical snapshots or roll back schema changes; audit data lineage.
Metadata efficiency	Update schemas, partition structures, or retention policies without rewriting data files.
Interoperability with existing tables	Read existing Delta or Parquet tables as Iceberg through adapters or format bridges.
Governance layer	Centralized access controls, lineage, and auditing across all engines accessing the tables.
REST Catalog API	External engines can discover and query tables via open HTTP endpoints.

How Databricks approaches Apache Iceberg

Databricks is a founding contributor to Apache Iceberg and actively leads community work on next-generation features. Databricks Lakehouse integrates Iceberg at multiple levels.
Managed Iceberg tables: Create and manage Iceberg tables directly in Unity Catalog using Databricks SQL, Spark, or external Iceberg-compatible engines. These tables automatically benefit from Predictive Optimization and Liquid Clustering, which apply layout improvements without user intervention.
Unity Catalog as governance hub: Unity Catalog serves as the central governance layer for Iceberg tables, whether managed by Databricks or external catalogs like AWS Glue or Hive metastore. Fine-grained access controls, audit logs, and lineage apply uniformly across engines.
Foreign Iceberg table support: Through Lakehouse Federation, access and govern Iceberg tables managed by external catalogs (AWS Glue, Hive metastore, and other catalog providers). Query foreign tables as if they were local; Unity Catalog handles the translation.
Iceberg REST Catalog API: Databricks exposes an open Iceberg REST Catalog API. Any Iceberg-compatible client can read and write Databricks-managed Iceberg tables using standard HTTP calls.
Delta-Iceberg interoperability (UniForm): For existing Delta Lake tables, enable Iceberg reads via UniForm. This feature automatically generates Iceberg metadata on the same Parquet files, letting Delta and Iceberg clients coexist on one copy of data without rewriting.
Iceberg v3 features: Databricks supports deletion vectors (physical row deletes without rewriting files), row-level lineage, and the VARIANT data type on both managed Iceberg and Delta tables.

Use cases

Multi-tool analytics: Teams using multiple query engines can share a single Iceberg table. One engine might handle bulk transforms while another serves interactive queries. Both read the same data without duplication.
Regulated data environments: Iceberg's versioning and time-travel support audit requirements. Combined with Unity Catalog's access controls, you can prove who accessed what data and when.
Data migration: Migrate from one engine to another without rewriting tables. Create Iceberg tables in the new engine and point it to the same data files.
Incremental analytics updates: Iceberg's schema evolution and snapshot isolation let you change table structure mid-pipeline without breaking downstream queries.

FAQs

Can I read existing Delta tables as Iceberg?

Yes, through UniForm. Enable Iceberg reads on a Delta table in Databricks, and the system automatically generates Iceberg metadata asynchronously. External Iceberg clients can then query the same Parquet files.

Does Iceberg lock me into Databricks?

No. Iceberg tables store metadata in a standard format (JSON and Parquet). You can export or migrate them to other Iceberg-compatible platforms or engines. Databricks does not require proprietary extensions to Iceberg.

How do I access Databricks Iceberg tables from outside Databricks?

Use the Iceberg REST Catalog API. Databricks exposes this endpoint; any Iceberg-compatible client can authenticate and query or write Iceberg tables using standard HTTP calls.

What if my external catalog already has Iceberg tables?

Use Lakehouse Federation. Register the external catalog (AWS Glue, Hive metastore, or other Iceberg-compatible catalogs) in Unity Catalog and query foreign Iceberg tables directly. Unity Catalog manages permissions and lineage across engines.

Is Iceberg slower than Delta for Databricks workloads?

No. Databricks applies the same performance optimizations (Predictive Optimization, Liquid Clustering) to Iceberg tables as Delta tables. Performance depends on data layout and compute, not the table format.