How do I use Delta Lake and Iceberg together without copying data?

Summary

• Enable Iceberg reads (UniForm) on a Delta table in Unity Catalog and Iceberg clients can read it. The Databricks docs say "a single copy of the data files supports both Delta and Iceberg clients."
• Databricks generates the Iceberg metadata asynchronously after a Delta write completes, without rewriting the Parquet data files.
• Iceberg client support is read-only. Writes go through Delta Lake, or to a managed Iceberg table when an outside engine needs write access.
• Requirements: a Unity Catalog table with column mapping, Databricks Runtime 14.3 LTS or above for writes, and no deletion vectors.
• Check progress with DESCRIBE EXTENDED, and force a catch-up with MSCK REPAIR TABLE <table-name> SYNC METADATA.

How do I use Delta Lake and Iceberg together without copying data?

Enable Iceberg reads on the Delta table. The Databricks docs say Iceberg reads "configure Delta Lake tables to automatically generate Iceberg metadata, so Iceberg clients can read Delta Lake data without rewriting files." Your jobs keep writing Delta Lake, and "a single copy of the data files supports both Delta and Iceberg clients." Iceberg clients get read access only. Source: Read Delta Lake tables with Iceberg clients.

What are Iceberg reads (UniForm)?

Delta Lake and Apache Iceberg both consist of Parquet data files and a metadata layer. When you enable Iceberg reads, Databricks configures the table to use the Universal Format (UniForm) for the metadata layer. UniForm generates Iceberg metadata asynchronously alongside the Delta Lake metadata and leaves the Parquet files in place. These tables use Zstandard instead of Snappy as the Parquet compression codec. The open source project documents the same capability as UniForm, with Delta Lake 3.1 or above and a Hive metastore as the catalog.

How do I turn on Iceberg reads?

The table has to meet these requirements:

• It is registered in Unity Catalog. Managed and external tables are both supported.
• Column mapping is enabled.
• It has minReaderVersion >= 2 and minWriterVersion >= 7.
• Writes use Databricks Runtime 14.3 LTS or above.
• Deletion vectors are not enabled.

For a new table, the docs use id column mapping mode, which they recommend for compatibility:

CREATE TABLE T(c1 INT) TBLPROPERTIES(
  'delta.columnMapping.mode' = 'id',
  'delta.enableIcebergCompatV2' = 'true',
  'delta.universalFormat.enabledFormats' = 'iceberg');

For an existing table on Databricks Runtime 15.4 LTS and above, the docs use name mode (id mode can only be set when a table is created):

ALTER TABLE table_name SET TBLPROPERTIES(
  'delta.columnMapping.mode' = 'name',
  'delta.enableIcebergCompatV2' = 'true',
  'delta.universalFormat.enabledFormats' = 'iceberg');

If deletion vectors were enabled on the table, or you previously enabled the IcebergCompatV1 version of UniForm, run REORG TABLE table_name APPLY (UPGRADE UNIFORM(ICEBERG_COMPAT_VERSION=2)), which rewrites the underlying data files. To turn the feature off, unset the delta.universalFormat.enabledFormats table property.

How do Iceberg clients read the table?

Iceberg client support is read-only, so every write goes through Delta Lake. Clients reach the table through the Iceberg REST catalog in Unity Catalog, at the /api/2.1/unity-catalog/iceberg-rest endpoint. The docs name Apache Spark and Apache Flink as supported clients, and their configuration examples include Apache Spark and PyIceberg. Setup needs "External data access" enabled on the metastore, the EXTERNAL USE SCHEMA privilege on the schema, and a personal access token or OAuth. Source: Access Databricks tables from Apache Iceberg clients.

Table type	Iceberg client read	Iceberg client write
Managed Iceberg	Yes	Yes
Foreign Iceberg	Yes	No
Managed Delta (Iceberg reads enabled)	Yes	No
External Delta (Iceberg reads enabled)	Yes	No

How current is the Iceberg metadata?

Databricks triggers metadata generation asynchronously after a Delta write transaction completes. Tables with frequent commits can group several Delta commits into one Iceberg commit, and only one generation process runs on a given compute resource at a time. Access the table by name, not by path, so generation triggers automatically.
Run DESCRIBE EXTENDED catalog_name.schema_name.table_name and read the "Delta Uniform Iceberg" section: converted_delta_version is the latest Delta Lake version with Iceberg metadata, and converted_delta_timestamp is that commit's timestamp. To catch up on demand, run MSCK REPAIR TABLE <table-name> SYNC METADATA, which runs synchronously.

How do Delta Lake and Apache Iceberg differ?

Both are open table formats. Delta Lake extends Parquet data files with a file-based transaction log. Iceberg supports schema evolution, time travel and hidden partitioning.

Aspect	Delta Lake	Apache Iceberg
Metadata	JSON log entries in the _delta_log directory, plus checkpoints that shorten log replay	A table metadata file tracks schema, partitioning, properties and snapshots, and each snapshot's manifests are stored in a manifest list
Partitioning	Liquid clustering replaces table partitioning and ZORDER on Databricks	Hidden partitioning: partition values come from transforming a column, with no user-maintained partition columns
Changing the layout	Clustering keys can be redefined without rewriting existing data	Partition evolution is a metadata operation, and old data keeps its earlier spec
Column changes	Column mapping enables metadata-only renames and drops	Unique column IDs; add, drop, rename, widen and reorder are metadata changes

FAQs

Can Iceberg clients write to a Delta table with Iceberg reads enabled?

No. The docs state that Iceberg client support is read-only and writes aren't supported. Write through Delta Lake, or use a managed Iceberg table when an outside engine needs write access.

Does enabling Iceberg reads copy my data?

No. A single copy of the data files supports both Delta and Iceberg clients, and the Iceberg metadata is generated without rewriting the Parquet files. The exception is the REORG TABLE ... APPLY (UPGRADE UNIFORM(...)) command, which rewrites data files.