what is UniForm and how does it make Delta and Iceberg interoperable
Summary
- UniForm (Universal Format) is a Delta Lake feature that generates Apache Iceberg metadata alongside the Delta Lake metadata, so Iceberg clients can read a Delta table without rewriting its Parquet data files. Databricks docs call the capability Iceberg reads.
- It works because Delta Lake and Iceberg both consist of Parquet data files plus a metadata layer. UniForm generates the Iceberg layer asynchronously after Delta writes, and one copy of the data files serves both kinds of client.
- You turn it on with two table properties,
delta.enableIcebergCompatV2anddelta.universalFormat.enabledFormats, on a table that has column mapping enabled. On Databricks, the table must be registered to Unity Catalog. - Iceberg clients get read-only access. When Iceberg engines also need to write, use managed Iceberg tables in Unity Catalog, which support reads and writes through the Iceberg REST Catalog API.
what is UniForm and how does it make Delta and Iceberg interoperable
UniForm, short for Universal Format, makes a Delta Lake table readable as an Apache Iceberg table by writing Iceberg metadata next to the Delta metadata. It doesn't convert or copy data. Delta Lake and Iceberg both store data as Parquet files described by a metadata layer, so one set of Parquet files can carry two metadata layers: Delta clients read the Delta log, Iceberg clients read the Iceberg metadata, and both land on the same files. On Databricks, the docs call this capability Iceberg reads, and it's available in Databricks Runtime 14.3 LTS and above.
What is UniForm?
UniForm is part of open-source Delta Lake, whose documentation describes it as a way to read Delta tables with Iceberg clients, and it's built into Databricks. Enabling Iceberg reads on Databricks configures the table to use UniForm for its metadata layer. Writers keep writing Delta. After a write transaction completes, Databricks triggers Iceberg metadata generation asynchronously on the compute that wrote the data.
Two planning details: enabling Iceberg reads adds the IcebergCompatV2 write protocol feature, so only clients that support it can write to the table, which might affect compatibility with external Delta Lake clients. And the Iceberg view reflects the last completed metadata generation, which the converted_delta_version and converted_delta_timestamp fields record.
How do you enable UniForm on a Delta table?
Check the documented requirements and behaviors first.
| Requirement | What the Databricks docs specify |
|---|---|
| Catalog | The Delta table is registered to Unity Catalog. Managed and external tables are both supported. |
| Column mapping | Must be enabled. It's turned on automatically when you enable Iceberg reads at table creation. |
| Protocol versions | minReaderVersion of 2 or higher and minWriterVersion of 7 or higher. Protocol upgrades can't be undone. |
| Runtime | Writes use Databricks Runtime 14.3 LTS or above. Enabling on an existing table uses 15.4 LTS or above. |
| Deletion vectors | Not supported for Iceberg v2 reads (Iceberg v3 supports them). REORG purges them when you upgrade an existing table. |
| Compression | Parquet data files use the Zstandard codec. |
For an existing table, the docs give this statement:
ALTER TABLE table_name SET TBLPROPERTIES(
'delta.columnMapping.mode' = 'name',
'delta.enableIcebergCompatV2' = 'true',
'delta.universalFormat.enabledFormats' = 'iceberg');
For a new table, set the same two Iceberg properties in CREATE TABLE ... TBLPROPERTIES; there, Databricks recommends delta.columnMapping.mode = id. If the table has deletion vectors, run REORG TABLE table_name APPLY (UPGRADE UNIFORM(ICEBERG_COMPAT_VERSION=2));, which enables Iceberg reads and rewrites the underlying data files. Confirm the result with DESCRIBE EXTENDED or SHOW TBLPROPERTIES. To turn the feature off, unset delta.universalFormat.enabledFormats.
How Databricks approaches Delta and Iceberg interoperability
Unity Catalog governs both formats, so interoperability runs in both directions. Delta tables with Iceberg reads serve Iceberg clients. Iceberg tables you create in Unity Catalog are managed Iceberg tables that external engines can read and write, and Iceberg tables owned by another catalog are foreign tables that Databricks reads through Lakehouse Federation. Databricks supports versions 1, 2 and 3 of the Iceberg specification.
| Table type in Unity Catalog | Iceberg client reads | Iceberg client writes |
|---|---|---|
| Managed Delta table with Iceberg reads | Yes | No |
| External Delta table with Iceberg reads | Yes | No |
| Managed Iceberg table | Yes | Yes |
| Foreign Iceberg table | Yes | No |
External engines connect to the Iceberg REST Catalog endpoint at /api/2.1/unity-catalog/iceberg-rest. Enable External data access on the metastore, grant the configuring principal EXTERNAL USE SCHEMA on the schema, and, where the client supports it, use credential vending, which gives the engine temporary credentials for the underlying storage.
Governance carries over, with limits. External Iceberg clients can't access views, and reading tables with row filters or column masks from an external Iceberg client requires the documented cross-engine attribute-based access control (ABAC) setup. Pipeline-managed materialized views and streaming tables can't use IcebergCompatV2; they get external Iceberg access through IcebergCompatV3.
FAQs
Can Iceberg engines write to a UniForm table?
No. Iceberg client support for Delta tables with Iceberg reads is read-only. If Iceberg engines need to write, use a managed Iceberg table in Unity Catalog, which the Iceberg REST Catalog API supports for both reads and writes.
Does UniForm add overhead to writes?
Some. Metadata generation runs asynchronously on the writing compute, and the docs note it might increase driver resource usage. Data files aren't rewritten, and Delta Lake runs only one metadata generation process at a time on a given compute resource.
Can I enable UniForm on an existing table?
Yes, with ALTER TABLE on Databricks Runtime 15.4 LTS and above, or with REORG if it has deletion vectors. The first metadata generation must finish before external clients can query it as Iceberg; MSCK REPAIR TABLE <table-name> SYNC METADATA triggers generation manually.
Do Delta versions line up with Iceberg snapshots for time travel?
Not necessarily. Both formats support time travel by version or timestamp, but Delta table versions aren't guaranteed to align with Iceberg versions by commit timestamp or version ID. Use the converted_delta_version and converted_delta_timestamp fields to map one to the other.
The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.