How do I convert a Parquet table to Delta or Iceberg?
Summary
- Delta Lake and Apache Iceberg can both take over an existing Parquet table in place: each writes its own metadata over the Parquet files you already have, so no data is copied.
CONVERT TO DELTAlists the files in the table directory, writes a_delta_logtransaction log and infers the schema from Parquet footers. Partitioned data needs its partition columns declared.- Iceberg's
snapshotprocedure builds a test table that leaves the source untouched;migratereplaces the source table in the catalog with an Iceberg table that reuses its files. - Stop writers before converting in place. Files added outside the new format are invisible to it, and Delta's VACUUM can delete them.
- On Databricks,
CONVERT TO DELTAalso accepts a Unity Catalog table name for external Parquet tables.
How do I convert a Parquet table to Delta or Iceberg?
Convert it in place. Delta Lake's CONVERT TO DELTA command and Iceberg's migrate procedure both add table metadata on top of the existing Parquet files instead of rewriting them, so either conversion is a metadata job. Pick the target by which engines will write the table afterward, pause every writer while the conversion runs, and rehearse first: Iceberg's snapshot procedure and Delta's shallow clone each give you a side-by-side copy that leaves the original alone.
Converting a Parquet table to Delta Lake
In open-source Delta Lake, CONVERT TO DELTA runs against the table's directory. By default it collects per-file statistics (minimum and maximum values per column); NO STATISTICS skips that. Partitioned data needs the partition columns and types. The session needs Delta's SQL extension (io.delta.sql.DeltaSparkSessionExtension) and catalog (org.apache.spark.sql.delta.catalog.DeltaCatalog).
CONVERT TO DELTA parquet.`<path-to-table>`;
CONVERT TO DELTA parquet.`<path-to-table>` NO STATISTICS PARTITIONED BY (part int, part2 int);
CREATE OR REPLACE TABLE <target_table_name> SHALLOW CLONE parquet.`/path/to/data`;
The first statement converts an unpartitioned table, the second a partitioned one without statistics. Python and Scala offer the same conversion as DeltaTable.convertToDelta. Afterward the Delta log is the source of truth: any file Delta doesn't track "is invisible and can be deleted when you run vacuum", so all later writes must go through Delta Lake. The third statement is the option when the Parquet table must stay in service. Delta Lake 2.3 and above can shallow clone Parquet data into a new Delta table, which references the source files, so they have to stay in place.
Converting a Parquet table to Apache Iceberg
Iceberg's in-place path is two Spark procedures, snapshot and migrate, which "help test and migrate existing Hive or Spark tables to Iceberg". They need the iceberg-spark-runtime jar and Iceberg's SQL extensions, which add CALL. For a table in Spark's built-in catalog, the Iceberg docs wrap that catalog with SparkSessionCatalog:
from pyspark.sql import SparkSession
spark = (SparkSession.builder
.config("spark.sql.extensions",
"org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions")
.config("spark.sql.catalog.spark_catalog",
"org.apache.iceberg.spark.SparkSessionCatalog")
.config("spark.sql.catalog.spark_catalog.type", "hive")
.getOrCreate())
spark.sql("CALL catalog_name.system.snapshot('db.sample', 'db.snap')")
spark.sql("CALL catalog_name.system.migrate('db.sample')")
Replace catalog_name with the Iceberg catalog you call the procedures through. snapshot creates db.snap, an Iceberg table that reads the original data files while writing its own changes to a separate location. Because it doesn't own those files, Iceberg blocks expire_snapshots on it, and a DELETE against the original table can remove files it needs. Drop it after testing.
migrate is the cutover. It copies the source's schema, partitioning, properties and location into an Iceberg table with the same identifier and keeps the original as a backup named table_BACKUP_ unless you set drop_backup. It locks and drops the source table from the catalog, so stop writers first. It fails if any partition uses a format other than Avro, Parquet or ORC, or if the table is bucketed.
| Option | When to use it | Trade-off |
|---|---|---|
CONVERT TO DELTA |
Moving the table to Delta Lake for good | No data copy; writers must pause |
| Delta shallow clone of Parquet | A Delta copy while the Parquet table stays live | Depends on the source files staying put |
Iceberg snapshot |
Testing Iceberg without touching the source | No expire_snapshots; source deletes can break it |
Iceberg migrate |
Cutting the table over to Iceberg | Drops the source entry; keeps a backup by default |
| CREATE TABLE AS SELECT | A fully isolated copy or a new layout | Copies every data file |
How Databricks approaches Parquet conversion
On Databricks, Unity Catalog supports CONVERT TO DELTA for Parquet and Iceberg tables in external locations it manages, and the command takes a table name as well as a path: CONVERT TO DELTA catalog_name.database_name.table_name, with PARTITIONED BY for partitioned tables. Passing a name updates the metastore entry to show the table is now Delta. The result is a Unity Catalog external table; for a legacy Hive metastore managed Parquet table, Databricks documents CREATE TABLE AS SELECT into a managed table instead.
Two related options fill the gaps. Incremental clone for Parquet and Iceberg (Public Preview) keeps a Delta target in sync with a source that still receives appends, updates and deletes, one way from source to target. For Iceberg readers, enable Iceberg reads (UniForm) on the converted Delta table, or create a managed Iceberg table with CREATE TABLE ... USING ICEBERG on Databricks Runtime 16.4 LTS and above, which writes new data files.
FAQs
Does converting a Parquet table rewrite the data?
No. CONVERT TO DELTA writes a transaction log over the existing files, and Iceberg's migrate and snapshot add those files to Iceberg metadata. A CREATE TABLE AS SELECT copy, by contrast, writes new data files.
What about files written during the conversion?
Avoid them. Delta's docs say not to update or append data files while converting. Iceberg's migration guide notes that concurrent writers can leave files outside the new table, and uses its Add Files action to bring them in.
Can I undo a conversion?
For Delta, the docs describe returning to plain Parquet: if you ran operations that changed data files, such as DELETE or MERGE, run VACUUM with zero retention, then delete _delta_log. Iceberg's migrate keeps the original table as a backup unless drop_backup is true.
The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.