Is Delta open source, or Databricks lock-in?

Summary

• Delta Lake is open source: Apache 2.0 code, an independent sub-project of the Linux Foundation Projects, and a published transaction log protocol. Using the format doesn't tie a table to Databricks.
• Lock-in risk sits around the files, not in them: which table features a table enables, which catalog holds its permissions, and which platform-only services your pipelines call.
• You can open the same table without Databricks, using open-source Apache Spark with the delta-spark package or the deltalake Python package from delta-rs, a native Rust library.
• Before enabling a table feature, confirm every engine that reads the table supports it. Delta Lake 4.0.0 and above can remove droppable features with ALTER TABLE ... DROP FEATURE.
• On Databricks, Unity Catalog serves tables to external engines through the Unity REST API and the Iceberg REST catalog, and Iceberg reads (UniForm) let Iceberg clients read Delta tables.

Is Delta open source, or Databricks lock-in?

Delta Lake is open source, and the format itself doesn't lock you in. The code is Apache 2.0 licensed, the project is an independent sub-project of the Linux Foundation Projects, and the transaction log protocol is published for anyone to implement. A Delta table is Parquet data files plus JSON log entries in a _delta_log directory, so open-source Spark, Rust and Python libraries, and other engines with Delta connectors can read it. Where lock-in shows up, it comes from choices made around the table: table features your other engines don't support, a catalog that's the only route to permissions, and pipeline code written against platform-only services. You can check each of those.

Where lock-in can and can't happen

Layer	What's open	What to check
Data files	Parquet data files and a JSON transaction log, with all table metadata stored alongside the data	That the files sit in storage your organization controls
Protocol	A public specification; "Delta Lake features are always backwards compatible"	Readers must support every reader feature a table enables
Table features	Deletion vectors, liquid clustering, row tracking, type widening and variant ship in open-source releases	Whether each engine you use supports the features you turn on
Readers and writers	delta-spark, Delta Kernel (Java and Rust), delta-rs, and connectors for Trino, Presto, Flink and Hive	Whether an engine can write as well as read, if it needs to
Catalog	Unity Catalog OSS (Apache 2.0, with an OpenAPI specification) and Iceberg REST catalogs	How access policies reach a second engine, since the Delta log records table metadata, not permissions
Iceberg clients	Delta Universal Format (UniForm), read-only for Iceberg and Hudi clients	It doesn't work on tables with deletion vectors enabled
Platform services	None are needed to read the data	Photon, Predictive Optimization, automatic liquid clustering and Lakeflow Pipelines are Databricks features

Test the exit path: read a Delta table without Databricks

Run this test early, against a copy of a real table, from an environment with no Databricks software. With open-source Apache Spark and the delta-spark package:

# pip install delta-spark
import pyspark
from delta import configure_spark_with_delta_pip

builder = (
    pyspark.sql.SparkSession.builder.appName("delta-exit-test")
    .config("spark.sql.extensions", "io.delta.sql.DeltaSparkSessionExtension")
    .config("spark.sql.catalog.spark_catalog",
            "org.apache.spark.sql.delta.catalog.DeltaCatalog")
)
spark = configure_spark_with_delta_pip(builder).getOrCreate()

df = spark.read.format("delta").load("/path/to/delta-table")
detail = spark.sql("DESCRIBE DETAIL '/path/to/delta-table'")
detail.select("minReaderVersion", "minWriterVersion").show()

Or with delta-rs, "a native Rust library for Delta Lake, with bindings into Python":

# pip install deltalake pandas pyarrow
from deltalake import DeltaTable

dt = DeltaTable("/path/to/delta-table")
print(dt.version())   # current table version
print(dt.protocol())  # reader and writer protocol versions
df = dt.to_pandas()

Cloud paths need storage credentials in both cases; delta-rs takes them through the storage_options argument. If a read fails because an engine doesn't support one of the table's features, you've found the real constraint before anything depends on it.

Before you enable a new table feature

• Forward compatibility can break. The Delta docs say "enabling some features breaks forward compatibility with workloads running in a lower Delta Lake version."
• Readers must keep up. To read a table, an implementation "must implement and respect all features listed in readerFeatures," according to the protocol. Databricks documents, for example, that liquid clustering tables use writer version 7 and reader version 3.
• Know the way back. Delta Lake 4.0.0 and above support ALTER TABLE <table-name> DROP FEATURE <feature-name> for droppable features such as deletionVectors, typeWidening, v2Checkpoint and columnMapping. Databricks notes that not all table features can be dropped once enabled.

How Databricks approaches Delta Lake portability

Databricks documents Delta Lake as open source software and says it "originally developed the Delta Lake protocol and continues to actively contribute to the open source project." On the platform, Unity Catalog is the access layer for outside engines. Delta clients read tables through the Unity REST API, Iceberg clients use the Iceberg REST catalog, and credential vending lets an external client inherit the privileges Unity Catalog grants. Iceberg reads (UniForm), available in Databricks Runtime 14.3 LTS and above, generate Iceberg metadata for Delta tables so Iceberg clients can read the same files without rewriting them. Photon, Predictive Optimization and automatic liquid clustering work on those Delta tables, and nobody reading the tables elsewhere needs them.

FAQs

Can I leave Databricks and keep my Delta tables?

Yes. The tables are Parquet files plus a Delta transaction log, and open-source engines read them. Plan for two things: confirm your next engine supports the table's features, and recreate access policies in your next catalog.

Can Iceberg engines read my Delta tables?

Yes, through Delta Universal Format (UniForm) in open-source Delta Lake or Iceberg reads (UniForm) on Databricks. Iceberg client support is read-only in both. Open-source UniForm doesn't work on tables with deletion vectors enabled, and on Databricks deletion vectors need Iceberg v3.

Does Unity Catalog lock tables to Databricks?

Unity Catalog exposes tables through documented interfaces, the Unity REST API for Delta clients and an Iceberg REST catalog for Iceberg clients. Unity Catalog is also available as Apache 2.0 open-source software, an LF AI & Data Sandbox project.