How do I avoid vendor lock-in with a lakehouse?

Summary

• Lock-in can sit in any lakehouse layer: table format, storage, catalog, engines, pipeline code, sharing and ML assets. Check each one, because open files alone don't make a platform portable.
• Store tables in an open format (Delta Lake, Apache Iceberg or Apache Hudi) in object storage your own cloud account controls, so more than one engine can work on them.
• Put a catalog with an open API in front of the tables. Iceberg defines a REST-based Catalog API with a published OpenAPI specification, and Unity Catalog OSS is Apache 2.0 licensed with an OpenAPI specification, server and clients.
• Write transformations against open-source APIs such as Spark DataFrames, Spark SQL and Spark Declarative Pipelines, track models with MLflow, and keep platform-only features at the edges of your design.
• Test the exit path early: read a production table from a second engine, such as PyIceberg, through the catalog with the same identity and permissions.

How do I avoid vendor lock-in with a lakehouse?

Make every layer of the lakehouse replaceable on its own. Keep data in an open table format in object storage you own, register it in a catalog that exposes an open API, let more than one engine work on the same tables, and write pipeline logic against open-source APIs. Lock-in collects wherever one of those layers is proprietary, even when the files underneath are plain Parquet.

Where does lock-in happen in a lakehouse?

Layer	What to check	Open option to look for
Table format	Can other engines read and write the files without conversion?	Delta Lake (a sub-project of Linux Foundation Projects), Apache Iceberg or Apache Hudi
Storage	Do files sit in a bucket your account owns, with metadata next to the data?	Your own cloud object storage; the Delta protocol stores all table metadata alongside the data
Catalog	Can any engine discover tables and get scoped access through a documented API?	Iceberg REST Catalog API (OpenAPI spec); Unity Catalog OSS (Apache 2.0)
Engines	Can several engines work on the same tables at once?	Iceberg's docs name Spark, Trino, Flink, Presto, Hive and Impala; Delta Kernel (Java and Rust) serves connector builders
Pipeline code	Is transformation logic written against open APIs?	Apache Spark DataFrame and SQL APIs; Spark Declarative Pipelines in SQL and Python
Sharing	Can partners read data without adopting your vendor?	The open Delta Sharing protocol, an open standard for secure data sharing
ML assets	Are experiments and models portable?	MLflow (Apache 2.0, a Series of LF Projects, LLC)

What is a practical checklist?

1. Pick the format by write coverage. List every engine that must write, not only read, and confirm a documented writer for each. When one table needs readers in two formats, metadata-translation features help, but check their direction: Delta Lake's Universal Format is read-only from the Iceberg and Hudi side.
2. Own the storage. Keep tables in buckets in your cloud account and know how to reach the files without the platform. Because the Delta log is self-describing, a table can be read without a separate metastore, and a static table can be copied with standard filesystem tools.
3. Use an open catalog API with vended credentials. A catalog that vends credentials lets an external client inherit the privileges the catalog governs.
4. Keep platform-only features at the edges. Use them where they save work, but keep table contents and core transformations open.
5. Run an exit drill. Read a production table from a second engine through the catalog. With PyIceberg against any Iceberg REST catalog:

from pyiceberg.catalog import load_catalog

catalog = load_catalog(
    "lakehouse",
    **{
        "uri": "https://<catalog-host>/<iceberg-rest-path>",
        "warehouse": "<warehouse-or-catalog-name>",
        "token": "<token>",
    },
)
table = catalog.load_table("<schema>.<table>")
print(table.scan(limit=10).to_pandas())

How does Databricks approach an open lakehouse?

Unity Catalog manages Delta Lake and Iceberg tables, and Databricks can also read foreign Iceberg tables from catalogs outside Unity Catalog. Its Iceberg REST catalog lets supported clients such as Apache Spark, Apache Flink and Trino read from and write to Unity Catalog-registered Iceberg tables, with credential vending for some reader clients. For the exit drill above, the endpoint is https://<workspace-url>/api/2.1/unity-catalog/iceberg-rest.
Writes through that endpoint apply to managed Iceberg tables. Foreign Iceberg tables and Delta tables with Iceberg reads are read-only there. Iceberg reads (UniForm) configure Delta Lake tables to generate Iceberg metadata automatically, so Iceberg clients can read them without rewriting files. Delta clients use the Unity REST API, and creating or writing to managed tables from Delta clients is in Public Preview.
Two more pieces cover the code and sharing layers. Lakeflow Pipelines extend and are interoperable with open-source Spark Declarative Pipelines, and OpenSharing (formerly Delta Sharing) recipients can use open-source connectors for pandas and Spark without being Databricks customers. Unity Catalog itself is also available as an open-source implementation.

FAQs

Are open table formats enough to avoid lock-in?

No. They make the files portable, but the catalog, access policies and pipeline code can still tie you to one product. Check each for an open API or a documented export path.

Do I have to choose between Delta Lake and Apache Iceberg?

Not necessarily. Delta Lake's Universal Format (UniForm) lets Iceberg clients read Delta tables (the open-source docs also list Hudi, in preview), and Unity Catalog manages both formats side by side.

How do I share data without locking partners into my platform?

Use an open sharing protocol. The open Delta Sharing protocol is an open standard for secure data sharing. On Databricks, OpenSharing is described in the docs as an open protocol developed by Databricks, and recipients can use it whether or not they run Databricks.

What's the simplest portability test?

Read a production table, and write to a test table, from a second engine through the catalog, with the same identity and permissions. If both work without copying data, the table, catalog and access layers are portable.