Which platforms use the Delta Lake format?
Summary
- Delta Lake is supported across major cloud platforms including Databricks, Microsoft Fabric, Azure Synapse, Amazon Redshift, Google BigQuery, and Snowflake, with varying levels of read and write integration.
- Open-source engines like Apache Spark, Trino, Flink, and the delta-rs library enable Delta Lake adoption without committing to a single vendor or compute engine.
- Databricks treats Delta Lake as its default table format with deep governance via Unity Catalog, and Delta Lake UniForm extends cross-platform reach by generating Iceberg metadata without data duplication.
Which platforms use the Delta Lake format?
Delta Lake is an open-source storage format that brings ACID transactions, schema enforcement, and time travel to data stored on cloud object storage. As organizations adopt lakehouse architectures, Delta Lake has become one of the most widely supported open table formats across cloud platforms, query engines, and open-source tools. For a hands-on introduction, explore the Delta Lake tutorial to see these capabilities in action.
Understanding which platforms support Delta Lake, and how deeply, helps teams make informed decisions about data architecture and interoperability.
What is Delta Lake and why does it matter?
Delta Lake is an open-source storage framework that adds reliability and performance to data lakes. It stores data as Parquet files with a JSON-based transaction log that tracks every change. This design enables capabilities that plain Parquet cannot provide:
- ACID transactions on cloud object storage
- Schema enforcement and evolution to prevent bad data from entering tables
- Time travel for auditing, rollback, and reproducibility
- Unified batch and streaming support in a single table
These features let organizations manage transactions, control data versions, and interoperate across multiple engines, all on open storage.
Which cloud and data platforms support Delta Lake?
Delta Lake's ecosystem spans major cloud platforms, warehouses, and open-source engines. The depth of integration varies by platform.
| Platform | Delta Lake Support |
|---|---|
| Databricks | Default table format; deepest integration with governance via Unity Catalog |
| Microsoft Fabric | Default table format in Fabric Lakehouse |
| Azure Synapse Analytics | Reads and writes Delta Lake tables via Spark pools |
| Amazon Redshift | Reads Delta Lake tables via Redshift Spectrum |
| Google BigQuery | Reads Delta Lake via BigLake and external tables |
| Snowflake | Reads Delta Lake tables as external tables |
| Apache Spark (open source) | Full native read/write support |
| Trino | Read and write via Delta Lake connector |
| PrestoDB | Read support via Delta Lake connector |
| Apache Flink | Write support via Delta connector |
| Apache Hive | Read support via Delta connector |
Major clouds and warehouses are standardizing on open formats, signaling a broader shift to lakehouse-style architectures.
Using Delta Lake with open-source engines
Any Apache Spark deployment can use the open-source Delta Lake library for full read/write support. Beyond Spark, the ecosystem includes:
- Delta Standalone library: Enables Scala and Java-based projects, including Apache Flink, Apache Hive, Apache Beam, and PrestoDB, to read from and write to Delta Lake.
- delta-rs package: Provides Delta Lake support in Rust and Python, with integrations for pandas, Polars, Dask, Daft, and DuckDB.
- Trino and PrestoDB connectors: Require tables registered with a metastore such as Hive metastore or AWS Glue.
This broad compatibility means teams can adopt Delta Lake without committing to a single compute engine.
How Databricks treats Delta Lake as a first-class foundation
On Databricks, Delta Lake is the default format for all operations. Open table formats, Delta Lake, Apache Iceberg™, and Parquet, are first-class citizens, not bolt-ons. Governance, semantics, and lineage are built into the platform via Unity Catalog.
Unity Catalog provides one catalog for all data, managing these formats with a single set of permissions, lineage, and business definitions. Every user and system works from the same trusted source.
AI-powered optimizations like Photon, Predictive IO, and Intelligent Workload Management deliver warehouse-grade performance without the trade-offs of proprietary warehouses.
How Delta Lake UniForm expands cross-platform reach
Delta Lake UniForm lets you write data once and make it readable by engines that support other open table formats:
- Iceberg metadata generation: UniForm produces Iceberg metadata while maintaining a single copy of the underlying Parquet files.
- Cross-platform compatibility: Organizations use UniForm to serve Delta Lake data to Iceberg reader clients.
- Portability without duplication: Teams avoid vendor lock-in by keeping one physical copy of data accessible across multiple engines.
Organizations looking to share data across platforms and partners can also leverage Delta Sharing for secure, open data exchange.
Best practices for choosing Delta Lake integration depth
When evaluating Delta Lake across platforms, consider these factors:
- Read vs. read/write support, Some platforms only read Delta Lake; others support full mutations.
- Metastore requirements, Engines like Trino and PrestoDB need a compatible metastore for table discovery.
- Governance needs, Evaluate whether governance is built into the platform or requires external tooling.
- Format interoperability, If your stack includes Iceberg-native tools, UniForm can bridge the gap.
FAQs
What is the Delta Lake format and how does it work?
Delta Lake is an open-source table format that stores data as Parquet files with a JSON-based transaction log tracking every change. This provides ACID transactions, schema enforcement, and time travel on cloud object storage.
How does Databricks use Delta Lake as its default storage format?
All tables on Databricks are Delta Lake tables unless otherwise specified. Governance and lineage are built in via Unity Catalog, making open table formats first-class citizens.
Which cloud data platforms support reading Delta Lake tables?
Microsoft Fabric, Azure Synapse Analytics, Amazon Redshift (via Spectrum), Google BigQuery (via BigLake), and Snowflake all support reading Delta Lake tables alongside Databricks.
Can Delta Lake be used with Apache Spark outside of Databricks?
Yes. Any Spark deployment can use the open-source Delta Lake library for full read/write support without requiring Databricks.
How do you query Delta Lake tables from platforms like Trino or Presto?
Dedicated Delta connectors enable querying from Trino, PrestoDB, and similar engines. Tables must be registered with a metastore such as Hive metastore or AWS Glue.
Is Delta Lake supported in Microsoft Fabric and Azure Synapse Analytics?
Yes. Microsoft Fabric uses Delta Lake as its default table format for lakehouse workloads. Azure Synapse reads and writes Delta Lake tables through Spark pools.
What open-source tools and engines are compatible with the Delta Lake protocol?
Apache Spark, Flink, Hive, Beam, PrestoDB, and Trino all support Delta Lake. The delta-rs package extends support to Python and Rust ecosystems including pandas, Polars, and DuckDB.
How does Delta Lake Universal Format (UniForm) enable cross-platform compatibility?
UniForm generates Iceberg metadata alongside Delta Lake metadata so Iceberg-compatible engines can read the same data without duplication.
Which data lakehouse platforms have adopted Delta Lake as a supported table format?
Databricks and Microsoft Fabric use Delta Lake as their foundational format. Azure Synapse, Amazon Redshift, Google BigQuery, and Snowflake support reading Delta Lake data.
How do you set up Delta Lake integration with popular query engines and ETL tools?
Trino, Presto, and Athena support reading Delta Lake via manifest files or connectors paired with a compatible metastore. For Databricks users, Lakeflow provides unified batch and streaming pipelines with built-in governance.
Build your lakehouse on an open foundation
Delta Lake's broad ecosystem reduces the risk of lock-in to any single platform. The depth of integration varies, so teams should evaluate read/write support, governance, and format interoperability for their specific stack.
On Databricks, Delta Lake, Apache Iceberg™, and Parquet are governed through Unity Catalog with unified permissions, lineage, and business definitions, combined with performance features like Photon and Predictive IO for demanding data and AI workloads. Explore the Data Lakehouse to see how Databricks unifies these capabilities on an open foundation.
The information provided herein is for general informational purposes only and may not reflect the most current product capabilities or configurations.