What's a good open-source lakehouse stack for 2026?

Summary

• A lakehouse is, in the Databricks docs' words, "a data management system that combines the benefits of data lakes and data warehouses." In open source, that means files on object storage, a table format, a compute engine, a catalog and an orchestrator.
• A reasonable 2026 default: Apache Spark 4.x, Delta Lake or Apache Iceberg tables, a catalog that speaks the Iceberg REST or Hive metastore API (Unity Catalog OSS or Apache Polaris), Apache Airflow for orchestration, and dbt for SQL transformations and tests.
• Choose the table format by the engines you must support. Row-level changes do not decide it, because Delta Lake and Spark on Iceberg both support merge, update and delete.
• Pin versions. As of October 2026 the Delta Lake compatibility table pairs Delta Lake 4.3.x with Spark 4.0.x and 4.1.x, and Apache Spark 4.2.0 shipped in July 2026.
• On Databricks the same open pieces run as managed services, and Iceberg tables work through Unity Catalog.

What's a good open-source lakehouse stack for 2026?

A good default stack has six layers: object storage, an open table format (Delta Lake or Apache Iceberg), Apache Spark for compute, a catalog, Apache Airflow for orchestration, and dbt for SQL transformation.

Which layers does an open-source lakehouse need?

Layer	Role	Typical options
Storage	Holds the data files	Amazon S3, Google Cloud Storage, Azure storage or HDFS (per the Delta Lake storage docs)
Table format	Adds transactions and schema management	Delta Lake, Apache Iceberg
Compute	Runs queries and pipelines	Apache Spark, Apache Flink, DuckDB
Catalog	Tracks tables and exposes them to engines	Unity Catalog OSS, Apache Polaris, Hive metastore, any Iceberg REST catalog
Orchestration	Schedules and monitors workflows	Apache Airflow
Transformation	SQL models and data tests	dbt

Delta Lake or Apache Iceberg: how do I choose?

Both are open table formats and both work with Spark. The Delta Lake docs list ACID transactions with serializable isolation, schema enforcement, time travel, and merge, update and delete. The Iceberg docs list serializable isolation, rollback to earlier snapshots, hidden partitioning, partition evolution, and branches and tags. Spark also supports MERGE INTO, UPDATE and DELETE FROM on Iceberg tables, so row-level changes will not settle the choice. These criteria usually do:

Criterion	What to look for
Engines that must read the tables	Iceberg's multi-engine page documents Spark, Flink and Hive directly and lists other engines, DuckDB among them, as third-party integrations. The Delta docs name Spark, Flink and Hive among the engines with connectors. Confirm a connector exists for each engine you need.
Converting existing tables	Iceberg's migrate procedure replaces a table with an Iceberg table loaded with the source's data files. snapshot makes a light-weight temporary copy for testing.
Partition layout that changes	Iceberg documents hidden partitioning and partition evolution: "Misconfigured tables can be fixed without an expensive migration."
One table for both ecosystems	Delta UniForm lets Iceberg and Hudi clients read Delta tables.

Which compute engine should run the queries?

Start with Apache Spark, whose 4.2.0 release came out in July 2026. Its docs cover Structured Streaming, which can ensure end-to-end exactly-once semantics with replayable sources and idempotent sinks, and Spark Declarative Pipelines (SDP), "a declarative framework for building reliable, maintainable, and testable data pipelines on Apache Spark."
Add other engines where they fit. Iceberg supports both Flink's DataStream API and Table API, and DuckDB reads Iceberg and Delta tables through its iceberg and delta extensions. Pin versions deliberately: as of October 2026 the Delta Lake compatibility table pairs Delta Lake 4.3.x with Spark 4.0.x and 4.1.x only. Because spark.sql.extensions takes a comma-separated list of classes, one Spark session can register both formats:

from pyspark.sql import SparkSession

# Needs the delta-spark and iceberg-spark-runtime packages for your Spark version
spark = (
    SparkSession.builder
    .config("spark.sql.extensions",
            "io.delta.sql.DeltaSparkSessionExtension,"
            "org.apache.iceberg.spark.extensions.IcebergSparkSessionExtensions")
    .config("spark.sql.catalog.spark_catalog",
            "org.apache.spark.sql.delta.catalog.DeltaCatalog")
    .config("spark.sql.catalog.local", "org.apache.iceberg.spark.SparkCatalog")
    .config("spark.sql.catalog.local.type", "hadoop")
    .config("spark.sql.catalog.local.warehouse", "/tmp/warehouse")
    .getOrCreate()
)
spark.sql("CREATE TABLE default.events_delta (id BIGINT, data STRING) USING DELTA")
spark.sql("CREATE TABLE local.db.events_iceberg (id BIGINT, data STRING) USING iceberg")

Which catalog should I choose?

Pick a catalog that engines can reach over an open API. Iceberg defines "a REST-based Catalog API for managing table metadata and performing catalog operations," and lists this benefit: "New languages and engines can support any catalog with just one client implementation." Unity Catalog OSS (Apache 2.0) implements the Iceberg REST Catalog APIs and is compatible with Apache Hive's metastore API. Apache Polaris is "an open-source, fully-featured catalog for Apache Iceberg" that implements Iceberg's REST API. Iceberg's Spark configuration accepts catalog types including hive and rest.

What orchestrates and transforms the data?

Apache Airflow is "a platform created by the community to programmatically author, schedule and monitor workflows," with pipelines defined in Python. dbt handles transformation: "You write simple SQL select statements," and you can write data quality tests on your tables. dbt's fully open-source Apache 2.0 distribution is called dbt OSS.

How does Databricks fit into this stack?

Databricks runs the same open building blocks as managed services. Its docs describe Delta Lake as "an optimized storage layer that supports ACID transactions and schema enforcement" and Unity Catalog as "a unified, fine-grained governance solution for data and AI." Lakeflow Pipelines "extend and are interoperable with Spark Declarative Pipelines, while running on the performance-optimized Databricks Runtime," and Lakeflow Jobs provides orchestration (Airflow can also run Databricks jobs through the open source Databricks provider). For Iceberg, Unity Catalog manages lifecycle tasks such as snapshot expiration and file compaction on managed Iceberg tables, all Iceberg tables in Unity Catalog are accessible through the Iceberg REST Catalog API, and Iceberg reads (UniForm) let Iceberg clients read Delta Lake tables without rewriting files.

FAQs

Can I mix Delta and Iceberg tables in the same lakehouse?

Yes. Each table uses one format, and a single Spark session can load both extensions, as in the snippet above.

Where does DuckDB fit?

DuckDB reads Iceberg tables after INSTALL iceberg; LOAD iceberg;. Tables read directly from storage are read-only, and attaching an Iceberg REST catalog enables writing.