Is Databricks open source?

Summary

• No. The Databricks platform is a commercial managed service, not open source. The Databricks docs say Databricks Runtime "includes additional optimizations and proprietary features that build on and extend Apache Spark", including Photon.
• It is built on open source. Databricks was founded in 2013 by the original creators of Apache Spark, Delta Lake, MLflow and Unity Catalog, and its docs say it continues to develop and release features to Apache Spark.
• You can use those projects without Databricks. Spark, Delta Lake, MLflow, Unity Catalog OSS and Apache Iceberg all carry Apache licenses.
• Tables use open formats. Databricks supports Delta Lake and Apache Iceberg tables in Unity Catalog, and external Iceberg engines can reach Iceberg tables through the Iceberg REST Catalog API.

Is Databricks open source?

No. Databricks is a commercial platform, and the platform itself is not open source. It is built on open source projects, several of which were created by the people who founded Databricks, and you can run those projects without a Databricks account. The Databricks docs say Databricks Runtime "includes additional optimizations and proprietary features that build on and extend Apache Spark", with Photon as the example, and Databricks operates the platform's control plane for you.

Which parts of the Databricks ecosystem are open source?

Capability	Open-source project	On Databricks
Distributed compute	Apache Spark (Apache Software Foundation, Apache License 2.0)	Databricks Runtime includes Apache Spark and adds proprietary features such as Photon, which is compatible with Apache Spark APIs
Table format	Delta Lake (Linux Foundation, Apache-2.0) and Apache Iceberg (Apache-2.0)	Delta tables are the default; Iceberg tables created in Unity Catalog are managed Iceberg tables
Governance	Unity Catalog OSS (LF AI & Data sandbox project, Apache 2.0)	Unity Catalog is the governance layer built into Databricks, and is also available as an open-source implementation
Declarative pipelines	Spark Declarative Pipelines (Apache Spark 4.1.0)	Lakeflow Pipelines extend and are interoperable with Spark Declarative Pipelines
Orchestration	For example Apache Airflow	Lakeflow Jobs
ML lifecycle	MLflow (Linux Foundation, Apache-2.0)	MLflow on Databricks
AI agents	Omnigent (Apache 2.0)	A fully managed version of Omnigent

Omnigent is an open-source meta-harness for AI agents. Its README lists "Status: alpha", and the Databricks docs say "Omnigent is in Beta".

How do open formats and standards work on Databricks?

Tables created on Databricks are Delta tables by default. Iceberg tables you create in Unity Catalog are managed Iceberg tables, and all Iceberg tables in Unity Catalog can be accessed through the Iceberg REST Catalog API. Iceberg reads (UniForm) configure Delta Lake tables to generate Iceberg metadata, so Iceberg clients can read Delta Lake data without rewriting files. It is available in Databricks Runtime 14.3 LTS and above.
Unity Catalog OSS publishes an OpenAPI spec and a server implementation under Apache 2.0, and its README says it is compatible with Apache Iceberg's REST catalog API.
Open formats do not mean identical features. Every Delta table has a protocol that lists the features a client must support, and Databricks recommends testing compatibility before you enable new table features on production tables that open source Delta Lake clients also use.

How does Databricks run Apache Spark?

The Databricks docs say Apache Spark is "at the heart" of the platform and is the technology powering compute clusters and SQL warehouses. When you deploy a compute cluster or SQL warehouse, Spark is configured and deployed for you, and you do not need to initialize a Spark context or Spark session. Photon is "the Databricks-native vectorized query engine". For supported operations it replaces the JVM-based Spark SQL execution engine with a native C++ runtime.

What does Databricks contribute to open source?

The Databricks docs say the company "continues to develop and release features to Apache Spark". In June 2025 Databricks announced that it was donating its Declarative Pipelines framework to Apache Spark, and the Spark 4.1.0 release notes list it as "SPIP: Declarative Pipelines". Databricks open sourced Unity Catalog in 2024, hosted by LF AI & Data, and a 2025 Databricks blog post says it actively contributes features such as deletion vectors to Iceberg v3.

FAQs

Can I run Databricks on my own servers?

Databricks operates the control plane, which holds the web application and backend services, so you do not self-host it. Classic compute runs in your own cloud account, and serverless compute runs in a serverless compute plane in your Databricks account. The open source projects in the table run on infrastructure you choose.

Can I run Apache Spark without Databricks?

Yes. Spark runs on Windows and UNIX-like systems, locally with Java installed, and on a cluster under Standalone, Hadoop YARN or Kubernetes. The Databricks docs say you can also use Databricks without using Apache Spark.

What stays portable if I use Databricks?

Table data lives in cloud object storage, in Delta Lake or Apache Iceberg format. Classic workspaces use a workspace storage bucket in your own cloud account, and Unity Catalog external tables point at storage you control. Notebooks export as source files or .ipynb. The proprietary Databricks Runtime features, such as Photon, exist only on Databricks.

Does Databricks contribute to Apache Spark?

Yes. The Spark committers page lists Databricks as an organization for 44 of the 103 committers it names (counted in October 2026), and Spark Declarative Pipelines came from a Databricks donation.