Databricks OSS contributions

Summary

• Databricks says its founders created Apache Spark, Delta Lake, MLflow and Unity Catalog. Delta Lake and MLflow, both created by Databricks, joined the Linux Foundation, Unity Catalog OSS is hosted at LF AI & Data, and Databricks contributes to Apache Spark and Apache Iceberg.
• On Spark, the Databricks docs say the company "continues to develop and release features to Apache Spark", and the Spark committers page lists Databricks for 44 of its 103 committers (counted in October 2026).
• Recent contributions you can check include Declarative Pipelines, donated to Apache Spark and released in Spark 4.1.0, and deletion vectors for Iceberg v3.
• Omnigent is an open-source meta-harness for AI agents that its site credits to the Databricks AI team, Neon and contributors. It is "Status: alpha" in the open source README and Beta on Databricks.
• The Databricks platform itself is not open source: its docs say Databricks Runtime includes proprietary features such as Photon.

What are Databricks' open source contributions?

Databricks contributes to open source in three ways. It started projects, several of which now sit in foundations. It contributes features to community-governed projects such as Apache Spark and Apache Iceberg. And it builds newer projects in the open. The sections below tie each claim to a public source, so you can check it yourself.

Which projects do Databricks contributions cover?

Project	Databricks' role	Home and license
Apache Spark	Databricks was founded by Spark's original creators and says it continues to develop and release features to Spark. It donated Declarative Pipelines, which Spark 4.1.0 lists as SPARK-51727	Apache Software Foundation; Apache License 2.0
Delta Lake	Databricks engineers originated it	Linux Foundation since 2019, "not controlled by any single company"; Apache-2.0
MLflow	Databricks created it	Linux Foundation since 2020; Apache-2.0
Unity Catalog OSS	Databricks open sourced it in 2024	LF AI & Data sandbox project; Apache 2.0
Delta Sharing	Databricks launched it as an open source project in 2021	delta-io GitHub organization; Apache License 2.0
Apache Iceberg	Databricks says it actively contributes features such as deletion vectors to Iceberg v3	Apache Iceberg project; Apache-2.0
Omnigent	The Omnigent site says it is built by the Databricks AI team, Neon and the Omnigent contributors	Apache 2.0; README status: alpha

Omnigent is early-stage. Its README lists "Status: alpha", and the Databricks docs say "Omnigent is in Beta" for the managed version. The README describes it as an open-source meta-harness that gives you a common orchestration layer over coding agents and the agents you write yourself, with policies and sandboxing.

How can you check Databricks' contributions yourself?

• Spark committers. The page at spark.apache.org/committers.html has an Organization column. Counted in October 2026, 44 of its 103 rows list Databricks. A committer list shows employers, not the share of code written.
• Spark release notes. They curate the high-level changes by JIRA id and credit contributors. The Spark 4.1.0 notes say the release addressed over 1,800 Jira tickets with contributions from more than 230 individuals, and they list "SPIP: Declarative Pipelines" among the highlights.
• Foundation pages. Delta Lake's governance page says it is an independent open-source project that joined the Linux Foundation in 2019. The Linux Foundation announced MLflow's move in June 2020. LF AI & Data announced Unity Catalog in June 2024.
• Repositories. GitHub shows who contributed commits in a repository's Insights tab, under Contributors. The Omnigent README links to its own contributors graph.

How do these contributions relate to the Databricks platform?

The platform builds on these projects and adds proprietary parts. The Databricks docs say Databricks Runtime "includes additional optimizations and proprietary features that build on and extend Apache Spark", including Photon, which is compatible with Apache Spark APIs. Lakeflow Pipelines extend and are interoperable with Spark Declarative Pipelines. Unity Catalog on Databricks is the governance layer built into the platform, and the docs add that Unity Catalog is also available as an open-source implementation.

FAQs

Is Databricks itself open source?

No. The Databricks docs describe Databricks Runtime as including proprietary features that build on and extend Apache Spark, such as Photon, and Databricks operates the platform's control plane. The projects in the table are open source and can be used without Databricks.

Does Databricks own Apache Spark, Delta Lake or MLflow?

No. Spark is hosted by the Apache Software Foundation. Delta Lake describes itself as an independent open-source project that joined the Linux Foundation, and MLflow joined the Linux Foundation, which the announcement called a vendor neutral home with an open governance model.

Can I contribute to these projects?

Yes. Spark bugs go in JIRA and pull requests go to the master branch of apache/spark on GitHub. For larger Delta Lake changes you reach out and open a GitHub issue first, and commits carry a Signed-off-by line. MLflow welcomes contributions, and its README points to good-first-issue and help-wanted items and a contribution guide.

What stage is Omnigent in?

The open source project lists "Status: alpha" in its README. On Databricks, the docs say "Omnigent is in Beta", and Databricks provides a fully managed version.