What does Databricks contribute to open source?

Summary

• Databricks states that it was founded in 2013 by the original creators of Apache Spark, Delta Lake, MLflow and Unity Catalog. Spark began as a UC Berkeley research project and has been hosted by the Apache Software Foundation since 2013.
• Delta Lake and MLflow each joined the Linux Foundation, and Unity Catalog OSS is hosted by LF AI & Data, so each of these projects has a foundation home outside Databricks.
• Databricks also contributes to Apache Spark and Apache Iceberg. It donated its declarative pipelines framework to Spark, where it ships as Spark Declarative Pipelines in Spark 4.1.0, and it says it actively contributes features such as deletion vectors to Iceberg v3.
• Every project on this page carries an Apache license and can run without a Databricks account. Databricks-only components, such as the Photon query engine, are added on top.

What does Databricks contribute to open source?

Databricks contributes in three ways. It started projects such as Delta Lake, MLflow, Unity Catalog OSS and Delta Sharing. It contributes to community-governed projects such as Apache Spark and Apache Iceberg. And it donates technology to community-governed homes. Databricks says its founders are the original creators of Apache Spark, Delta Lake, MLflow and Unity Catalog.

Which open source projects did Databricks or its founders create?

Project	What it is	Home and license
Apache Spark	A multi-language engine for data engineering, data science and machine learning on single-node machines or clusters	Started at UC Berkeley AMPLab in 2009; Apache Software Foundation since 2013; Apache License 2.0
Delta Lake	An open-source storage framework for building a lakehouse architecture with compute engines including Spark, Flink and Hive	Joined the Linux Foundation in 2019; Apache-2.0
MLflow	An open source AI engineering platform for agents, LLMs, and ML models	Joined the Linux Foundation in 2020; Apache-2.0
Unity Catalog OSS	An open source catalog for data and AI, with an OpenAPI spec, a server and clients	Hosted by LF AI & Data as a sandbox project; Apache 2.0
Delta Sharing	An open protocol for secure real-time exchange of large datasets	Repository in the delta-io GitHub organization; Apache License 2.0
Omnigent	An open-source meta-harness that orchestrates coding agents	Credited to the Databricks AI team, Neon and community contributors; Apache 2.0

Omnigent is early-stage software. Its README lists "Status: alpha", and the Databricks docs say "Omnigent is in Beta" for the managed version on Databricks.

How does Databricks contribute to Apache Spark and Apache Iceberg?

Apache Spark. The committers page on spark.apache.org names Databricks as an organization for 44 of the 103 committers it lists (counted in October 2026). In June 2025 Databricks announced that it was donating its Declarative Pipelines framework to Apache Spark. The Spark 4.1.0 release notes list "SPIP: Declarative Pipelines, a new component to define and run data pipelines" (SPARK-51727), and the Spark docs describe the result as Spark Declarative Pipelines (SDP).
Apache Iceberg. In June 2024 Databricks announced an agreement to acquire a company founded by the original creators of Apache Iceberg, and said it intends to work with the Delta Lake and Iceberg communities on format compatibility. A 2025 Databricks blog post says "we actively contribute features such as deletion vectors to Iceberg v3."

How does the Databricks platform relate to these projects?

Databricks builds on these projects and adds components that exist only on Databricks. Its docs say Databricks Runtime "includes Apache Spark but also adds a number of components and updates" for usability, performance and security. Photon is the Databricks-native vectorized query engine. For supported operations it replaces the JVM-based Spark SQL execution engine with a native C++ runtime, and it is compatible with Apache Spark APIs.
Unity Catalog on Databricks is the governance layer built into the platform, and the docs add that Unity Catalog is also available as an open-source implementation. Databricks now documents its sharing feature as OpenSharing (formerly Delta Sharing), while the open Delta Sharing protocol keeps its name.
Databricks also supports Apache Iceberg. Iceberg tables you create in Unity Catalog are managed Iceberg tables, and external Iceberg engines can reach all Iceberg tables in Unity Catalog through the Iceberg REST Catalog API. Iceberg reads (UniForm) configure Delta Lake tables to generate Iceberg metadata, so Iceberg clients can read them without rewriting files. It is available in Databricks Runtime 14.3 LTS and above.

How can you contribute to these projects?

Project	Where to start
Apache Spark	Report bugs in JIRA. Open pull requests against the master branch of apache/spark on GitHub, with a title of the form [SPARK-xxxx][COMPONENT] Title. Discuss a large change on dev@spark.apache.org first
Delta Lake	Open a GitHub issue. For major features (more than 100 lines of code or user-facing changes), reach out first. Small patches and bug fixes need no prior discussion. Add a Signed-off-by line to commits
MLflow	Open a GitHub issue, or pick up good-first-issue and help-wanted items. The README links a contribution guide (CONTRIBUTING.md)
Apache Iceberg	Iceberg tracks issues in GitHub and prefers contributions as pull requests. Community discussion happens mainly on the dev mailing list

FAQs

Is Apache Spark owned by Databricks?

No. Spark is hosted by the Apache Software Foundation, a 501(c)(3) nonprofit, and the Databricks Spark page calls the foundation "vendor-independent".

Can I use these projects without Databricks?

Yes. Each project named here carries an Apache license. The MLflow README says it can be used in "your local environment, on-premises clusters, cloud platforms, and managed services", and Delta Lake works with compute engines including Spark, Flink and Hive.

Does Databricks support both Delta Lake and Apache Iceberg?

Yes. Tables created on Databricks are Delta tables by default, Iceberg tables created in Unity Catalog are managed Iceberg tables, and Iceberg reads let Iceberg clients read Delta Lake tables.

Where is the code?

The repositories are github.com/apache/spark, github.com/delta-io/delta, github.com/mlflow/mlflow, github.com/unitycatalog/unitycatalog, github.com/delta-io/delta-sharing, github.com/apache/iceberg and github.com/omnigent-ai/omnigent.