Unity Catalog vs DataHub

Summary

• Unity Catalog is Databricks' unified governance layer, managing data and AI assets within the lakehouse through automatic access control, lineage tracking, and audit logging.
• DataHub is an open-source metadata management platform, originally developed at LinkedIn, deployable as a self-hosted service or through Acryl DataHub Cloud, a managed SaaS offering.
• Unity Catalog governs Databricks-native assets including tables, volumes, functions, and ML models, with governance built into the same platform as compute and storage.
• DataHub uses an entity-aspect-relationship metadata model and exposes GraphQL and REST APIs for programmatic metadata access, with connectors to cloud data platforms, BI tools, and orchestration tools.
• Teams using Databricks as their primary platform rely on Unity Catalog for built-in governance; DataHub offers a standalone catalog layer for mixed-stack environments.

Unity Catalog vs DataHub

Unity Catalog is the governance layer built into Databricks, managing data and AI assets within the lakehouse. DataHub is an open-source metadata management platform designed to help organizations discover, govern, and understand their data assets. Unity Catalog governs workloads running on Databricks as an integrated capability; DataHub is a standalone catalog connectable to multiple data systems.

What is DataHub?

DataHub was originally developed at LinkedIn and is now an Apache 2.0-licensed open-source project hosted at github.com/datahub-project/datahub. It models metadata through three core primitives: entities (data assets like datasets or dashboards), aspects (collections of attributes describing an entity), and relationships (named connections between entities). Each entity is identified by a URN (Uniform Resource Name). DataHub supports self-hosted deployment via Docker or Kubernetes, or through Acryl DataHub Cloud, a managed SaaS offering with additional observability features. It exposes a GraphQL API, a REST API, a CLI, and a Python SDK (acryl-datahub).

Unity Catalog vs DataHub: how they compare

Capability	Unity Catalog	DataHub
Deployment	Built into Databricks; auto-enabled for new workspaces after Nov 8, 2023	Self-hosted (Docker, Kubernetes) or Acryl DataHub Cloud (managed SaaS)
License	Managed service; UC OSS under Apache 2.0	Apache 2.0 (OSS); Acryl DataHub Cloud is a commercial SaaS offering
Metadata model	Three-level namespace (catalog.schema.object) with typed asset categories	Entities, aspects, and relationships; extensible with custom entity types
Access control	SQL GRANT/REVOKE; row and column filters; attribute-based policies; workspace bindings	Role-based access control; policy definitions; personal access tokens
Data lineage	Automatic tracking across all Databricks workloads	Column-level lineage; upstream/downstream graph queries
AI asset governance	ML models, serving endpoints, functions, Unity Gateway traffic as catalog objects	Metadata management extensible to model assets
Ingestion	Native to Databricks workloads; external access via Iceberg REST catalog and Hive metastore APIs	Extensible ingestion framework with connectors to cloud data platforms, BI tools, and orchestration tools
API surface	OpenAPI REST spec; Hive metastore and Iceberg REST catalog compatibility	GraphQL API, REST API, Python SDK (acryl-datahub), CLI
Data quality	Built-in profiling and anomaly detection in managed Unity Catalog	Integrates with external quality tools; surfaces quality checks in the catalog

How Databricks approaches catalog governance

Unity Catalog is integrated into Databricks, not deployed as a separate service. It uses a three-level namespace (catalog.schema.object) for all data and AI assets, and enforces access control through SQL-standard GRANT and REVOKE statements with fine-grained extensions: row-level security, column masks, and attribute-based policies.
Lineage in Unity Catalog is automatic. When Databricks workloads run, the catalog records how data flows: which tables a job reads, which it writes, and what transformations are applied. This spans tables, models, and pipelines without manual instrumentation. The audit log records every access event in the system.access system table, queryable with SQL.
Unity Catalog governs AI assets natively. ML models, model serving endpoints, and AI functions are catalog objects with the same access control and lineage as data tables. Unity Gateway, Databricks' AI traffic governance layer, integrates with Unity Catalog for policy enforcement on model traffic.
For external engines accessing Databricks data, Unity Catalog exposes open APIs compatible with Apache Hive's metastore API and Apache Iceberg's REST catalog API. The open-source Unity Catalog (UC OSS), at github.com/unitycatalog/unitycatalog, provides the core catalog server under Apache 2.0 for teams running multi-engine environments.

FAQs

Does Unity Catalog replace DataHub?

Unity Catalog and DataHub serve different primary use cases. Unity Catalog governs data and AI assets within Databricks with enforcement automatically applied to all platform workloads. DataHub is a standalone catalog you deploy and connect to multiple data systems. Teams running mixed-stack environments may use both.

How does DataHub's metadata model differ from Unity Catalog's?

DataHub models metadata through entities (data assets), aspects (attribute collections), and relationships (connections between assets), with custom extension points via the typed metadata API. Unity Catalog uses a three-level namespace with first-class support for data and AI asset types including tables, models, and serving endpoints.

Is DataHub available as a managed service?

Yes. Beyond self-hosted deployment, Acryl DataHub Cloud provides DataHub as a managed SaaS offering with additional data quality monitoring and observability features.

What open APIs does Unity Catalog expose?

Unity Catalog's REST API is based on an OpenAPI specification. It also provides compatibility with Apache Hive's metastore API and Apache Iceberg's REST catalog API, enabling external compute engines to access Databricks-governed tables through standard protocols.